OSDev.org

@Octacone..

I'm in a bit of a hurry, so don't have time to quote properly, also let me know if I missed anything..

Wrt to the "why 5 and not 3", not sure what you meant by that..? If you are referring to the "5" in the "100x5B", then that 5 is picked up from your earlier post (32-27), so I was just continuing with your example and pointing out that wasting 5B per allocation is irrelevant if there's only ever going to be a 100 such allocations (ie. 500B). Also it's irrelevant if there's 10k such allocations on modern x86 systems (though embedded may of course be different). So I was trying to point out that while you want to be frugal with most of your memory, in some cases (especially wrt to the kernel) it really doesn't matter.

Whether you want to optimize for cache lines is of course up to you, but only supporting a couple of allocation sizes for the kernel malloc (kmalloc) makes things simpler on that side. If your userland malloc crashes it's easier to diagnose, if your kmalloc crashes it's more difficult, so keeping kmalloc simple has benefits, keeping malloc simple isn't really feasible..

Didn't really get your 7247 => 200+B, where's the 200 come from? But regardless, if there's only handful of such structs allocated then 200B wasted on each is again meaningless.



Wrt to 33 -> 64 rounding, I was just giving some examples, the exact rounding you want to do is up to you. There's two things to consider, one is having a relatively low number of allocation sizes to keep the allocator simple and the other is for cache line optimizations. You can ignore the cache line optimizations for now if you like, but keeping things simple does help. So 33 could also be rounded to 36 or 48, etc.. Once you know all the structs you need you can decide how you want to round them, for starters you could just give each 4KiB to get the ball rolling and fix those once you have all of your structs ready, since you probably don't know all the sizes yet..

The basic idea is that when you call kmalloc:
void* ptr = kmalloc(32); // ok
void* ptr = kmalloc(33); // kpanic, invalid allocation size
void* ptr = kmalloc(64); // ok
etc..

So then inside kmalloc you just check the requested size, and based on it you have different page for each size from which you allocate, that way you don't have to deal with odd sizes, etc. The idea here is to keep it as simple as possible. So if you only needed three different allocation sizes, say 32B, 64B and 256B you might have:
void* free32BList;
void* free64BList;
void* free256BList;
And then just keep track of those, but if this complicates things for you, you can certainly do it anyway you like =)

Note also that I would not use identity mapping at all, what benefit do you think you get from it?

Finally, the userland malloc is not something that comes with the OS at all, it's part of the language runtime, yes, you'll have to create it if you want to compile C apps for your OS, but it's not part of the OS and you should _NOT_ want to enforce any malloc on anyone.

Even if you did enforce your malloc on your userland, you couldn't. Consider the following code on _any_ OS:



The point here is that you can't enforce your malloc nor should you even attempt. AFAIK Windows apps that use malloc, the malloc internally uses VirtualAlloc (part of Win32 API) or something similar.

So the idea is that the language runtime malloc uses the OS's VMM to allocate what it wants. And you can offload the burden of figuring out what they want to userland, which makes simpler/stabler kernel, leaves more room for userland optimizations, etc.

So when userland calls:
VMM_Alloc(/*start*/ 0x10000, /*stop*/ 0x20000);

All your kernel VMM has to do is check that the userland is not requesting allocation on top of kernel memory. You may also want to check it's not doing something stupid, like requesting allocation on top of something that's already allocated to it (or you may want to allow it as a quick way of allowing to change the allocation type (read-only vs read-write vs execute, etc)). My VMM_Alloc example above didn't include "type", you may want to do that though.

So all the VMM_Alloc needs to do is basic sanity and security checking and then just do what it was asked to do, which makes its implementation quite simple.

Then as a starting malloc (for userland) you can do something simple (though inefficient) and just always round up the requested allocation to the nearest 4KiB and allocate that (yeah, I said it's inefficient). And couple of days later fix it when it becomes a problem.

So all four pieces (PMM, VMM, kmalloc and malloc) become really simple, and only reason to change them is to improve efficiency, but that's just optimization.

I need the way of knowing if the specific page table exists without causing a page fault.

You could probably "format" the whole memory with paging when your system starts, creating paging structures in a contiguous area after the kernel with a size appropriate for the amount of RAM actually installed. Then you could probably map all pages initially to the kernel. After that you could probably reassign any pages to other address spaces.

But if you always keep the paging structures of all processes somehow in the address space of the memory manager, by the initial memory formatting, then you probably wouldn't cause page faults.

Or you could probably handle the page fault using the faulting address in CR2 and correct. Anyway the page fault is supposed to tell you when a page isn't found so you make it available whenever possible, whenever it isn't because of an actual bug in a program.

Hey, I tried this:


and got this: (I might be a sketchy developer, but look at this abstract art)


Do you think there is a way of dong it where I could use those free bits as a reference for something?

Edit:
I found a way to do it:


But the problem is:
I can map everything just fine before I enable paging and all the mapped address work, but when I try to map something after enabling paging I keep getting a page fault. Is this supposed to happen? How do I determine which of the "callers/files" issued a read action?

For me, I divide allocation into 2 parts. First, I allocate page to the caller process. Each process has some page directories (so the unit is 2M); and I map the page to the address space of the process. Second, I allocate memory in that page.
Pseudo-code:

That's for requests with size < 2M. For that with size>= 2M, I allocate pages directly.
All the sizes is aligned.

@LtG

So here's pseudo-code: For the request with size<0x200000

Size >=0x200000: Free pages directly.
I use Accessed bit to manage pages, and use memory blocks to manage the memory in each page
. My kernel uses this way, and so far no fault has been found.