OSDev.org

I think people are mixing up handling the virtual address space with the physical. The use of recursive mapping is not related to physical memory management. The unity mapping of linear to physical memory is a transient state in the boot process. This mapping should be discarded as soon as the kernel is initialized.

Physical memory might be handled by mapping all physical memory in the virtual address space, but I think this is a security risk (what is not mapped cannot be corrupted). For protected mode, mapping physical memory in the virtual address space means you cannot use more than 4G of physical memory, which is a pretty poor design. I think using bitmaps for physical memory management is the best solution both for protected mode and long mode. Translating between virtual and physical addresses can be done effectively even without mapping all physical memory in the linear address space, it's just a bit more complicated.

Oh thank for the answers, now i understand what i need to do.

Hi! So i returned to my old code, and after adding splash image as C array into the kernel, my paging code has broken, my question is how i can correctly map whole kernel in the VMM, because my realization didn't work anymore.
Because i use the BrokenThorn tutorial as the tutorial for VMM in the init function i just maps the BIOS arena and the 3gb arena and didn't map the kernel, can i use the kernel start and kernel end address then recursivly maps it into kernel?
I just tested my theory and it didn't work, so how i can correctly map the kernel in virtual address space?
EDIT: How I can correctly map large amount of memory that doesn't fit into one page table? Okay I also get triple fault if i load module with size > = 2MB.

Use more than one page table.

Okay another question, how I can integrate VMM code into method that allocated specific amount of pages, like if returned address is not mapped?

I don't 100% understand what you mean, but if you're talking about any function that needs to allocate memory on the kernel heap, those functions should never even bother with stuff like pages, or be expected to return unmapped memory addresses, perhaps except special values like NULL.

Such functions should use the kernel heap manager. Heap manager manages a heap (duh), which will contain a number of pages. When asked to allocate memory, heap manager will check if there is enough free space on the heap; if there is, it will allocate from there. If not, it needs to expand the heap somehow. How you do that depends on you, but here is a reasonable solution: First ask the PMM to allocate a page, then map that page into designated heap addresses via the VMM.

Heap manager will also need to free pages that are unused. This usually means unmapping the page, and giving it back to PMM, so it can be allocated for something else.

Once you have the system in place, you should be able to have most of your kernel allocate memory using the heap manager. Of course, you're likely to encounter special cases where your heap manager isn't adequate or desirable. Such code may use the PMM/VMM directly, or have a different dedicated heap manager, separate from the main one. However, most of your kernel will use it, so you should make the main heap manager first

I think the kernel both needs a byte-aligned allocator (the heap) and a page-aligned. Page-based allocations can set things like physical adress of page frame, access rights and similar, while memory from the heap allocator cannot be manipulated by page-based operations.

The heap typically use links to implement the allocator, while the page allocator will search page tables for free entries.

In my design, I generally do not allow byte-aligned allocations to flat addresses, rather these needs to be mapped to selectors. The links are too fragile and are easily corrupted. I separate the kernel address space into heap-area (rather small) and page-based allocation area (most of kernel space).

I also have a "block" allocator that subdivides pages into smaller entries. These is used in device schedules, and are good when a lot of same-size memory blocks are needed. The block allocator is not a general allocator, rather is setup by a device (like an USB device), and then memory within it is allocated. The block allocator keeps track of the physical address, and so makes it faster & easier to convert between linear & physical addresses without a need to map all physical memory in the linear address space. The block allocator makes garbage collection easier since all allocations can be freed in a single call by the device. Of course, the block allocator also localize memory corruptions.

Okay so i just added test kernel heap implementation. And got the same problem that i have before, when the heap is trying to extend himself(like when i allocate 120MB of memory) the pmml_alloc(used for mapping) returns physically valid but not mapped address. So i here to ask did i need map full memory to correctly use kernel heap, if no can you say how i can correctly fix the problem with mapping? My kernel heap implementation use physical page allocator to extend himself.
Also what I should use for non kernel heap memory management? Can I use my PMM that I use previous to extend the kernel heap?

Why does your kernel need to allocate so much memory? That seems like too much.


You need to map memory before you can access it. Where do you want to map your kernel heap? Map your newly allocated memory there.


User mode applications will manage their own heaps using a system call to request more memory. That system call will use your VMM to map memory, and your VMM will use your PMM to allocate memory.

You can manage kernel memory in the same way as you manage user memory.

You have memory segments (not x86 segments, but base/size tuples that define a virtual address range.)

You have segments for kernel text and data, then a segment for the heap, as well as any segments to describe further memory mappings in the kernel virtual address space.

Then all you need is some code to map a page fault address to a segment in the kernel address space, then add segment code to translate a segment offset to a physical page, then you just need code to map a virtual address to a physical address.

Once you have that in place, your kernel then uses the same virtual address resolution that your user space will use. All pretty much general purpose.

I have special kernel heap segment drivers, that work without using the heap (can't have the heap address space dependent on the heap itself), but all the other kernel virtual address segments work the same for kernel and user address spaces.

Early in my boot process, I reserve address space for the heap using the heap segment driver, but without allocating any physical memory to the underlying mappings. Physical pages are then mapped on demand as my heap grows, up to the heap limit, using the same page fault handling code. Of course, you have to be really careful when you run out of memory, as anything that allocates heap memory when handling out of memory situations can result in death spiral with recursive failing page faults.

It's also a bit fiddly to bootstrap, as the underlying early stages of virtual memory management needs to work without access to heap memory.

The PMM cannot be used to implement the kernel heap. The kernel heap is dependent on virtual (mapped) memory. The PMM comes in either automatically (you access part of the heap, and the pagefault handler maps it to physical memory), or by directly mapping all of the heap. The latter will consume lots of physical memory for no valid reason.

I think that's quite reasonable. My reply was intended to simply give a rudimentary example of how it could work, not to explain a full design in detail