Octacone wrote:
Thank you both guys for further explanation. I think I know what it going on!
So:
Page directory is fake, it doesn't even exist. It is just a creation of our imagination, an abstraction. What I really want to put inside CR3 is an address of all the physical page directory entries! Then each one of those page directory entries contain an address of its virtual representative we use for translating. I think that was the key part missing in this entire story. Now I just need a strategy for actually managing the virtual memory. What are the functions I need, standard ones perhaps? Something like allocate block/s free blocks/s but with ability to set the flags and a virtual address itself (the one we want). Okay let's put that aside. Now the real question is, are all the structures already pre-allocated since there is a finite number of entries (cause compiler knows what to reserve), that is the data section right? So that means I only need to allocate virtual page tables, right? This is so exciting I think I am slowly putting everything in perspective.
Few thoughts:
- The Page Directory is an array that contains 1024 entries, that collection of 1024 entries is what we call a PD, each entry is called PDE. The address of the PD is the same as the address of the first PDE, though the first PDE (as all the PDE's) contains a physical address that points to the respective PT.
- So both the PD and PT are simple 4 byte arrays of 1024 elements or entries each. They work like arrays in the C language.
- Consider creating a PMM (Physical Memory Manager) that only manages physical memory and only gives it or receives it from the VMM (Virtual Memory Manager). This makes the PMM very simple. The PMM when receiving a page frame (of physical memory during deallocation; or initialization) records that somewhere (stack, linked list, bitmap, etc.) so that later when a page frame is requested it can allocate one and return it to the VMM.
- Remember that the PMM (and VMM, like any other code) is running in virtual memory, so the container (stack, linked list, bitmap, etc) is addressed with virtual memory, not physical, but the data you store in that container is physical memory addresses (page frames).
- The VMM is simple if you make it simple. When app requests memory from VMM you do only basic sanity check, like app is not allowed to request virtual addresses that belong to kernel, you may also want to check that the requested address is not already mapped to the app (though later for performance reasons you may want to allow apps to arbitrarily request remapping of their memory).
- After that the VMM marks the page(s) in the current processes PD (and associated PDE(s), PT(s) and PTE(s)) as allocated, but _NOT_ present. You can simplify this too by only allocating one 4KiB page at a time, later for better performance you can allow arbitrary sizes to be mapped.
- When the app attempts to use the address it just requested to be mapped it will cause a #PF (Page Fault exception) because we the relevant PTE as not present. In the #PF handler you check the address that caused the fault, go thru the tables and see that the address has been _allocated_ but is not present, so now you request a physical page frame from the PMM and map the relevant entry to point to that physical page frame you got from the PMM and mark it as present and then return from the exception handler.
- That last part can also be done in the VMM when doing the allocation, but that means all memory that has been requested is also allocated which means more physical memory is used when it might not all be necessary, this is usually called "demand allocation". Which way you want to do it is up to you..
- This leaves only the app side remaining, in C you usually use malloc and C++ "new" operator. You will need to implement those yourself or use some pre-existing portable library.
- Normally you might say malloc(100) which would allocate 100 bytes, but the OS can only give out 4KiB (or more, but with 4KiB granularity at the minimum) at a time, so usually malloc checks if it has already requested memory and has some available (100+ bytes in this case) and if so, returns that, the OS/kernel is not involved. If malloc does not itself have it, then it will allocate as much as is needed and then it does have it, and thus the previous sentence now applies. Or it fails (because OS/kernel refused) and then you go into error handling mode.
- Initially you can create extremely simple malloc that always requests 4KiB for all allocations and doesn't hold any itself, this is less efficient but easier, you can later improve it. This is the way I would do it, so you get the whole memory system working. Then when the bad performance, both higher memory consumption as well as higher runtime overhead, becomes a problem you can optimize it. At that point you have something you can actually measure and see what optimizations make sense. And at that point you may want to use a pre-existing library and put in the effort it takes to port it to your system, instead of creating your own. Malloc itself can be somewhat complex and time consuming task and unless you have some novel ideas or are particularly interested on that specific topic I see no good reason why you need to create your own proper malloc implementation.
Hopefully this will clarify the situation further. So you have PMM, VMM and then "something" in userland, from kernel perspective that "something" is irrelevant and it doesn't care. The common API between userland and kernel is either brk/sbrk or some form of mmap. Brk/sbrk is slightly simpler, it just moves a "high-water mark", which means VMM would simply mark the relevant pages as allocated/not-allocated based on what was previously allocated and what was requested. Mmap is superior and the way forward, brk/sbrk isn't sufficiently simpler that I would recommend it, only reason to even consider brk/sbrk is if you need interoperability with POSIX/Linux or some other legacy system. So go for mmap, ignore brk/sbrk.
Login
Register
FAQ
Search
