PatSanf wrote:
I have a doubly linked list. Each node on the list is defined as a data structure, and has a pointer to a place in memory along with a value for the size of the memory pointed to by the node. When I initialize the memory manager I create the initial node on the list. The node's address will be the first byte of free memory after the kernel, the memory pointer will be the first address after the node data structure, and the size will represent all of the free memory. That gets put on a list of free memory areas. When I allocate some memory, I split the first node on the list that's large enough for the allocation into a properly sized node with a header, and a node for whatever's left over. I then remove the node being allocated from list of free memory nodes, and return the pointer to the beginning of the memory allocation. When I free memory I get the pointer that was returned, back up to the beginning of the header, put the node back on the linked list, and compact adjacent nodes.
This sounds actually a lot like my first physical memory manager approach I wrote for the previous version of my kernel
This can be optimized though by not storing the address of the free region in its header, but instead just calculating (address_of_header + sizeof(header)) which gives you the address of the free memory behind.
Anyway, you can't use this for paging.
PatSanf wrote:
I guess this would work if I was wanting to do a simple segmented memory manager, but I'm wanting to do paging.
Xactly.
PatSanf wrote:
Since I'm going to have to redo the memory manager, would it be worth the extra effort to get things set up in the higher half before I get into that again? How will that affect writing to the VGA portion of memory? Will I need to successfully get paging going before the VGA addresses will be properly mapped, or am I worrying about nothing? Will doing things in the higher half require me to do the paging in ASM, before the kernel and vga stuff come into memory, or will I be able to easily do it just in C?
Whether you're doing higher half or not, I'd highly recommend you to implement paging as fast as possible because it will make things a lot easier.
PatSanf wrote:
Maybe I'm not understanding where I'm supposed to store the data structures. I'm concerned that every time I allocate memory, I'll need to store the information in the data structure somewhere, and this storage requirement may be unpredictable. This is what led me to keep the data structure with the memory segments. I'd never need to worry about reserving a large enough segment to store headers because the headers would stay with the segments. I know that paging does provide the option to use a set amount of memory to store meta data about the memory being used. Do you know of a "for dummies" kind of guide to how the paging data structures are implemented and used?
There's no "for dummies", because what you want to achieve is the first step to actually being able to call your software an "OS", and not just a booting "Hello world" binary. But don't worry, at one point it will *click* and you've got it.
PatSanf wrote:
I think that's one of the big things I don't understand. How will software use the virtual addresses? When a piece use user software allocates memory, how does it then make use of it? Will the software need to calculate the proper segment:offset stuff to get the proper physical address, do I need to write functions to handle that on demand, or will it automatically be done by the MMU? If I enable paging, what will happen with all of the stuff I've already written which relies on using specific physical addresses? Like, the VGA driver to write text to the screen, or my keyboard driver using a specific area of memory for an input buffer. Will I need to rewrite those to use paged memory?
Forget the segment:offset stuff nao (until you use vm86) because you do (i hope you do) protected mode, and theres no such thing.
Okay, I'll try to explain it to you very easy.
This will not be "the only way", or "the best way", but one way that actually works.
The first thing is, as you said: you'll never know how much memory you need to manage the memory. That is why the physical memory manager must be very simple. Lets make a stack allocator: the entire physical address space 0xFFFF FFFF consists of 1024 * 1024 pages of a size of 0x1000. To store the info for possibly each of these pages, you will at maximum need 1024 * 1024 pointers, each pointing to the start of a page: therefore you have 1024 * 1024 * 4 bytes, thats 4 MiB.
So first you find a space where the stack for your pages will be; then you read the multiboot memory map that GRUB provides you and add the address of EACH free page to that stack. This will be very fast, because pushing = set address and increment pointer; popping = decrement pointer and get address.
Now we have the physical memory manager ready. So, now when it comes to paging: Once you have enabled paging, the physical address space is
no more accessible, EXCEPT the parts that you have mapped. That means, if you switch to a directory that contains only zeros, the full address space from 0x00000000 - 0xFFFFFFFF will exist, but you cant access ANY of this memory.
If you do now map a page into the directory, so that for example the page at 0xA0000000 is available, then 0xA0000000 - 0xA0001000 is the only area in the address space that can actually be written to. Now you will wonder: how can the CPU still access the code of my kernel if it is not available in the address space anymore? For this, it is very usual to simply identity-map the area where your kernel binary lays (lets say its 0x100000 - 0x102000). It is also very common to just identity map the entire lower megabyte into the first address space, so it's still accessible. Why identity-map? Because your binary must be available at where its loaded (if your code is not compiled as position independent).
If you have now set it up this way, you can enable paging, and your kernel will be happily running just where it is and you will be able to access the lower memory. But everything else in memory is no more accessible! Now you might wonder, "How can I then still edit the page directory and its tables??
" - For this, see the page
about recursive mapping.Once you have come this far, I bet you'll have a better understanding how paging works
From there on it shouldn't be too far anymore: if you want to have multiple processes, then each process gets its own page directory. Be careful to keep your kernel mapped inside the process directory; otherwise an interrupt will cause a double fault because the IDT is no more accessible, which will lead to a triple fault because the double fault handler couldn't be called either.
For the process part: Due to the fact that you made a page directory for each process, each process has its own "virtual address space" for his own. The kernel is responsible for managing the address space of each process. What does it mean that each process has its own space? Lets say you have "Process A" and "Process B". Each has its own directory. Now we map a physical page (1) into the directory of Process A, lets say so that it covers adress 0xDEADF000. Now we map a
different physical page (2) at the
same location in the page directory of Process B. On task switch, you will now always switch page directories: when Process A has its turn, the page directory of Process A is loaded as the current directory. When Process A now writes to the virtual address 0xDEADF000, he will write to the physical page 1. Now the timer interrupt comes, and we switch to Process B; also we also load the other page directory. Process B has it's turn, and if he now writes to the address 0xDEADF000, he will write to the physical page 2 instead!
If you have now understood paging, you will still have to figure out some things yourself. Like, "Where do I map the stack of the PMM to make it available after enabling paging?" But I'll keep that to you and your fantasy.
Hope this helps