Hi guys, Thanks for your comments.
davmac314 wrote:
I think the one thing I'd consider doing differently is to not have the distinction between physical and virtual memory, insist that all buckets are physically contiguous (as well as virtually contiguous), and have the user (caller) do the virtual -> physical address conversion if necessary.
If I understand you correctly, the heap would allocate contiguous pages of physical memory, all identity mapped to virtual memory addresses, and visa-versa. If this is what you mean, then there is no need for paging, correct? Or did I understand that incorrectly?
Though not added in the code, in the specification (which is what my much-advanced heap code does), I add the Physical distinction, the ability to allocate physical contiguous memory because this is used by the kernel. The kernel needs to be able to allocate physical memory for hardware drivers. In a user privileged heap, the ability is disabled by the kernel, with a simple mask of the flags, not allowing physical allocation.
My memory structure has three layers. The physical layer, the virtual layer, and the heap layer. All allocation goes to the heap. If it can satisfy the request, it does so and returns, never calling the underlining layer. If it cannot, it calls the virtual layer to extend the heap, which in turn calls the physical layer.
My kernel supports 32 Gig of RAM. In fact, the way it is coded, my kernel thinks there is 32 gig of RAM at all times, period, even if there is considerably less. The virtual layer supports this 32 Gig independent of how much is actually physically installed. The kernel's heap, as well as a user heap, can request memory as it pleases. The kernel has the 32-gig limit, while a user heap has a much smaller limit, though it can be lifted if needed. The physical layer is where all the work is done, paging out to the media, etc.
Anyway, back to what I originally had in mind with this post. The code I link to is a simple heap allocator, usable even if there is no underlining layers. A beginner may simply point the mmap() call to a string of physical pages and return. As long as these pages are guaranteed to be free and present, *and* there is never a call to add a bucket to the heap, this code works just fine. A beginner can have a heap allocation malloc() drop-in with no other memory layers available. Then when they write the virtual layer, simply implement the mmap() functionality and this heap continues to work as is, no modifications.
The reason was that most of the code I looked at, when deciding what kind of heap layer I wanted, was all either extremely simple, but memory hungry; or more complex, but very difficult to understand what was happening. In fact, I did see Doug Lea's code. Without sounding like I am disrespecting it in any way, there were so many macros used, a beginner would be completely lost.
The code I provided is for the beginner. It will allocate and free memory, very similar to a libc call. Granted, the alignment part is not included. This is left for the beginner to implement. When the beginner has enough of a project to allocate physical memory for hardware drivers, I believe they should have the knowledge to advance this code, or even write their own. For now, in my humble opinion, it allows a beginner to understand and use a memory chunk less than the page size, eliminating fragmentation, but still allocating large chunks as well.
I do appreciate the comments. As I said, I started many years ago and would not be anywhere near what I have today if it wasn't for forums like this one. As a nostalgic note, how many of us remember BBSs. :-)
Thanks,
Ben