OSDev.org

The Place to Start for Operating System Developers
It is currently Thu Mar 28, 2024 11:06 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 6 posts ] 
Author Message
 Post subject: Memory managment
PostPosted: Mon Jun 01, 2020 8:39 am 
Offline
Member
Member

Joined: Sun Apr 05, 2020 1:01 pm
Posts: 182
Hello everyone.

I've just started working on memory allocation/paging/etc for my OS and I have a few questions:

I've seen a few implementations of kernel heap and they all seem to have a hardcoded size pool of a few megabytes just above the kernel.
1. Is this OK? For example if I load the kernel at 1 megabyte physical, then I should have a few free megabytes above me, is it safe to assume that I do?
2. If not, how does one handle setting up kernel heap without any free memory at hand? (so basically chicken and egg problem).
3. Can the kernel use other physical ranges of memory after its done parsing the memory map (or rather is this a normal practice?) or is it limited with that hardcoded heap size?

If I understand everything correctly the right memory initialization order is:
init basic kernel heap of hardcoded size -> init paging and move kernel to 3GB (obv. can be any other location) -> parse memory map and detect free pages -> init allocator with free pages
Is this accurate? ^

Would really appreciate any other memory handling advice and how to do things right from experienced osdev people. :)


Top
 Profile  
 
 Post subject: Re: Memory managment
PostPosted: Mon Jun 01, 2020 9:02 am 
Offline
Member
Member
User avatar

Joined: Mon Jun 05, 2006 11:00 pm
Posts: 2293
Location: USA (and Australia)
Do you need a kernel heap before you setting up a physical allocator and paging?

I ask this because the paging structures (page tables, page directories, etc.) are 4KB - the size of a page. I allocate my boot stack and boot-time paging structures in the linker script, and I use GRUB as my bootloader, so I'd asssume GRUB would give me an error if it tries to load my kernel into memory that doesn't exist.
One of the first things I do is loop through the multiboot header that GRUB populates, and create a linked stack of free memory pages (I store the pointer to the next free page at the start of the previous page.) Now that I can request/release 4KB pages, I have all I need to set up my paging structures. (After paging is set up, I can release my boot-time paging structures back to the physical allocator.)

With your physical and virtual allocators set up, your malloc implementation can request a page to use as the heap. liballoc is popular because it's super simple to port.

_________________
My OS is Perception.


Top
 Profile  
 
 Post subject: Re: Memory managment
PostPosted: Mon Jun 01, 2020 6:28 pm 
Offline
Member
Member

Joined: Tue Apr 03, 2018 2:44 am
Posts: 401
AndrewAPrice wrote:
Do you need a kernel heap before you setting up a physical allocator and paging?

I ask this because the paging structures (page tables, page directories, etc.) are 4KB - the size of a page. I allocate my boot stack and boot-time paging structures in the linker script, and I use GRUB as my bootloader, so I'd asssume GRUB would give me an error if it tries to load my kernel into memory that doesn't exist.
One of the first things I do is loop through the multiboot header that GRUB populates, and create a linked stack of free memory pages (I store the pointer to the next free page at the start of the previous page.)


Do you store the pointer in the page itself, or in some external management structure? I ask, because while your scheme has the benefit of O(1) allocation, it does require all your physical memory to be mapped into virtual memory if you store the pointers in the page itself.

Of course, that's not likely to be a problem for a 64-bit only OS, with vast swathes of VM at its disposal. My OS, on the other hand, is currently 32-bit (and I want to keep it 32-bit capable and not assume I can map all of HW memory in VM) and just as an added bonus, I'm trying to keep the OS mapped above 0xf0000000 on x86, to give me as much user address space as possible.

My page allocator is bit mapped, and so there is a small cost to scanning memory for free pages.


Top
 Profile  
 
 Post subject: Re: Memory managment
PostPosted: Mon Jun 01, 2020 8:50 pm 
Offline
Member
Member
User avatar

Joined: Mon Jun 05, 2006 11:00 pm
Posts: 2293
Location: USA (and Australia)
thewrongchristian wrote:
Do you store the pointer in the page itself, or in some external management structure? I ask, because while your scheme has the benefit of O(1) allocation, it does require all your physical memory to be mapped into virtual memory if you store the pointers in the page itself.

The pointer is stored in the page itself.

You don't need to keep all the physical memory mapped into virtual memory, just the page you're currently operating on.

_________________
My OS is Perception.


Top
 Profile  
 
 Post subject: Re: Memory managment
PostPosted: Tue Jun 02, 2020 4:51 pm 
Offline
Member
Member

Joined: Tue Apr 03, 2018 2:44 am
Posts: 401
AndrewAPrice wrote:
thewrongchristian wrote:
Do you store the pointer in the page itself, or in some external management structure? I ask, because while your scheme has the benefit of O(1) allocation, it does require all your physical memory to be mapped into virtual memory if you store the pointers in the page itself.

The pointer is stored in the page itself.

You don't need to keep all the physical memory mapped into virtual memory, just the page you're currently operating on.


Right, and the link is the next physical page number rather than a VM pointer?

I'll definitely put that one on the list to investigate.


Top
 Profile  
 
 Post subject: Re: Memory managment
PostPosted: Tue Jun 02, 2020 5:10 pm 
Offline
Member
Member
User avatar

Joined: Mon Jun 05, 2006 11:00 pm
Posts: 2293
Location: USA (and Australia)
thewrongchristian wrote:
Right, and the link is the next physical page number rather than a VM pointer?


That's correct. I store the physical address for the next page inside the page. Then you can temporarily map it into memory to ready/write this address.

_________________
My OS is Perception.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 6 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: Bing [Bot] and 51 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group