OSDev.org

The Place to Start for Operating System Developers
It is currently Sun Jul 03, 2022 11:16 pm

All times are UTC - 6 hours




Post new topic Reply to topic  [ 7 posts ] 
Author Message
 Post subject: memory management with multiple cores
PostPosted: Sun Jun 12, 2022 10:46 am 
Offline
Member
Member
User avatar

Joined: Sun Mar 21, 2021 1:09 pm
Posts: 37
Location: current location
Hello, I was wondering how memory management with multiple cores would work.
My guess is that you have one master CPU which keeps track of the others.
So the master CPU tells the slaves:
    1. the address of the page directory of the process
    2. the address that the CPU needs to start executing
Assuming this is correct, how do the Master and Slave communicate?
over interrupts (LAPIC, IOAPIC)?




[/code]

_________________
ensis iustitae iustae illos iniustos iudicat


Top
 Profile  
 
 Post subject: Re: memory management with multiple cores
PostPosted: Sun Jun 12, 2022 12:45 pm 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 1252
acccidiccc wrote:
My guess is that you have one master CPU which keeps track of the others.

Whatever for? Far better to manage your memory in such a way that any CPU can just call up the manager and request memory. The easiest (though not fastest) option for that is to just use a big allocator lock. Whenever memory is being allocated, you take a lock, look up the address in whatever data structure you have, mark it as used, and release the lock. While not the fastest option, it is still easier than doing what you wanted to do. No need to limit the allocator to a single core.

_________________
Carpe diem!


Top
 Profile  
 
 Post subject: Re: memory management with multiple cores
PostPosted: Sun Jun 12, 2022 6:50 pm 
Offline
Member
Member

Joined: Tue Feb 18, 2020 3:29 pm
Posts: 930
No, that leads to asymmetric (i.e., your OS will be AMP not SMP) treatment of the CPUs, which will turn into a bottleneck. Instead, I would start out by having global memory management structures (i.e., your physical memory slab / bitmap / free list, and your virtual memory structures) that all CPUs work with equally, as if there was one CPU.

One catch though: these global structures will need to have locks. If you haven't looked it locking, now's the time to do it - it comes up everywhere is multi-processor systems.

The other option would be to have per-CPU memory management structures - that would be lockless (which is faster), but would be a pain to implement right, as memory is typically a global resource, not a per-CPU resource.

One great guide to memory management is on the wiki: https://wiki.osdev.org/Brendan%27s_Memory_Management_Guide


Top
 Profile  
 
 Post subject: Re: memory management with multiple cores
PostPosted: Mon Jun 13, 2022 12:31 am 
Offline
Member
Member

Joined: Mon Aug 27, 2018 12:50 pm
Posts: 43
As the others have said, you effectively have to ensure that only one processor is accessing the structures at any one time. You can do this either by having a separate structure for each processor, or by using locks. Locks are the way most Operating Systems handle it. When it comes to locks, you need to hit a balance between making them fine-grained, but low-contention - a fast memory allocator will also help with this. If done right, there should be no need to communicate anything to the other processors, they just need to be able to check the lock.

In terms of virtual memory management, each processor should have a different MMU, and as such can have a different virtual address space. There can be some troubles if multiple processors are using the same address space, since they'll still be using separate TLBs, so any invalidated pages will need to be communicated to all processors via an IPI. Handling this is up to you and how you design your kernel, you can use IPIs per page, use an IPI to clear the entire TLB, or schedule selectively to stop processors sharing address spaces. I just thought I'd throw this in since all other posts have been about physical memory management.

acccidiccc wrote:
    1. the address of the page directory of the process
    2. the address that the CPU needs to start executing
It seems like you're confused over the job of the Memory Manager, and the job of the Scheduler. These two things should be stored in the relevant Process Control Block. Each processor should just be able to ask the scheduler for another task, and it should switch that processor the next (currently not running) task. This job includes saving the current task's registers and updated information, then loading the new task's address space and jumping to the new task's saved instruction pointer. The Memory Manager should just be there to allocate memory to tasks that need it.

Thanks,
Barry


Top
 Profile  
 
 Post subject: Re: memory management with multiple cores
PostPosted: Mon Jun 13, 2022 9:40 am 
Offline
Member
Member
User avatar

Joined: Sun Mar 21, 2021 1:09 pm
Posts: 37
Location: current location
Thanks for the Answers.
So Instead of having some master-slave setup, I have some global structure that implements locking. (would be the same for the scheduler)
So when a core allocates some a page, it sets some lock variable to true and all other cores have to poll? or wait for some IPI to allocate memory.

Barry wrote:
In terms of virtual memory management, each processor should have a different MMU, and as such can have a different virtual address space. There can be some troubles if multiple processors are using the same address space, since they'll still be using separate TLBs, so any invalidated pages will need to be communicated to all processors via an IPI. Handling this is up to you and how you design your kernel, you can use IPIs per page, use an IPI to clear the entire TLB, or schedule selectively to stop processors sharing address spaces. I just thought I'd throw this in since all other posts have been about physical memory management.


thanks for throwing this in :) . The cores would use the same address space when multithreading, right?
So upon invalidating (what do you mean by that?) the pages you need to invalidate the TLB to clear it. So if the cores share a page directory, and new pages get allocated, the TLBs of the cores in question need to be updated to prevent invalid entries.

nexos wrote:
One great guide to memory management is on the wiki: https://wiki.osdev.org/Brendan%27s_Memo ... ment_Guide

thanks for this link.

_________________
ensis iustitae iustae illos iniustos iudicat


Top
 Profile  
 
 Post subject: Re: memory management with multiple cores
PostPosted: Mon Jun 13, 2022 10:39 am 
Offline
Member
Member

Joined: Mon Aug 27, 2018 12:50 pm
Posts: 43
acccidiccc wrote:
Thanks for the Answers.
So Instead of having some master-slave setup, I have some global structure that implements locking. (would be the same for the scheduler)
So when a core allocates some a page, it sets some lock variable to true and all other cores have to poll? or wait for some IPI to allocate memory.
Look into spinlocks, and some other synchronisation primitives - you need them when you have a multi-processor system.

acccidiccc wrote:
thanks for throwing this in :) . The cores would use the same address space when multithreading, right?
So upon invalidating (what do you mean by that?) the pages you need to invalidate the TLB to clear it. So if the cores share a page directory, and new pages get allocated, the TLBs of the cores in question need to be updated to prevent invalid entries.
Yes, multi-threading will cause that, but only if multiple threads of the same process are being run at the same time by different processors.
When a processor wants to translate a virtual address to a physical address, it checks the TLB first, if it can't find it there, then it checks the page tables. If you update the page tables, e.g. changing a page's frame or removing it, then you have to also remove that entry in the TLB so that the change is visible to the processor. This is easy on the processor that made the change, since you already know which entry to invalidate, but you have to communicate this to the other processors that are sharing the address space too, or they'll keep using their cached version of the mapping.
A very easy solution is just to have an IPI that clear's the entire TLB on the processor that receives it, this is called a TLB shootdown. This is what happens when you switch address spaces too.

Thanks,
Barry


Top
 Profile  
 
 Post subject: Re: memory management with multiple cores
PostPosted: Mon Jun 13, 2022 4:07 pm 
Offline
Member
Member

Joined: Wed Oct 01, 2008 1:55 pm
Posts: 2812
You shouldn't think of cores as things that run code, rather as resources your scheduler can use to get things done. The scheduler should decide which threads are allocated to which cores and thus what the cores are running. So, your "master" is the scheduler, not a specific core.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 7 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group