OSDev.org

The Place to Start for Operating System Developers
It is currently Thu Mar 28, 2024 4:42 pm

All times are UTC - 6 hours




Post new topic Reply to topic  [ 7 posts ] 
Author Message
 Post subject: Reclaiming pagetable pages, common?
PostPosted: Mon Oct 19, 2020 2:50 pm 
Offline
Member
Member

Joined: Mon Jul 05, 2010 4:15 pm
Posts: 595
When unmapping memory in the pagetable it seems like many systems just leave the pages used in pagetable itself and those pages will be reclaimed first when the process ends. However, are there operating systems that actually cleans up the pagetable itself when doing an unmap. Usually this involves scanning the pagetable and if all entries are empty, the page can be removed. Scanning takes time and requires that you scan the range you unmap including adjacent ones. Another possibility is to have a counter for each pagetable page but requires extra data which would be roughly 50% more memory.

Just leaving those pages will take up space if only allocating a big chunk and then free it.

Do you know how commercial and mission ready operating systems deal with this? Just leave them or reclaim?


Top
 Profile  
 
 Post subject: Re: Reclaiming pagetable pages, common?
PostPosted: Sat Nov 07, 2020 8:54 pm 
Offline
Member
Member

Joined: Tue Apr 03, 2018 2:44 am
Posts: 402
OSwhatever wrote:
When unmapping memory in the pagetable it seems like many systems just leave the pages used in pagetable itself and those pages will be reclaimed first when the process ends. However, are there operating systems that actually cleans up the pagetable itself when doing an unmap. Usually this involves scanning the pagetable and if all entries are empty, the page can be removed. Scanning takes time and requires that you scan the range you unmap including adjacent ones. Another possibility is to have a counter for each pagetable page but requires extra data which would be roughly 50% more memory.

Just leaving those pages will take up space if only allocating a big chunk and then free it.

Do you know how commercial and mission ready operating systems deal with this? Just leave them or reclaim?


I'm not aware of any that do or don't do this, but it shouldn't be as expensive as scanning the page table entries. Your virtual memory manager should know what regions are mapped, and when unmapping a region, it'll be easy to test whether that region spans an entire page table and just unmaps that single page table range at the page directory level if so.

Whether it's worth doing is another matter.

Still, there are operating systems where page mappings are entirely transient, and can be discarded and rebuilt at any time. I believe Coherent didn't even have a per-processes page table, and on a task switch, just cleared and reused a single, shared page table. Potentially quite expensive when switching address spaces, but at the time a program might only use a handful of page table pages anyway.


Top
 Profile  
 
 Post subject: Re: Reclaiming pagetable pages, common?
PostPosted: Sun Nov 08, 2020 12:42 am 
Offline
Member
Member
User avatar

Joined: Sat Mar 31, 2012 3:07 am
Posts: 4591
Location: Chichester, UK
If more memory is required, and pages are allocated but not being used, they should be swapped out to the swap space. So unless the system is tight on both physical memory and swap space it doesn't matter. I'd say that the overhead of continually monitoring pages in the way you suggest would be worse than running out of resources.


Top
 Profile  
 
 Post subject: Re: Reclaiming pagetable pages, common?
PostPosted: Sun Nov 08, 2020 12:34 pm 
Offline
Member
Member

Joined: Tue Aug 11, 2020 12:14 pm
Posts: 151
iansjack wrote:
If more memory is required, and pages are allocated but not being used, they should be swapped out to the swap space. So unless the system is tight on both physical memory and swap space it doesn't matter. I'd say that the overhead of continually monitoring pages in the way you suggest would be worse than running out of resources.

Swapping out pages that shouldn't exist any more seems wasteful. At the time that the decision is made to page them, you're better off deallocating them. That's sure to take less CPU and clock time than writing to disk.

"continually monitoring" isn't automatically necessary either. If you don't want to take the time at the termination of a process to reclaim space used by page tables, keep a list of the top-level table page for each terminated process and either force a cleanup when memory drops too low, or (if you're capable of it) dispatch a low-priority kernel task thread to clean up in the background. Unless you're low on memory, there's no reason all post-termination cleanup must be synchronous.


Top
 Profile  
 
 Post subject: Re: Reclaiming pagetable pages, common?
PostPosted: Sun Nov 08, 2020 1:12 pm 
Offline
Member
Member
User avatar

Joined: Sat Mar 31, 2012 3:07 am
Posts: 4591
Location: Chichester, UK
I understood the OP to be talking about pages in a running process. Disposing of pages once the process has terminated is relatively simple and something that every OS does. But to keep track of pages in a running process that are no longer used is not trivial. Just because a page is a candidate for swapping doesn't mean that it is unused. Truth is, with a reasonable amount of RAM swapping hardly ever occurs so this is a solution in search of a non-existant problem.

In any case, surely an unused page in a running process is either the result of dynamic memory allocation or a stack that has shrunk. In either case the memory is likely to be needed again in the future, so what is the point of needlessly freeing and reallocating it?


Top
 Profile  
 
 Post subject: Re: Reclaiming pagetable pages, common?
PostPosted: Sun Nov 08, 2020 1:20 pm 
Offline
Member
Member

Joined: Tue Feb 18, 2020 3:29 pm
Posts: 1071
For me, when I unmap an address, I don't reclaim the physical memory used by the page tables. That would be slow to scan through. But I do delete them when the process is terminated.

_________________
"How did you do this?"
"It's very simple — you read the protocol and write the code." - Bill Joy
Projects: NexNix | libnex | nnpkg


Top
 Profile  
 
 Post subject: Re: Reclaiming pagetable pages, common?
PostPosted: Sun Nov 08, 2020 2:46 pm 
Offline
Member
Member

Joined: Mon Jul 05, 2010 4:15 pm
Posts: 595
thewrongchristian wrote:
Your virtual memory manager should know what regions are mapped, and when unmapping a region, it'll be easy to test whether that region spans an entire page table and just unmaps that single page table range at the page directory level if so.


That's actually a good idea to use the virtual memory memory manager, I never thought about something so obvious even as my OS have one.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 7 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 24 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group