OSDev.org

The Place to Start for Operating System Developers
It is currently Thu Mar 28, 2024 11:09 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 10 posts ] 
Author Message
 Post subject: Virtual Memory Manager
PostPosted: Fri May 11, 2007 11:28 am 
Offline
Member
Member

Joined: Tue Nov 07, 2006 7:37 am
Posts: 514
Location: York, England
Should a (good) Virtual Memory Manager (Paging to secondary Storage) mechanism Run as a seperate thread (in any context?) or only work when reacting (new process interupt etc.)?


Last edited by Tyler on Fri May 11, 2007 4:01 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Virtual Memory Manager
PostPosted: Fri May 11, 2007 11:53 am 
Offline
Member
Member
User avatar

Joined: Fri Dec 15, 2006 5:26 pm
Posts: 437
Location: Church Stretton Uk
Tyler wrote:
Showuld a (good) Virtual Memory Manager (Paging to secondary Storage) mechanism Run as a seperate thread (in any context?) or only work when reacting (new process interupt etc.)?


Windows has a daemon, which runs in the background freeing up pages which haven't been used for a while.


Top
 Profile  
 
 Post subject: Re: Virtual Memory Manager
PostPosted: Fri May 11, 2007 1:45 pm 
Offline
Member
Member
User avatar

Joined: Tue Oct 17, 2006 6:06 pm
Posts: 1437
Location: Vancouver, BC, Canada
mathematician wrote:
Windows has a daemon, which runs in the background freeing up pages which haven't been used for a while.


It's not a daemon (otherwise known as a "Service" in Windows). It's actually just a dedicated kernel thread called the Balance Set Manager.

@Tyler:
It depends on the architecture of your OS. For a microkernel, you may want to implement the swapping mechanism in a separate process. For a monolithic kernel, dedicated kernel threads like Windows uses seems like a good idea to me.

_________________
Top three reasons why my OS project died:
  1. Too much overtime at work
  2. Got married
  3. My brain got stuck in an infinite loop while trying to design the memory manager
Don't let this happen to you!


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 14, 2007 2:21 am 
Offline
Member
Member

Joined: Tue Nov 07, 2006 7:37 am
Posts: 514
Location: York, England
By Kernel thread you mean the kernel polls it as a normal thread but it always runs in kernel mode? Would it be polled without changing the context (i.e. Doesn't matter which process owns bottom half of memory)?


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 14, 2007 8:21 am 
Offline
Member
Member
User avatar

Joined: Tue Oct 17, 2006 6:06 pm
Posts: 1437
Location: Vancouver, BC, Canada
Tyler wrote:
By Kernel thread you mean the kernel polls it as a normal thread but it always runs in kernel mode?
Yes.
Tyler wrote:
Would it be polled without changing the context (i.e. Doesn't matter which process owns bottom half of memory)?
Yes.

_________________
Top three reasons why my OS project died:
  1. Too much overtime at work
  2. Got married
  3. My brain got stuck in an infinite loop while trying to design the memory manager
Don't let this happen to you!


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 14, 2007 3:25 pm 
Offline
Member
Member

Joined: Tue Nov 07, 2006 7:37 am
Posts: 514
Location: York, England
That is the biggest waste of Running time i have ever heard of... plus all the wasted space on manging every detail about every page just so it can decide to poll something out.

No one would happen to have links to good explanations/algorithms for efficient swapping? I am quite deep into it at the moment but it seems to simply get slower.


Top
 Profile  
 
 Post subject: Memory Manager Kernel Thread Swapper Thresholds
PostPosted: Mon May 14, 2007 3:48 pm 
Offline
Member
Member
User avatar

Joined: Tue Nov 09, 2004 12:00 am
Posts: 843
Location: United States
I am not sure how much this might help for a explanation but:

Let the kernel over allocate memory. For example when the process wants memory let it request it and be granted with success, but *do not* actually map this memory yet. Instead wait until the process accesses it and produces a page fault then map the memory.

Keep some threshold values around so that you may say something like:

unsigned char vmm_swap_percentUnused = 10;
unsigned char vmm_swap_percentCache = 80;


(Note: I use the word 'swapper' which I have no idea why. I just started writing it calling it 'swapper'; Trying to avoid too much confusion.)

You have free memory that is not being used by anything. The job of the swapper is to keep this balanced between cache and unused. I am just suggesting unused out of inexperience but my idea is that you always maintain some number of pages that can immediately be mapped for a processor on a page fault when accessing a newly allocated page of memory. This little bit percentUnused gives the swapper thread a little latency to start balancing the memory by writting some cache to disk and making these pages unused.

The swapper thread might run every second or in a emergency when a processor has allocated and used memory so fast all of the vmm_swap_percentUnused have been allocated and some are needed. Hopefully the later does not happen or there are run-time calibrations to alleviate this.

Never the less about every second or so the swapper will balance between the set of unused and the set of cache to maintain the threshold.

Since the swapper is a kernel thread there is no context switch overhead (address change).

And. Yes I actually got this idea from the 2.6 kernel which does a very similar action but most likely very complicated compared to the (above). See:

(If you have Linux)
/usr/src/linux/mm/page-writeback.c
/usr/src/linux-xxx/mm/page-writeback.c

(if you download linux)
/linux-xxx/mm/page-writeback.c

Also the 2.6 Linux kernel let you alter these values during run-time to change the MM's policy if you will. Such that I can actually tell the kernel to leave cache in memory longer to increase battery life on my laptop by making less disk activity.

I also know the Linux kernel will somehow track hot and cold pages. This could also be performed by the swapper in which it tries to keep all hot pages or (high page fault count) pages in memory longer than cold ones.

_________________
Projects


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 14, 2007 4:18 pm 
Offline
Member
Member
User avatar

Joined: Tue Oct 17, 2006 6:06 pm
Posts: 1437
Location: Vancouver, BC, Canada
Tyler wrote:
That is the biggest waste of Running time i have ever heard of...


Sorry, I didn't think you literally meant "poll". The balance set manager thread wakes up every few seconds.

Quote:
plus all the wasted space on manging every detail about every page just so it can decide to poll something out.


Why would there be wasted space? The balance set manager uses the "clock" algorithm to decide which pages to steal from a working set. All this requires is an "accessed" bit, which is already in each PTE.

I don't really have time now to totally grasp kmcguire's post, but I think he's describing page buffering, which is the general term for the technique used by the Windows MM. I wouldn't be at all surprised to find that most modern OSes do things very similarly.

If you want a good explanation of how the Windows MM works, grab a copy of Windows Internals. Even if you otherwise don't care much for Windows :) the explanations in the book are detailed and I find them very helpful.

_________________
Top three reasons why my OS project died:
  1. Too much overtime at work
  2. Got married
  3. My brain got stuck in an infinite loop while trying to design the memory manager
Don't let this happen to you!


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 14, 2007 4:42 pm 
Offline
Member
Member
User avatar

Joined: Tue Nov 09, 2004 12:00 am
Posts: 843
Location: United States
I think I want to buy that book now.. When I save up I am going to see if a local store has it in stock. :)

_________________
Projects


Top
 Profile  
 
 Post subject:
PostPosted: Wed May 16, 2007 10:26 am 
Offline
Member
Member

Joined: Tue Nov 07, 2006 7:37 am
Posts: 514
Location: York, England
It definetly does look good... i think i may invest in that once i have bought every thing else of my list, which is extremely long at the moment.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 10 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 27 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group