OSDev.org

The Place to Start for Operating System Developers
It is currently Thu Apr 18, 2024 11:59 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 12 posts ] 
Author Message
 Post subject: Writing VMM PMM
PostPosted: Thu Jan 05, 2017 5:00 am 
Offline
Member
Member
User avatar

Joined: Fri Aug 07, 2015 6:13 am
Posts: 1134
I have a question that confuses me a lot. I know what heap is, that is not my problem. What are the key differences between VMM and PMM? Is paging VMM or PMM (I think VMM)?
Can you show me an example where I would have to use them. What are the key functions I need to implement within them?

_________________
OS: Basic OS
About: 32 Bit Monolithic Kernel Written in C++ and Assembly, Custom FAT 32 Bootloader


Top
 Profile  
 
 Post subject: Re: Writing VMM PMM
PostPosted: Thu Jan 05, 2017 5:22 am 
Offline
Member
Member

Joined: Tue Mar 04, 2014 5:27 am
Posts: 1108
How about a small homeworkish research? :)


Top
 Profile  
 
 Post subject: Re: Writing VMM PMM
PostPosted: Thu Jan 05, 2017 5:28 am 
Offline
Member
Member
User avatar

Joined: Fri Aug 07, 2015 6:13 am
Posts: 1134
alexfru wrote:
How about a small homeworkish research? :)


Already done, I just can't find anything that I would personally need. Where ever I look there is a different implementation, one guy puts paging in PMM one in VMM, etc...

_________________
OS: Basic OS
About: 32 Bit Monolithic Kernel Written in C++ and Assembly, Custom FAT 32 Bootloader


Top
 Profile  
 
 Post subject: Re: Writing VMM PMM
PostPosted: Thu Jan 05, 2017 5:35 am 
Offline
Member
Member
User avatar

Joined: Thu Apr 16, 2015 7:37 am
Posts: 64
PMM and VMM in this case refer to "Physical Memory Manager" and "Virtual Memory Manager", and (from my understanding) they're each responsible for managing their respective address spaces at the page level. For example, the physical memory manager manages the physical address space, which includes handing out and managing physical pages, and possibly other physical memory management responsibilities like swapping out pages and optimising the location of data in memory. The virtual memory manager, on the other hand, manages the virtual address space, of which usually there's one per process (but not always). This involves stuff like mapping in program code and data, and mapping in files and other things to be read by the application.

Note however, that they're not set in stone, that's just my understanding of what people mean when they say PMM and VMM. It depends from kernel to kernel what parts do what actions, especially in microkernels, where parts of (or most of) the memory managers may be implemented in userspace. Perhaps an expert could give you a more formal definition? As others have mentioned, they're just terms, and actual implementation details are left up to you. For example, even if your VMM handles mapping, there might be paging stuff involved in both the PMM and VMM (especially if the PMM handles stuff like swapping or optimisation).

- Mikumiku747


Top
 Profile  
 
 Post subject: Re: Writing VMM PMM
PostPosted: Thu Jan 05, 2017 5:39 am 
Offline
Member
Member

Joined: Tue Mar 04, 2014 5:27 am
Posts: 1108
If you don't need a feature, you don't need a feature.
If you don't know whether you need a feature, it's time to study up. Open up any decent book on operating systems and read the chapters dedicated to memory management and related things. That may give you some ideas why you may need a feature. However, you may still not need it, but at least it will be an informed decision. Right now you're uninformed.


Top
 Profile  
 
 Post subject: Re: Writing VMM PMM
PostPosted: Thu Jan 05, 2017 5:55 am 
Offline
Member
Member
User avatar

Joined: Fri Aug 07, 2015 6:13 am
Posts: 1134
Mikumiku747 wrote:
PMM and VMM in this case refer to "Physical Memory Manager" and "Virtual Memory Manager", and (from my understanding) they're each responsible for managing their respective address spaces at the page level. For example, the physical memory manager manages the physical address space, which includes handing out and managing physical pages, and possibly other physical memory management responsibilities like swapping out pages and optimising the location of data in memory. The virtual memory manager, on the other hand, manages the virtual address space, of which usually there's one per process (but not always). This involves stuff like mapping in program code and data, and mapping in files and other things to be read by the application.

Note however, that they're not set in stone, that's just my understanding of what people mean when they say PMM and VMM. It depends from kernel to kernel what parts do what actions, especially in microkernels, where parts of (or most of) the memory managers may be implemented in userspace. Perhaps an expert could give you a more formal definition? As others have mentioned, they're just terms, and actual implementation details are left up to you. For example, even if your VMM handles mapping, there might be paging stuff involved in both the PMM and VMM (especially if the PMM handles stuff like swapping or optimisation).

- Mikumiku747


I know most of that, read it online. :D That is about what I know in general when it comes to PMM/VMM theory. I just don't need swapping and stuff nor I will be implementing it. What are those functions that is what I am asking.

_________________
OS: Basic OS
About: 32 Bit Monolithic Kernel Written in C++ and Assembly, Custom FAT 32 Bootloader


Top
 Profile  
 
 Post subject: Re: Writing VMM PMM
PostPosted: Thu Jan 05, 2017 5:58 am 
Offline
Member
Member
User avatar

Joined: Fri Aug 07, 2015 6:13 am
Posts: 1134
alexfru wrote:
If you don't need a feature, you don't need a feature.
If you don't know whether you need a feature, it's time to study up. Open up any decent book on operating systems and read the chapters dedicated to memory management and related things. That may give you some ideas why you may need a feature. However, you may still not need it, but at least it will be an informed decision. Right now you're uninformed.


I know exactly what I need, I don't want to implement some random function for no reason.
My question is what are those other commonly used functions inside VMM and PMM that are like allocate page and free page.

_________________
OS: Basic OS
About: 32 Bit Monolithic Kernel Written in C++ and Assembly, Custom FAT 32 Bootloader


Top
 Profile  
 
 Post subject: Re: Writing VMM PMM
PostPosted: Thu Jan 05, 2017 6:16 am 
Offline
Member
Member
User avatar

Joined: Thu Oct 13, 2016 4:55 pm
Posts: 1584
octacone wrote:
My question is what are those other commonly used functions inside VMM and PMM that are like allocate page and free page.

I use PMM for allocating physical pages only (RAM management in other words). It's called by my kalloc() and VMM routines.

And in my VMM I've the following:
1. allocating and mapping bss memory for user threads
2. IDT points here, as the page fault handler is a VMM routine for me
3. I'm not planning to implement brk() and sbrk() system calls, but if I was, I would put them here.

That's all. My malloc(), calloc() and free() implementations will be purely in user space once I'll implement them, and they will rely on the functionality provided by VMM.


Top
 Profile  
 
 Post subject: Re: Writing VMM PMM
PostPosted: Thu Jan 05, 2017 6:16 am 
Offline
Member
Member

Joined: Tue Mar 04, 2014 5:27 am
Posts: 1108
octacone wrote:
alexfru wrote:
If you don't need a feature, you don't need a feature.
If you don't know whether you need a feature, it's time to study up. Open up any decent book on operating systems and read the chapters dedicated to memory management and related things. That may give you some ideas why you may need a feature. However, you may still not need it, but at least it will be an informed decision. Right now you're uninformed.


I know exactly what I need, I don't want to implement some random function for no reason.
My question is what are those other commonly used functions inside VMM and PMM that are like allocate page and free page.


Nuno may have been contagious.


Top
 Profile  
 
 Post subject: Re: Writing VMM PMM
PostPosted: Thu Jan 05, 2017 6:30 am 
Offline
Member
Member
User avatar

Joined: Sat Jan 15, 2005 12:00 am
Posts: 8561
Location: At his keyboard!
Hi,

octacone wrote:
I have a question that confuses me a lot. I know what heap is, that is not my problem. What are the key differences between VMM and PMM? Is paging VMM or PMM (I think PMM)?
Can you show me an example where I would have to use them. What are the key functions I need to implement within them?


Physical memory manager is primarily "allocate_one_physical_page()" and "free_one_physical_page()". It might have less frequently used variations (that are mostly only needed for device drivers) to allocate multiple physical pages, possibly with various restrictions (e.g. "must have 32-bit physical address"), and possibly with "cache-ability controls". It might support things like large pages, and NUMA optimisations, and cache/page colouring, and maybe even hot-plug RAM and ECC "corrected/uncorrected" error reporting. It can also be used to manage the whole physical address space, including the ability to keep track of which areas of the physical memory map are used for memory mapped devices, and including being responsible for managing MTRRs.

Virtual memory management is about creating an illusion - the illusion of multiple separate address spaces. "Virtual" means the same as it does in "virtual machine". Historically, virtual memory and processes come from the idea of using a computer to emulate multiple (virtual) computers, where each process has one (or more) virtual CPUs (threads) and its own virtual address space. To create the illusion, the virtual memory manager typically uses paging to map "things" into the virtual address space (and control permissions, etc). Things that are mapped into the virtual address include physical memory (obtained from the physical memory manager), and swap space, and files, and whatever else. It also includes trickery to reduce (physical) memory consumption, like various "copy on write" tricks. You can think of it as some function/s to create and destroy virtual address spaces; plus keeping track of the "virtual type" of areas within a virtual address space (e.g. if an area is supposed to behave like it's RAM, or not); plus whatever is necessary to make areas behave according to how their virtual type says they should even when (e.g.) the actual type of a page is nothing like the virtual type (e.g. when a virtual page is supposed to behave like RAM, but it's actually been sent to swap space and isn't RAM at all).

The result of the virtual memory manager is that a process has its own address space (that might be 3 GiB, and could be part of larger address space that the process needn't know about) and it can do whatever it likes with its virtual address space (or at least, it can do whatever kernel lets it do).

Note that this gets us to the boundary between "OS" and "language". A programming language isn't just a language, it's a specification for (yet another) abstract machine. For example, for the C abstract machine there is no stack, no registers, no flags, no instructions and no interrupts/exceptions (but there is variables and statements and functions and signals).

At the boundary between OS and language there's "mystical voodoo" to make the language's abstract machine work as a process. This mystical voodoo includes compilers, libraries, etc; and it can also include virtualisation (e.g. Java's virtual machine). For C, "malloc()" and "free()" are part of the mystical voodoo that makes C's abstract machine work as a process. For C++ and Java "new" and "delete" are part of the mystical voodoo that makes their abstract machines work. The mystical voodoo for a language is designed to match the OS (and designed to use the kernel's API, and virtual memory manager that the kernel provides).


Cheers,

Brendan

_________________
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.


Top
 Profile  
 
 Post subject: Re: Writing VMM PMM
PostPosted: Thu Jan 05, 2017 7:09 am 
Offline
Member
Member
User avatar

Joined: Fri Aug 07, 2015 6:13 am
Posts: 1134
bzt wrote:
octacone wrote:
My question is what are those other commonly used functions inside VMM and PMM that are like allocate page and free page.

I use PMM for allocating physical pages only (RAM management in other words). It's called by my kalloc() and VMM routines.

And in my VMM I've the following:
1. allocating and mapping bss memory for user threads
2. IDT points here, as the page fault handler is a VMM routine for me
3. I'm not planning to implement brk() and sbrk() system calls, but if I was, I would put them here.

That's all. My malloc(), calloc() and free() implementations will be purely in user space once I'll implement them, and they will rely on the functionality provided by VMM.


That is something I have no use of at the moment. Looks like I don't need a VMM for now. ;)

_________________
OS: Basic OS
About: 32 Bit Monolithic Kernel Written in C++ and Assembly, Custom FAT 32 Bootloader


Top
 Profile  
 
 Post subject: [Answered] Writing VMM PMM
PostPosted: Thu Jan 05, 2017 7:16 am 
Offline
Member
Member
User avatar

Joined: Fri Aug 07, 2015 6:13 am
Posts: 1134
Brendan wrote:
Hi,

octacone wrote:
I have a question that confuses me a lot. I know what heap is, that is not my problem. What are the key differences between VMM and PMM? Is paging VMM or PMM (I think PMM)?
Can you show me an example where I would have to use them. What are the key functions I need to implement within them?


Physical memory manager is primarily "allocate_one_physical_page()" and "free_one_physical_page()". It might have less frequently used variations (that are mostly only needed for device drivers) to allocate multiple physical pages, possibly with various restrictions (e.g. "must have 32-bit physical address"), and possibly with "cache-ability controls". It might support things like large pages, and NUMA optimisations, and cache/page colouring, and maybe even hot-plug RAM and ECC "corrected/uncorrected" error reporting. It can also be used to manage the whole physical address space, including the ability to keep track of which areas of the physical memory map are used for memory mapped devices, and including being responsible for managing MTRRs.

Virtual memory management is about creating an illusion - the illusion of multiple separate address spaces. "Virtual" means the same as it does in "virtual machine". Historically, virtual memory and processes come from the idea of using a computer to emulate multiple (virtual) computers, where each process has one (or more) virtual CPUs (threads) and its own virtual address space. To create the illusion, the virtual memory manager typically uses paging to map "things" into the virtual address space (and control permissions, etc). Things that are mapped into the virtual address include physical memory (obtained from the physical memory manager), and swap space, and files, and whatever else. It also includes trickery to reduce (physical) memory consumption, like various "copy on write" tricks. You can think of it as some function/s to create and destroy virtual address spaces; plus keeping track of the "virtual type" of areas within a virtual address space (e.g. if an area is supposed to behave like it's RAM, or not); plus whatever is necessary to make areas behave according to how their virtual type says they should even when (e.g.) the actual type of a page is nothing like the virtual type (e.g. when a virtual page is supposed to behave like RAM, but it's actually been sent to swap space and isn't RAM at all).

The result of the virtual memory manager is that a process has its own address space (that might be 3 GiB, and could be part of larger address space that the process needn't know about) and it can do whatever it likes with its virtual address space (or at least, it can do whatever kernel lets it do).

Note that this gets us to the boundary between "OS" and "language". A programming language isn't just a language, it's a specification for (yet another) abstract machine. For example, for the C abstract machine there is no stack, no registers, no flags, no instructions and no interrupts/exceptions (but there is variables and statements and functions and signals).

At the boundary between OS and language there's "mystical voodoo" to make the language's abstract machine work as a process. This mystical voodoo includes compilers, libraries, etc; and it can also include virtualisation (e.g. Java's virtual machine). For C, "malloc()" and "free()" are part of the mystical voodoo that makes C's abstract machine work as a process. For C++ and Java "new" and "delete" are part of the mystical voodoo that makes their abstract machines work. The mystical voodoo for a language is designed to match the OS (and designed to use the kernel's API, and virtual memory manager that the kernel provides).


Cheers,

Brendan


Thank you so much!! This is the answer I was looking for. Now I know what to look for and also code. Actually I have never used any sort of a virtual address space mechanism. I need to do some research on that topic to fill all the theoretical gaps I have. :wink:

_________________
OS: Basic OS
About: 32 Bit Monolithic Kernel Written in C++ and Assembly, Custom FAT 32 Bootloader


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 12 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: Google [Bot], SemrushBot [Bot] and 177 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group