Strategies for creating a new virtual address space.

[stillconfused]

Hello.
I'm wondering how people are mapping new paging structures into the current virtual address space when creating a new virtual address space.
Note: This is for x86, without PAE.

Forgive what might be a silly, or unusual question.
I'm currently beginning the task of creating a new virtual address space for a new process. The 'process specific' side of things I think I understand. What I'm confused about is a good strategy for how to create the new paging structures within the current virtual address space. I have a basic working PMM/VMM with the page directory recursively mapped into itself. My current method is as follows:
* Allocate new page frame in the physical memory manager for the new page directory.
* Map this to an arbitrary free frame in the current task's kernel address space.
* Initialize the directory structure.
* Copy in the current page table addresses from the recursively mapped current page dir into the new directory.
* Unmap the structure from memory when no longer needed.

Where are people mapping the new structures to? Is it common to reserve a specific page frame for doing this, or do people just temporarily map the new structures into the first free frame? I have a working implementation either way. I'm looking for a bit of guidance on what people think is a good idea here. Forgive me if I've worded this post poorly.
Any links/articles/documentation are absolutely appreciated.
Thanks in advance for any help.

[nullplan]

I think it is important to differentiate between the CPU and OS structures. In the OS, I have a binary tree structure for the kernel address space, and one for each task's address space. Both just tell the kernel what file (if any) has been mapped with what offset and to where, and which accesses are allowed. The CPU side of things is almost only handled in the page fault handler.

Since I am working in 64-bit mode, I can just map all of main memory to a fixed address. That means, converting between physical and virtual address merely means adding or subtracting a compile-time constant. When a new task is allocated, I allocate the PML4 and initialize it to have all high entries point to the same thing as the kernel's PML4, since the kernel's address space is shared everywhere, as well as all low entries set to zero. In the VMM tree, I have written down where the executable file is mapped. When the task starts, it will immediately page fault, and the page fault handler will allocate the PDPT, PDT, and PT, as necessary. When a task is freed, so are all of its mapping structures. In fact, they are deleted by the time the process is zombified.

The only time I need to touch the CPU structures outside of the page fault handler is if a mapping is changed or deleted. So only in mmap(), if MAP_FIXED is used and a mapping is overwritten, and that mapping has already had real memory allocated to it. Or in munmap(), or in madvise(MADV_DONTNEED). And in all of those cases, the CPU-bound structures are simply flushed, and the PF handler fills them back in.

[stillconfused]

Thanks for your reply.
This is a very interesting model that you're describing here. It's a little advanced for my purposes right now though. I've edited my post to make it clear that I'm using 32bit x86, and without PAE enabled. I'll definitely look into 64bit, and more advanced techniques in the long-term. For now I'm just working on a very simplified model in x86 so I can get a good grasp on all of the theoretical concepts. So far so good, it's been very fulfilling.

[nullplan]

Well, that was just my point. 64-bit mode allows you to do things you can't in 32-bit mode. In 32-bit mode you have to muck around with temporary mappings and, for SMP, TLB shootdowns, just to manage your page tables.

The model of just mapping all memory to a certain point in kernel space used to be a very popular one. Linux employed it unchanged until RAM sizes grew to be 1GB. Then suddenly they had to decide which allocations remain eternal, and which will have to be swapped out as needed. Then, 64-bit mode arrived and now you can map all memory to one point again.

In fact, if you can contain your hunger for memory, you might want to start with the classic Linux model (768 MB of RAM mapped to the 3GB line. The rest used for ioremap(), i.e. MMIO, which may use high physical addresses). At least, it'll get your OS off the ground. And you can figure out how to do more RAM later.

[stillconfused]

Well, that was just my point. 64-bit mode allows you to do things you can't in 32-bit mode. In 32-bit mode you have to muck around with temporary mappings and, for SMP, TLB shootdowns, just to manage your page tables.

A very valid point. I hadn't spent much time thinking about what benefits 64-bit mode could gain me in this area. The reason for me choosing to develop a 32-bit kernel at first is that there is much more tutorial material available online, as well as it being much easier to get guidance on these concepts for x86 from articles online and books.

The model of just mapping all memory to a certain point in kernel space used to be a very popular one. Linux employed it unchanged until RAM sizes grew to be 1GB. Then suddenly they had to decide which allocations remain eternal, and which will have to be swapped out as needed. Then, 64-bit mode arrived and now you can map all memory to one point again.

In fact, if you can contain your hunger for memory, you might want to start with the classic Linux model (768 MB of RAM mapped to the 3GB line. The rest used for ioremap(), i.e. MMIO, which may use high physical addresses). At least, it'll get your OS off the ground. And you can figure out how to do more RAM later.

That's a very interesting suggestion. I had read some minimal information about this by searching these forums for an answer. It seems controversial. Is this something that's really recommended for new developers? I can see how this would resolve my issues, but It sounds like something best avoided. I'm willing to accept any criticism for this, I feel like my current ideas of reserving a page in the kernel memory space for the purpose of mapping new structures seems like a better solution, even with refreshing the TLB being involved.

[nullplan]

A very valid point. I hadn't spent much time thinking about what benefits 64-bit mode could gain me in this area. The reason for me choosing to develop a 32-bit kernel at first is that there is much more tutorial material available online, as well as it being much easier to get guidance on these concepts for x86 from articles online and books.

64-bit mode is 13 years old by now. Tutorials are unlikely to catch up if they haven't by now. But 64-bit mode does ease quite a few things. Also, Schol-R-LEA keeps talking about a time when no mode will exist except 64-bit mode. And that may not be too far off.

Thing is, if you are developing a new thing now, would you not want to base it on the technology that will still be around when you're done?

That's a very interesting suggestion. I had read some minimal information about this by searching these forums for an answer. It seems controversial.

Welcome to the bikeshed argument, everyone will have an opinion.

Is this something that's really recommended for new developers?

I would recommend it, since it eases the mental burden of modelling paging, since at least in kernel space, all you are doing is adding a constant. Makes it so much easier to work with physical addresses.

I'm willing to accept any criticism for this, I feel like my current ideas of reserving a page in the kernel memory space for the purpose of mapping new structures seems like a better solution, even with refreshing the TLB being involved.

That only solves one tiny problem, the setting up of a new address space. In your model, you can never change an address space that is not currently running. You may get away with this until you try to implement swapping, because there, the entire idea is to evict pages that haven't been used in a while.

[stillconfused]

64-bit mode is 13 years old by now. Tutorials are unlikely to catch up if they haven't by now. But 64-bit mode does ease quite a few things. Also, Schol-R-LEA keeps talking about a time when no mode will exist except 64-bit mode. And that may not be too far off.

This is a very good point. When you put it like that, you make a very compelling case.
I don't intend to work with x86 forever. My current plan is to get my hobby OS project to a certain point with x86 and then move to a 64-bit RISC architecture like ARM. My reasons for working with x86 up to this point are the wide availability of good emulators and tutorial material. I think that many people who are already successful in this area underestimate the difficulty in learning the basic concepts that underpin osdev. The more complexity you can remove from the implementation side of things, the better.
I have done some basic x86_64 osdev before, but you need not look much further than the articles in the wiki on this site to see that it has its own unique pitfalls: Creating_a_64-bit_kernel. The way I figured it, the less time I spend dealing with these issues, the more I can learn.

I would recommend it, since it eases the mental burden of modelling paging, since at least in kernel space, all you are doing is adding a constant. Makes it so much easier to work with physical addresses.

Thank you very much for taking the time to suggest this. I'll definitely take this into consideration. It's helped me think the problem out.

That only solves one tiny problem, the setting up of a new address space. In your model, you can never change an address space that is not currently running. You may get away with this until you try to implement swapping, because there, the entire idea is to evict pages that haven't been used in a while.

Point taken. Something I hadn't thought about at all. Maybe I'm looking in all the wrong places, but I never see any discussion of this topic. Are there any resources you or anyone else can point to that discuss these aspects of VMM in greater detail? I'll accept book recommendations too.

[iansjack]

There's a good discussion of the problem, and possible solutions, here: https://os.phil-opp.com/paging-implementation/