OSDev.org

The Place to Start for Operating System Developers
It is currently Fri Nov 22, 2019 9:54 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 7 posts ] 
Author Message
 Post subject: Higher-half implementation question
PostPosted: Fri Nov 08, 2019 5:49 pm 
Offline

Joined: Fri Nov 08, 2019 5:35 pm
Posts: 2
Every example I've found of a higher-half kernel uses the linker script to set the kernel's VMA to the higher-half. I'm a little confused why this step is done in the linker script.
I understand why this works. Setting the VMA to 0xC0100000, LMA to ADDR(.section) - 0xC000000 then using an absolute jump from the identity mapped physical offset to the mapped virtual offset to enter the higher-half makes sense. I'm just a bit confused as to why it is done in this manner. Couldn't this be accomplished without setting the Kernel's virtual memory offset in the linker script? Couldn't I just load the kernel at 1MB, map that physical address to virtual address 0xC0100000 in the page directory then perform the absolute jump to begin operating in the higher-half?
Forgive me if I'm missing something obvious here. I'm guessing the answer lies in specifying where the kernel should think it is. If anyone could help give me some guidance here I'd be grateful. Any links to supporting documentation or books that help explain this theory would be appreciated too.

Please forgive me if this question has been better addressed elsewhere. I conducted a thorough search of the forum and I didn't find an exact answer to my specific question.
Just to clarify: so far I've been able to successfully implement a simple higher-half kernel using the linker script to set the virtual addresses. Everything is working well on my project so far. I've been able to successfully parse the GRUB memory map and I've got a working physical memory allocator. I'm just a bit confused about some of the higher-half theory.

Thank you!


Top
 Profile  
 
 Post subject: Re: Higher-half implementation question
PostPosted: Fri Nov 08, 2019 11:57 pm 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 278
Well, you are going to set the address somewhere, and if you're building a kernel, more than likely you will need a linker script anyway, so why not set it in there? In the end, you need to tell your linker where the code will be executing, and you will need a piece of code that creates a page directory and maps the kernel where it should be before handing control to the higher half, and how exactly you divvy that up is up to you.

I have a loader kernel linked at 1MB that maps the real kernel according to its ELF headers (the real kernel being a multiboot module), but as I'm working in 64-bit mode, this is just necessary. Well, its either this or an ungodly hack that results in a link that mixes architectures.


Top
 Profile  
 
 Post subject: Re: Higher-half implementation question
PostPosted: Sat Nov 09, 2019 12:39 am 
Offline
Member
Member
User avatar

Joined: Sat Mar 31, 2012 3:07 am
Posts: 3567
Location: Chichester, UK
If you don't tell the linker where the kernel is loaded, what happens when you access a variable using an absolute address? Unless all of your code is position independent, you have a problem.


Top
 Profile  
 
 Post subject: Re: Higher-half implementation question
PostPosted: Sat Nov 09, 2019 10:59 pm 
Offline
Member
Member

Joined: Wed Mar 09, 2011 3:55 am
Posts: 333
stillconfused wrote:
Forgive me if I'm missing something obvious here. I'm guessing the answer lies in specifying where the kernel should think it is.


Exactly. If the linker puts any absolute calls or jumps into the kernel, then those jumps have to go to wherever you have the kernel mapped, otherwise, the first absolute call/jump the kernel makes will go off into lala-land, and, depending on specifics, it's very likely that you'll trigger a triple fault and your machine will unceremoniously reboot.


Top
 Profile  
 
 Post subject: Re: Higher-half implementation question
PostPosted: Mon Nov 11, 2019 6:11 am 
Offline

Joined: Fri Nov 08, 2019 5:35 pm
Posts: 2
Thank you everyone for your responses. After having read them, this does seem obvious. The way I had figured it was that one way or another I would have to use the VMA offset when referencing absolute addresses. But now that I think about it, it would become untenable very quickly in a higher level language after having enabled paging.

There's a few more things I'm still confused ( ha ) about.
TL;DR: I've got everything working, I'm just confused about what my next steps are.
I've been reading through every thread on here I can find for more information on how people have implemented higher-half kernels. I've seen a few people here recommend using the page directory set up during the boot process as only a preliminary page directory to begin executing in the higher half, then setting up a permanent one after the jump to higher-level code. Are there any resources available that anyone could recommend which explain the theory behind this? I can see how I would accomplish this, but I'm still a bit confused about what my next steps are and why.
Prior to implementing the higher-half initialization code for my kernel, I was using the physical memory allocator to give me page frames at runtime to use for my kernel page directory/first kernel table. Now I've just reserved some space in the linker script for the boot page dir/table. I imagine my next step is just to create a permanent page dir/table and begin working towards userspace.
Is there a good example anyone can think of to show how and why these steps are accomplished? I've looked at the Linux/BSD source code and it seems like they have their own approaches to this. I haven't had enough time to decipher them properly to get any guidance.
Any help would be appreciated!
Thanks again.


Top
 Profile  
 
 Post subject: Re: Higher-half implementation question
PostPosted: Mon Nov 11, 2019 6:54 am 
Offline
Member
Member

Joined: Thu Jul 05, 2007 8:58 am
Posts: 222
stillconfused wrote:
I've been reading through every thread on here I can find for more information on how people have implemented higher-half kernels. I've seen a few people here recommend using the page directory set up during the boot process as only a preliminary page directory to begin executing in the higher half, then setting up a permanent one after the jump to higher-level code. Are there any resources available that anyone could recommend which explain the theory behind this? I can see how I would accomplish this, but I'm still a bit confused about what my next steps are and why.


There isn't a "theory" reason for this per-se, at least not in the strict sense of the meaning of OS system theory. However, there are practical considerations that make this an oft-used approach. First, before initialization of paging, all code used either needs to be specifically compiled to work at the low-memory boot adresses, or be position independent. This can be quite a hastle, so people generally try to keep this code small. That in turn usually means that a rather simplistic approach is used to create the page directory, usually by using a buffer pre-allocated during build in the kernel's data segment. However, doing this means these structures are allocated differently than paging structures created during normal operations. This isn't neccessarily a problem if you want to keep using them, but you then need to ensure that all the runtime code for manipulating paging structures can handle these special boot structures, either by special casing them or by making them act similar enough that the differences don't matter. As both of these options can be a bit of a hassle, it is just a lot easier to discard the boot paging structures once everything else is sufficiently initialized and proceed circumvent all that.

stillconfused wrote:
Prior to implementing the higher-half initialization code for my kernel, I was using the physical memory allocator to give me page frames at runtime to use for my kernel page directory/first kernel table. Now I've just reserved some space in the linker script for the boot page dir/table. I imagine my next step is just to create a permanent page dir/table and begin working towards userspace.
Is there a good example anyone can think of to show how and why these steps are accomplished? I've looked at the Linux/BSD source code and it seems like they have their own approaches to this. I haven't had enough time to decipher them properly to get any guidance.


This is a harder question to answer. The (in my opinion) best thing to do is get a picture of what your finished run time state should look like, and what pieces should function at that time. Then start working out how those bits should work, and how they interrelate. After you have done that, you'll probably find just how the various bits and pieces depend on each other, which should give you a good feel both for what order would be useful in implementing them, and how the rest of your initialization code then has to work.

A lot of the decisions here depend intimately on how various parts of your kernel, such as the scheduler and memory management subsystems, interact and function, so a specific order is hard to indicate.


Top
 Profile  
 
 Post subject: Re: Higher-half implementation question
PostPosted: Mon Nov 11, 2019 10:35 am 
Offline
Member
Member
User avatar

Joined: Wed Oct 27, 2004 11:00 pm
Posts: 865
Location: WA
for me, the memory management process works something like this

1) the bootloader (legacy stage 3, UEFI stage 5) detects available physical memory, divides it into FRAMEs
memory map is obtained (or constructed, depending on what the platform provides)
in use memory is added as reserved (unusable) entries
the memory map is sorted
the memory map is corrected to remove overlapping entries
reserved/restricted/unusable entries of the memory map are removed
memory map is sorted by "type" (including special memory zones, such as below 1MB and below 16MB) -- entries split if necessary
neighboring entries in the memory map are combined (as appropriate)
starting point for each entry in map is rounded up to the next FRAME boundary
ending point/length for each entry is adjusted down to nearest FRAME boundary
first FRAME is removed from map, and used to create a list of FRAME addresses
--(note: memory below 16MB is handled separately, and different localities/characteristics are each handled in a separate list)

2) the bootloader (stage 3 (legacy only) and stage 5) provides getFRAME functions to return available addresses for use by the bootloader

3) the bootloader (stage 5) initializes paging
--(assuming legacy paging, since it is simpler than PAE/LMode)
FRAME is requested, and zero'd, for use for the pageDirectory
entry is added for mapping itself into the virtual address space
entry is added for identity mapping the paging enable code
entry is added for mapping the unzero'd FRAME list
FRAME is requested, zero'd, and entries written for page Table for identity mapping paging enable code
FRAME is requested, zero'd, and entries written for mapping unzero'd FRAME list
paging is enabled
FRAME is requested, zero'd, and entries written for mapping stack
FRAMEs requested and mapped for necessary portions of page tables for identity mapping bootloader stages 4 & 5
--(stage 6 isn't loaded yet at this time)
stage 3 GetFRAME function (under UEFI this is located in stage 5) is replaced with stage 5 GetFRAME function
--(difference is primarily the later expects to work under paging, while the former does not)

4) the bootloader (stage 4) loads stage 6 and the kernel

5) from this point onwards everything runs under paging, paging is never disabled (with the exception of any necessary stage 4 shuttle-work)
there is no need to discard the initial page directory, as it is perfectly valid, however it is the page directory belonging to the bootloader-process, and is discarded when the bootloader terminates (and by that time, there are several other processes running, each of which has its own page directory)

6) when additional page tables are needed, they are mapped
a zero'd FRAME is requested (from the zero'd FRAME list, rather than from the unzero'd FRAME list)
the FRAME is mapped into the page directory as needed to represent the missing page table
the PAGE (rather than the FRAME) is written to create the necessary entries

7) when a new process is created:
a zero'd FRAME is requested to serve as page directory, and mapped to any available part of the address space
an entry is created to map itself in the normal location
entries are created to map the GLOBAL portion of kernel space
FRAMEs are requested, mapped into the address space (wherever available), wherever page tables are necessary for:
-the new process information structure
-the new process kernel stack
a zero'd FRAME is requested (and mapped into the address space) and mapped to the new kernel stack
a stack frame containing a return address pointing to an initialization function in global kernel space
a zero'd FRAME is requested (and mapped) and mapped to the new page tables for the new process information structure
the new process information structure is filled in with the appropriate information for the new process
the new page directory and page tables are all unmapped from the address space (but the FRAMEs are not released)
the new process information structure, and new kernel stack are unmapped (but the FRAMEs are not released)
the scheduler is updated to add the new process, specifying the FRAME of the new page directory as the processes CR3
--(this process only requires 5 FRAMEs)

_________________
## ---- ----- ------ Intel Manuals
OSdev wiki


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 7 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: thomtl and 20 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group