OSDev.org

The Place to Start for Operating System Developers
It is currently Mon Mar 18, 2024 10:17 pm

All times are UTC - 6 hours




Post new topic Reply to topic  [ 10 posts ] 
Author Message
 Post subject: Paging, and interpretation of the Legacy memory map
PostPosted: Mon Jan 23, 2023 5:58 am 
Offline
Member
Member
User avatar

Joined: Fri Oct 27, 2006 9:42 am
Posts: 1925
Location: Athens, GA, USA
Following on from an earlier discussion here, I am working on replacing the stub paging from my bootloader with kernel-defined paging, specifically with regards to the ACPI firmware tables.

I was hoping to determine the location of said firmware and related MMIO from the memory map I retrieved in the boot loader using INT 15h, EAX=0xE820. Unfortunately, the map has limited (for lack of a better term) resolution regarding what is where - it shows 'free' and 'reserved' memory regions, but doesn't indicate what each reserved section contains.

I know empirically from my code instrumentation (e.g., debugging print outs, QEMU logs) where the specific ACPI RSDT I need is, but that doesn't really solve the problem - how to use the memory map to locate specific firmware and hardware mappings. I can only assume I am missing, or misunderstanding, something.

On a related note, I am finding myself in a position where I am going to need to start managing free memory, but I am uncertain if it makes sense to create a kernel memory manager before I have demand paging working. The two issues seem to be independent of each other, at least in a naive implementation, but I can't help feel uncertain if one has to come before the other.

On a final note, I am also looking to remap the kernel stack to a location set in my linker script. I'm not certain just how to ensure that this will work without smashing the existing stack.

Please bear with me, I am sorry if my anxieties are getting the better of me on these matters.

_________________
Rev. First Speaker Schol-R-LEA;2 LCF ELF JAM POEE KoR KCO PPWMTF
Ordo OS Project
Lisp programmers tend to seem very odd to outsiders, just like anyone else who has had a religious experience they can't quite explain to others.


Top
 Profile  
 
 Post subject: Re: Paging, and interpretation of the Legacy memory map
PostPosted: Mon Jan 23, 2023 10:52 am 
Offline
Member
Member

Joined: Sat Nov 21, 2009 5:11 pm
Posts: 852
The memory map is only for reporting usable memory.

As described in the ACPI specification, to find the ACPI RSDT, look for "RSD PTR " in the EBDA, and if it isn't there, search 0e0000h-0fffffh, every 16 bytes. This contains a pointer to the RSDT at offset 12.

ACPI gives you the address of fixed function registers that it defines, whereas other hardware is detected through PCI and recognized based on vendor/device ID or class code, with addresses defined in PCI BARs. ACPI can also declare devices with a PNP ID and a resource descriptor.


Top
 Profile  
 
 Post subject: Re: Paging, and interpretation of the Legacy memory map
PostPosted: Mon Jan 23, 2023 11:58 am 
Offline
Member
Member

Joined: Sat Feb 04, 2012 5:03 pm
Posts: 110
To expand a little more on this -
Gigasoft wrote:
The memory map is only for reporting usable memory.

As described in the ACPI specification, to find the ACPI RSDT
It is unfortunately true that the E820 memory map can't be used to figure out which pages to map.

A typical approach then is to map pages as needed, on demand.

Load the "RSD PTR " (aka RSDP) to find the RSDT. Since you know about what size it is, map those bytes into your virtual address space and then parse the RSDT. That may take one page, or two if the RSDT crosses a page boundary. (Highly unlikely, maybe you prefer to just refuse to boot if the RSDT crosses a boundary.)

From the RSDT you will find more tables. For each one, figure out how big it is, then go and do a page mapping on that physical address, if needed. Hopefully many of the tables fall into the same small number of pages.

Keep in mind this is what ACPICA does, and you may find it easier to just link to that. Or, more specifically, ACPICA expects the underlying OS to provide it with hooks to map pages, and then ACPICA makes sure to map the virtual memory before each table is loaded.


Top
 Profile  
 
 Post subject: Re: Paging, and interpretation of the Legacy memory map
PostPosted: Mon Jan 23, 2023 12:11 pm 
Offline
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 5069
Schol-R-LEA wrote:
I can only assume I am missing, or misunderstanding, something.

The memory map isn't supposed to tell you what's in the reserved regions, just that they exist and they're not usable RAM. That way, you won't try to assign some other hardware to that address (using PCI BARs or whatever), and you won't try using that address as RAM.

If you need to access those reserved memory regions for some reason, you'll find out some other way. For example, the ACPI specification says to search a couple of those reserved regions for the RSDP (unless you're using UEFI), and the RSDP will tell you that the RSDT or XSDT is also located in a reserved region, and so on. You don't always need to map the entire reserved region, just enough to access what you're looking for.

Schol-R-LEA wrote:
On a related note, I am finding myself in a position where I am going to need to start managing free memory, but I am uncertain if it makes sense to create a kernel memory manager before I have demand paging working. The two issues seem to be independent of each other, at least in a naive implementation, but I can't help feel uncertain if one has to come before the other.

What would you do with demand paging without memory allocation? Your kernel should always know beforehand what it needs to access...

Schol-R-LEA wrote:
On a final note, I am also looking to remap the kernel stack to a location set in my linker script. I'm not certain just how to ensure that this will work without smashing the existing stack.

Easy: do it while there's nothing in the stack. For example, right before your entry code calls kernel_main().


Top
 Profile  
 
 Post subject: Re: Paging, and interpretation of the Legacy memory map
PostPosted: Mon Jan 23, 2023 12:46 pm 
Offline
Member
Member
User avatar

Joined: Fri Oct 27, 2006 9:42 am
Posts: 1925
Location: Athens, GA, USA
sounds wrote:
To expand a little more on this -
Gigasoft wrote:
The memory map is only for reporting usable memory.

As described in the ACPI specification, to find the ACPI RSDT

It is unfortunately true that the E820 memory map can't be used to figure out which pages to map.

A typical approach then is to map pages as needed, on demand.

Load the "RSD PTR " (aka RSDP) to find the RSDT. Since you know about what size it is, map those bytes into your virtual address space and then parse the RSDT. That may take one page, or two if the RSDT crosses a page boundary. (Highly unlikely, maybe you prefer to just refuse to boot if the RSDT crosses a boundary.)

From the RSDT you will find more tables. For each one, figure out how big it is, then go and do a page mapping on that physical address, if needed. Hopefully many of the tables fall into the same small number of pages.


Ah, OK then. I take it that the RSDT (or XSDT in ACPI >1.0) is more or less the key to determining what other pages need to be mapped, or at the very least, where several key ones can be found?

sounds wrote:
Keep in mind this is what ACPICA does, and you may find it easier to just link to that. Or, more specifically, ACPICA expects the underlying OS to provide it with hooks to map pages, and then ACPICA makes sure to map the virtual memory before each table is loaded.


As tempting as it is, I think I need to go through the process manually to so I can understand what is actually going on.

_________________
Rev. First Speaker Schol-R-LEA;2 LCF ELF JAM POEE KoR KCO PPWMTF
Ordo OS Project
Lisp programmers tend to seem very odd to outsiders, just like anyone else who has had a religious experience they can't quite explain to others.


Top
 Profile  
 
 Post subject: Re: Paging, and interpretation of the Legacy memory map
PostPosted: Mon Jan 23, 2023 12:52 pm 
Offline
Member
Member
User avatar

Joined: Fri Oct 27, 2006 9:42 am
Posts: 1925
Location: Athens, GA, USA
Octocontrabass wrote:
Schol-R-LEA wrote:
On a related note, I am finding myself in a position where I am going to need to start managing free memory, but I am uncertain if it makes sense to create a kernel memory manager before I have demand paging working. The two issues seem to be independent of each other, at least in a naive implementation, but I can't help feel uncertain if one has to come before the other.

What would you do with demand paging without memory allocation? Your kernel should always know beforehand what it needs to access...


Octocontrabass wrote:
Schol-R-LEA wrote:
On a final note, I am also looking to remap the kernel stack to a location set in my linker script. I'm not certain just how to ensure that this will work without smashing the existing stack.

Easy: do it while there's nothing in the stack. For example, right before your entry code calls kernel_main().

Ah, that makes sense. I assume I would need to map the intended location in the boot loader paging stub, however.

_________________
Rev. First Speaker Schol-R-LEA;2 LCF ELF JAM POEE KoR KCO PPWMTF
Ordo OS Project
Lisp programmers tend to seem very odd to outsiders, just like anyone else who has had a religious experience they can't quite explain to others.


Top
 Profile  
 
 Post subject: Re: Paging, and interpretation of the Legacy memory map
PostPosted: Mon Jan 23, 2023 1:00 pm 
Offline
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 5069
Schol-R-LEA wrote:
I assume I would need to map the intended location in the boot loader paging stub, however.

Your bootloader maps memory according to the program headers, right? Just make sure there's a program header that covers the stack area.


Top
 Profile  
 
 Post subject: Re: Paging, and interpretation of the Legacy memory map
PostPosted: Mon Jan 23, 2023 1:08 pm 
Offline
Member
Member
User avatar

Joined: Fri Oct 27, 2006 9:42 am
Posts: 1925
Location: Athens, GA, USA
On the matter of the stack, I have changed the link script thusly:

Code:
    /* set up the kernel stack well above the system tables */
    . = 0xC1000000;

    .stack :
   {
      kernel_stack_base = .;
      kernel_stack_top = . + (4M - 4);
   }


and in kstart.asm:
Code:
   extern kernel_main
   extern kernel_stack_top
   global kstart

[SECTION .text]
kstart:
        mov eax, kernel_stack_top
        mov esp, eax

        call kernel_main

   .halted_loop:
        hlt
        jmp short .halted_loop


There were no regressions from this change, so I assume that this is more or less what Octocontrabass had in mind, but I felt the need to confirm this.

Again, please be patient regarding my anxiety.

_________________
Rev. First Speaker Schol-R-LEA;2 LCF ELF JAM POEE KoR KCO PPWMTF
Ordo OS Project
Lisp programmers tend to seem very odd to outsiders, just like anyone else who has had a religious experience they can't quite explain to others.


Top
 Profile  
 
 Post subject: Re: Paging, and interpretation of the Legacy memory map
PostPosted: Mon Jan 23, 2023 1:22 pm 
Offline
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 5069
Code:
      kernel_stack_top = . + (4M - 4);
Code:
        mov eax, kernel_stack_top
        mov esp, eax

        call kernel_main

The System V psABI requires the stack to be 16-byte aligned before making a function call. Subtracting 4 here misaligns your stack; subtract 0 instead. (Or tell your compiler that kernel_main() is called with a misaligned stack.)

Also, ESP is a general-purpose register, you can MOV directly into it. Also, 4 megabytes is extremely large for a kernel stack - 4 kilobytes is usually enough.


Top
 Profile  
 
 Post subject: Re: Paging, and interpretation of the Legacy memory map
PostPosted: Mon Jan 23, 2023 1:30 pm 
Offline
Member
Member
User avatar

Joined: Fri Oct 27, 2006 9:42 am
Posts: 1925
Location: Athens, GA, USA
Ah, thank you. I'll make the changes you are recommending now.

_________________
Rev. First Speaker Schol-R-LEA;2 LCF ELF JAM POEE KoR KCO PPWMTF
Ordo OS Project
Lisp programmers tend to seem very odd to outsiders, just like anyone else who has had a religious experience they can't quite explain to others.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 10 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: Bing [Bot], SemrushBot [Bot] and 11 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group