OSDev.org

The Place to Start for Operating System Developers
It is currently Thu Mar 28, 2024 2:27 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 15 posts ] 
Author Message
 Post subject: Parsing multiboot memory map.
PostPosted: Tue Apr 06, 2021 11:35 am 
Offline
Member
Member

Joined: Sun May 24, 2020 9:11 am
Posts: 61
Location: /dev/null
According to documentation memory map entry looks like this:
Code:
        +-------------------+
-4      | size              |
        +-------------------+
0       | base_addr         |
8       | length            |
16      | type              |
        +-------------------+


It only specifies four values for type:
Code:
..., where a value of 1 indicates available RAM, value of 3 indicates usable memory holding ACPI information, value of 4 indicates reserved memory which needs to be preserved on hibernation, value of 5 indicates a memory which is occupied by defective RAM modules and all other values currently indicated a reserved area.


In qemu, however, this field has different values, for example 20. Anyone know what this means?


Top
 Profile  
 
 Post subject: Re: Parsing multiboot memory map.
PostPosted: Tue Apr 06, 2021 12:39 pm 
Offline
Member
Member

Joined: Mon Feb 02, 2015 7:11 pm
Posts: 898
In the case of Grub, the values for type match the ones from the E820 memory map. This is likely true here is well.

Unfortunately I cannot seem to find what memory type 20 is. Linux doesn't seem to know (or care) either:

https://elixir.bootlin.com/linux/latest ... 20/types.h

It could have something to do with SMBIOS:

https://seabios.seabios.narkive.com/KlM ... es-in-qemu

Here it says it iss indeed something to do with SMBIOS:

Quote:
/* SMBIOS type 20 - Memory Device Mapped Address */

https://android.googlesource.com/platfo ... 4dc308a08/

_________________
https://github.com/kiznit/rainbow-os


Top
 Profile  
 
 Post subject: Re: Parsing multiboot memory map.
PostPosted: Tue Apr 06, 2021 4:26 pm 
Offline
Member
Member

Joined: Tue Feb 18, 2020 3:29 pm
Posts: 1071
How are you parsing your memory map? If you are not parsing it right, you might be getting invalid results.

_________________
"How did you do this?"
"It's very simple — you read the protocol and write the code." - Bill Joy
Projects: NexNix | libnex | nnpkg


Top
 Profile  
 
 Post subject: Re: Parsing multiboot memory map.
PostPosted: Wed Apr 07, 2021 1:04 am 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 1593
For reference (and because that offset of -4 threw me off), this is my code for that. Hope it helps:
Code:
struct mbinfo {
    uint32_t flags,
             mem_lower,
             mem_upper,
             boot_device,
             cmdline,
             mods_count,
             mods_addr,
             syms[4],
             mmap_length,
             mmap_addr,
             drives_length,
             drives_addr,
             config_table,
             bootloader_name,
             apm_table,
             vbe_control_info,
             vbe_mode_info;
    uint16_t vbe_mode,
             vbe_interface_seg,
             vbe_interface_off,
             vbe_interface_len;
    uint64_t framebuffer_addr;
    uint32_t framebuffer_pitch,
             framebuffer_width,
             framebuffer_height;
    uint8_t  framebuffer_bpp,
             framebuffer_type,
             color_info[6];

};

struct mbmmap {
    uint32_t size;
    uint32_t addr[2], len[2];
    uint8_t type;
};

static void print_mem_info(unsigned len_mmap, const struct mbmmap *mmap, uint64_t *maxram)
{
    const unsigned char *const start = (void*)mmap;
    uint64_t mr = 0;
    while ((const unsigned char*)mmap - start < len_mmap)
    {
        uint64_t addr, len;
        addr = (uint64_t)mmap->addr[1] << 32 | mmap->addr[0];
        len = (uint64_t)mmap->len[1] << 32 | mmap->len[0];
        if (mmap->type == 1 && addr + len > mr)
            mr = addr + len;
        dbg_printf("%8X-%8X type %d\n", addr, addr + len, mmap->type);
        if (addr < 0x100000000)
        {
            if (addr + len > 0x100000000)
                len = 0x100000000 - addr;

            if (mmap->type == 1)
                add_avail_range(addr, len);
            else
                add_reserved_range(addr, len);
        }

        mmap = (const void*)((const unsigned char*)mmap + mmap->size + 4);
    }
    *maxram = mr;
}

void mbmain(uint32_t mbmagic, const struct mbinfo *bootinfo)
{
    [...]
        if (bootinfo->flags & 64)
            print_mem_info(bootinfo->mmap_length, (const void*)bootinfo->mmap_addr, &maxram);
}
So the "mmap_addr" in the multiboot info points immediately to a length. And afterwards all memory maps are offset by a further four bytes. And since now everything is misaligned, I have to break the 64-bit members down into two 32-bit members, else the compiler would insert padding. Yes, I am aware of __attribute__((packed)) and I choose not to use such a crutch. And so far it is working out.

_________________
Carpe diem!


Top
 Profile  
 
 Post subject: Re: Parsing multiboot memory map.
PostPosted: Sun Apr 18, 2021 5:50 am 
Offline
Member
Member

Joined: Sun May 24, 2020 9:11 am
Posts: 61
Location: /dev/null
What is the purpose of GRUB's map?

It for example lists following regions as available in qemu:
Code:
0x0 - 0x9fc00
0x100000 - 0x7fe0000
0x0 - 0x1b00000000


Of course, this list makes no sense if its entries are not disjoint.
Moreover, in qemu's emulated memory, definitely, there is not address 0x1b000000000000. I don't even have that much memory in my PC :).

So what's the matter?
Are those these BIOS errors that bootboot fixes?


Top
 Profile  
 
 Post subject: Re: Parsing multiboot memory map.
PostPosted: Sun Apr 18, 2021 8:10 am 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 1593
antoni wrote:
Moreover, in qemu's emulated memory, definitely, there is not address 0x1b000000000000. I don't even have that much memory in my PC :).
That's a hint that you are not parsing the entries correctly. Which is curious, because the first two look alright. But they make no sense without the actual memory type. I figured out the format of the region by hexdumping all of it; maybe that helps you out.

In general, there are no guarantees about the memory map. You might have multiple entries for the same address, they might not be sorted and you must deal with that. In general, assume the worst. If the same address is given as both RAM and something else, assume it is something else.

_________________
Carpe diem!


Top
 Profile  
 
 Post subject: Re: Parsing multiboot memory map.
PostPosted: Sun Apr 18, 2021 8:22 am 
Offline
Member
Member
User avatar

Joined: Sat Mar 31, 2012 3:07 am
Posts: 4591
Location: Chichester, UK
antoni wrote:
What is the purpose of GRUB's map?
So what's the matter?

If you are using the default amount of RAM in QEMU, the highest address will be 0x8000000. Which is suspiciously close to the second range you list (allowing for the fact that some of that address space is not useable). I'd guess that your third range is reading rubbish from beyond the end of the GRUM memory map.

It is my experience that the GRUB memory map reports the correct ranges in qemu.


Top
 Profile  
 
 Post subject: Re: Parsing multiboot memory map.
PostPosted: Sun Apr 18, 2021 9:39 am 
Offline
Member
Member

Joined: Sun May 24, 2020 9:11 am
Posts: 61
Location: /dev/null
You're right,
I miscalculated length of the array because I thought one variable is (uint8_t*) but it was pointer to larger type. In correct array there are only two entries.


Top
 Profile  
 
 Post subject: Re: Parsing multiboot memory map.
PostPosted: Thu Apr 22, 2021 10:14 am 
Offline
Member
Member

Joined: Sun May 24, 2020 9:11 am
Posts: 61
Location: /dev/null
Ok,
So now the highest address in memory should be 0x7fe0000, however, according to RSDP, RSDT's address is... 0x7fe1550.

This address is correct. I mapped this page in qemu and checked this:
Code:
(gdb) p *(RSDT_t*) 0x7fe1550
$3 = {header = {signature = "RSDT", length = 52, revision = 1 '\001', checksum = 109 'm', OEMID = "BOCHS ", OEM_table_ID = "BXPCRSDT", OEM_revision = 1,
    creator_ID = 1129338946, creator_revision = 1}, pointer_to_other_SDT = {134091780}}


So, how is this possible that some ACPI tables are above higher bound of memory reported by GRUB?

PS

Region 0x7fe0000 - 0x8000000 is marked as reserved, so maybe reserved regions are also usable, but in shuch case, why they are marked as reserved?


Top
 Profile  
 
 Post subject: Re: Parsing multiboot memory map.
PostPosted: Thu Apr 22, 2021 11:37 am 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 1593
antoni wrote:
So now the highest address in memory should be 0x7fe0000, however, according to RSDP, RSDT's address is... 0x7fe1550.
I assume you mean this is the highest address of type 1, right? "Usable RAM"? Well, ACPI tables are not in usable RAM. The best you can get is "ACPI reclaimable" RAM. But some tables may be in ROM as well. "Usable RAM" would mean you can overwrite it, and you cannot overwrite the ACPI tables before reading them. "Reserved" can also mean "in use by firmware".

_________________
Carpe diem!


Top
 Profile  
 
 Post subject: Re: Parsing multiboot memory map.
PostPosted: Thu Apr 22, 2021 1:24 pm 
Offline
Member
Member

Joined: Sun May 24, 2020 9:11 am
Posts: 61
Location: /dev/null
Ok, nice.
So, I need to include RESERVED entries too while parsing map. The problem with these entries is that they are not necessarily listing real, physical memory. For example in my qemu machine, where the highest address is 0x8000000, GRUB's map looks like this:

Code:
base_addr = 0x0, length = 0x9fc00, available RAM                               
base_addr = 0x9fc00, length = 0x400, reserved RAM                             
base_addr = 0xf0000, length = 0x10000, reserved RAM                           
base_addr = 0x100000, length = 0x7ee0000, available RAM                       
base_addr = 0x7fe0000, length = 0x20000, reserved RAM                         
base_addr = 0xfffc0000, length = 0x40000, reserved RAM


So obviously there is no 0x100000000 nor 0xfffc0000 address in memory. Maybe it is memory mapped IO or something?
However, for my purposes, I need to know the actual ammount of RAM, even if some of its parts are "RESERVED".


Top
 Profile  
 
 Post subject: Re: Parsing multiboot memory map.
PostPosted: Thu Apr 22, 2021 1:45 pm 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 1593
antoni wrote:
So, I need to include RESERVED entries too while parsing map.
What for? The physical memory manager needs to know about available memory, so it can give you a new page when you need one. But the Reserved memory is not that. And if you come across some physical address you need to map (like ACPI tables of MMIO windows on PCI), you just access those directly, circumventing the PMM.
antoni wrote:
So obviously there is no 0x100000000 nor 0xfffc0000 address in memory.
Don't be so certain. Chipsets can be complicated.
antoni wrote:
However, for my purposes, I need to know the actual ammount of RAM, even if some of its parts are "RESERVED".
What for? If you want to know how much memory is usable, then only available RAM counts (and maybe ACPI reclaimable, after you've read the tables). If you want to know how much RAM is installed, you may be better served with DMI information. That contains a string about how big the memory stick is. You will never see as much RAM usable as the sticks you installed put together. Some of it always goes to other uses.

_________________
Carpe diem!


Top
 Profile  
 
 Post subject: Re: Parsing multiboot memory map.
PostPosted: Thu Apr 22, 2021 2:18 pm 
Offline
Member
Member

Joined: Sun May 24, 2020 9:11 am
Posts: 61
Location: /dev/null
Quote:
If you want to know how much RAM is installed, you may be better served with DMI information.


Yes, that is exactly what I want. But I use GRUB, so obviously I can't get any BIOS information that is not passed to me by bootloader.

However, GRUB provides me with these values:
Code:
multiboot_uint32_t mem_lower;
multiboot_uint32_t mem_upper;


I wonder if maybe total amount of memory is mem_lower + mem_upper? (these are values from BIOS int 0x12)


Top
 Profile  
 
 Post subject: Re: Parsing multiboot memory map.
PostPosted: Fri Apr 23, 2021 1:44 am 
Offline
Member
Member

Joined: Sun May 24, 2020 9:11 am
Posts: 61
Location: /dev/null
No, its not... "The value returned for upper memory is maximally the address of the first upper memory hole minus 1 megabyte. It is not guaranteed to be this value. "
How can I detect it?


Top
 Profile  
 
 Post subject: Re: Parsing multiboot memory map.
PostPosted: Fri Apr 23, 2021 12:31 pm 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 1593
antoni wrote:
Yes, that is exactly what I want.
I still don't know what for, but if it is just about displaying to the user how much memory they have, here you go. That's the DMI specification which contains, in rather lengthy detail, ways to read out all the strings associated with certain system components, RAM sticks included. Not exactly a bedtime read, but take some time to read it. I have no idea what you would use this information for. "You have 8GB of RAM installed" is rather useless information if you also concede "but I can make use of only 7.5GB". Isn't that second number more meaningful? That second number is the sum of usable RAM. That first one is what you can get from the DMI information.

_________________
Carpe diem!


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 15 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: Amazonbot [bot], Bing [Bot] and 53 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group