OSDev.org

The Place to Start for Operating System Developers
It is currently Mon Sep 23, 2019 1:45 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 12 posts ] 
Author Message
 Post subject: UEFI GetMemoryMap success with odd results
PostPosted: Mon May 21, 2018 11:56 pm 
Offline

Joined: Mon May 21, 2018 11:45 pm
Posts: 3
Hi. I've been working on a UEFI bootloader and have been having some issues with the memory map. I'm sure its just a stupid error on my part but I'd gone over this several times and would appreciate if someone might be able to point out what I'm doing wrong.

Here's the relevant code:
It succeeds with no errors but the memory map 'seems' to contain random data. At least, that's what I assume for a few reasons.
For one, the UEFI spec states that the EFI_MEMORY_DESCRIPTOR.PhysicalStart will be 4KiB aligned - I get values of 0, 7, 0, 1857182, - you get the idea.
Same situation for the VirtualStart field. I assumed since the paging is identity-mapped, the VirtualStart would either be equal to the PhysicalStart or 0. But it appears to be random.
Finally, the thing that struck me as most odd was that the EFI_MEMORY_DESCRIPTOR.Type values are large. The Type value is from an the EFI_MEMORY_TYPE enum with only a handful of values, yet I'm getting extremely high values for this field.

Again, I'm probably missing something simple, in which case I'm sorry for wasting anyone's time, but I've pored over this as well as the spec and I can't seem to find the problem.
Maybe my assumptions as the specified values are wrong? Either way, thanks.

Code:
EFI_STATUS
EFIAPI
efi_main (EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable) {
    InitializeLib(ImageHandle, SystemTable);

    EFI_STATUS status = EFI_SUCCESS;

    EFI_MEMORY_DESCRIPTOR *memory_map = NULL;
    UINT32 version = 0;
    UINTN map_key = 0;
    UINTN descriptor_size = 0;
    UINTN memory_map_size = 0;

    uefi_call_wrapper(ST->ConOut->ClearScreen, 1, ST->ConOut);

    status = uefi_call_wrapper(BS->GetMemoryMap, 5, &memory_map_size, memory_map, &map_key, &descriptor_size, &version);
    // descriptor size and version should be correct values.  memory_map_size should be needed size.

    if (status == EFI_BUFFER_TOO_SMALL) {
        UINTN encompassing_size = memory_map_size + (2 * descriptor_size);
        void *buffer = NULL;
        status = uefi_call_wrapper(BS->AllocatePool, 3, EfiLoaderData, encompassing_size, &buffer);
        if (status == EFI_SUCCESS) {
            memory_map = (EFI_MEMORY_DESCRIPTOR*) buffer;
            memory_map_size = encompassing_size;

            status = uefi_call_wrapper(BS->GetMemoryMap, 5, &memory_map_size, memory_map, &map_key, &descriptor_size, &version);
            if (status != EFI_SUCCESS) {
                Print(L"Second call to GetMemoryMap failed for some reason.");
            } else {
                Print(L"Second call to GetMemoryMap succeeded.\n");
                for (int i = 0; i < (memory_map_size / descriptor_size); ++i) {
                    Print(L"Physical Address of i-th memory descriptor:\t%x\n", memory_map[i].PhysicalStart);
                    Print(L"Virtual Address of i-th memory descriptor:\t%x\n", memory_map[i].VirtualStart);
                    Print(L"Memory Type of i-th memory descriptor:\t%d\n", memory_map[i].Type);
                    uefi_call_wrapper(BS->Stall, 1, 4000000);
                }
            }

        } else {
            Print(L"AllocatePool failure.");
        }
    } else if (status == EFI_SUCCESS) { // shouldn't happen.
        Print(L"First call to GetMemoryMap should never succeed... ???");
    } else {
        Print(L"(First) GetMemoryMap usage failure.");
    }


    return EFI_SUCCESS;
}


Top
 Profile  
 
 Post subject: Re: UEFI GetMemoryMap success with odd results
PostPosted: Tue May 22, 2018 12:10 am 
Offline
User avatar

Joined: Mon Apr 23, 2018 6:25 pm
Posts: 15
Location: San Francisco
spencerb wrote:
Code:
                for (int i = 0; i < (memory_map_size / descriptor_size); ++i) {
                    Print(L"Physical Address of i-th memory descriptor:\t%x\n", memory_map[i].PhysicalStart);
                    Print(L"Virtual Address of i-th memory descriptor:\t%x\n", memory_map[i].VirtualStart);
                    Print(L"Memory Type of i-th memory descriptor:\t%d\n", memory_map[i].Type);
                    uefi_call_wrapper(BS->Stall, 1, 4000000);
                }


In my experience, sizeof(EFI_MEMORY_DESCRIPTOR) is never the same as the returned descriptor_size. So instead of indexing like an array, you'll need to do pointer arithmetic instead.

_________________
Developing: popcorn - UEFI-booted x64 kernel


Top
 Profile  
 
 Post subject: Re: UEFI GetMemoryMap success with odd results
PostPosted: Tue May 22, 2018 7:05 am 
Offline

Joined: Mon May 21, 2018 11:45 pm
Posts: 3
That worked, thanks!
I was really banging my head on a brick wall.
Seems odd that it would behave like that. Is it padding between descriptors in the map?

Regarding forum etiquette, do I need to put a [solved] tag or anything like that?


Top
 Profile  
 
 Post subject: Re: UEFI GetMemoryMap success with odd results
PostPosted: Tue May 22, 2018 10:56 am 
Offline
User avatar

Joined: Mon Apr 23, 2018 6:25 pm
Posts: 15
Location: San Francisco
Mostly I think it's for alignment? The values I was seeing looked like they were 16-byte aligning the descriptors, even though most compilers would give them smaller alignment restrictions.

I've been posting [solved] tags on my threads when they're solved, but I haven't seen it explicitly mentioned in the rules. Helps for people searching, though.

_________________
Developing: popcorn - UEFI-booted x64 kernel


Top
 Profile  
 
 Post subject: Re: UEFI GetMemoryMap success with odd results
PostPosted: Tue May 22, 2018 11:48 am 
Offline
Member
Member
User avatar

Joined: Fri Feb 17, 2017 4:01 pm
Posts: 389
Location: Ukraine, Bachmut
seriously, do you really think that
Code:
(memory_map + i)->Type

is any different from
Code:
memory_map[i].Type

?
Of course not. It's the same. Your compiler relies on the typedef of EFI_MEMORY_DESCRIPTOR struct anyway. So if it's not as in the UEFI implementation you are working with, it will be broken in both cases.
So you did fix something, but what exactly - nobody knows. :)
As of alignment. EFI_MEMORY_DESCRIPTOR structure really is defined in a braindamaged way in the spec, namely:
Code:
typedef struct {
    UINT32 Type;
    EFI_PHYSICAL_ADDRESS PhysicalStart;
    EFI_VIRTUAL_ADDRESS VirtualStart;
    UINT64 NumberOfPages;
    UINT64 Attribute;
} EFI_MEMORY_DESCRIPTOR;

As we see, the Type is defined as UINT32 and all other members following are UINT64, so, basically compiler needs to insert a padding after the Type member. But it can't be a problem you've experienced.

And also:
Quote:
The Type value is from an the EFI_MEMORY_TYPE enum with only a handful of values, yet I'm getting extremely high values for this field.

It's not exactly that, because:
UEFI spec wrote:
MemoryType

The type of memory to allocate. The type EFI_MEMORY_TYPE is defined in “Related Definitions” below. These memory types are also described in more detail in Table 25 and Table 26. Normal allocations (that is, allocations by any UEFI application) are of type EfiLoaderData. MemoryType values in the range 0x80000000..0xFFFFFFFF are reserved for use by UEFI OS loaders that are provided by operating system vendors. The only illegal memory type values are those in the range EfiMaxMemoryType..0x7FFFFFFF.

Yes, it says that it's for "OS loaders", but who knows what the implementation you work with is "thinking", maybe it also wants to be an "OS Loader" a little. :)
Show the fixed code.

_________________
future big goal: ANT - NT-like OS for mips, arm and x86.
current smaller goal: efify - UEFI for a couple of boards (mips and arm).


Top
 Profile  
 
 Post subject: Re: UEFI GetMemoryMap success with odd results
PostPosted: Tue May 22, 2018 12:35 pm 
Offline
User avatar

Joined: Mon Apr 23, 2018 6:25 pm
Posts: 15
Location: San Francisco
zaval wrote:
seriously, do you really think that
Code:
(memory_map + i)->Type

is any different from
Code:
memory_map[i].Type

?


No. But
Code:
(EFI_MEMORY_DESCRIPTOR*)(((uint8_t*)memory_map) + descriptor_size)
is. It's easy enough to verify this for yourself - it's even true on OVMF.

_________________
Developing: popcorn - UEFI-booted x64 kernel


Top
 Profile  
 
 Post subject: Re: UEFI GetMemoryMap success with odd results
PostPosted: Tue May 22, 2018 1:15 pm 
Offline
Member
Member
User avatar

Joined: Fri Feb 17, 2017 4:01 pm
Posts: 389
Location: Ukraine, Bachmut
^ ah, you mean casting to byte pointers. when you said "using pointer arithmetics", I didn't get that you meant this. Yes, it's "easy enough". But it's ugly AF as well. And shouldn't be needed. In the ideal world.

Maybe checking for sizeof(EFI_MEMORY_DESCRIPTOR) == MemoryDescriptor (as returned by GetMemoryMap()) and/or version match are needed.
Anyway, it's not OK to f&ck up the structure without indicating this in the specification. There is no "vendor" extension allowed, only "future standard extensions".
Interesting, what numbers are in your case, OP, for:
sizeof(EFI_MEMORY_DESCRIPTOR)
MemoryDescriptor (as returned by GetMemoryMap())
EFI_MEMORY_DESCRIPTOR_VERSION as defined in headers you use
MemoryDescriptorVersion as returned by GetMemoryMap()

_________________
future big goal: ANT - NT-like OS for mips, arm and x86.
current smaller goal: efify - UEFI for a couple of boards (mips and arm).


Top
 Profile  
 
 Post subject: Re: UEFI GetMemoryMap success with odd results
PostPosted: Tue May 22, 2018 1:44 pm 
Offline
User avatar

Joined: Mon Apr 23, 2018 6:25 pm
Posts: 15
Location: San Francisco
I agree that it's ugly (I hide it behind macros, GNU-EFI does something similar), but it's definitely needed. One of the major goals of UEFI is to allow for future extensibility while retaining backward compatibility.

This is how they allow for software written against older specs to run on firmware implementing newer specs - If later versions add fields, the initial fields must remain the same, but the structure will be larger. GetMemoryMap() returns the descriptor size, so the old software can still find the start of the next structure because it knows it at run time instead of using a version hard-coded at compile time.. Also note that the specification explicitly states that it covers only the interface between firmware and software, not the implementation or memory layout of either - vendors are free to implement it how they like.

_________________
Developing: popcorn - UEFI-booted x64 kernel


Top
 Profile  
 
 Post subject: Re: UEFI GetMemoryMap success with odd results
PostPosted: Tue May 22, 2018 2:22 pm 
Offline
Member
Member
User avatar

Joined: Fri Feb 17, 2017 4:01 pm
Posts: 389
Location: Ukraine, Bachmut
EFI_MEMORY_DESCRIPTOR is the interface between FW and its clients, defined by the specification. It is not allowed to have been changed arbitrarily by implementers, only by future expansions by the specification.

I just wanted to say, that it would be better to check for version match and take advantage of this (reading new fields, bringing even more mess to your code, yay). And only as a resort, when you face something you are unaware of, you do that trickery.

But in fact all those "expansions" look not on time, there is nothing to expand, that's why it more looks like a screwing up things. Maybe the poster shows us numbers I asked for.

_________________
future big goal: ANT - NT-like OS for mips, arm and x86.
current smaller goal: efify - UEFI for a couple of boards (mips and arm).


Top
 Profile  
 
 Post subject: Re: UEFI GetMemoryMap success with odd results
PostPosted: Tue May 22, 2018 7:54 pm 
Offline

Joined: Mon May 21, 2018 11:45 pm
Posts: 3
I'm using gnu-efi for lack of a need (atm) of anything more advanced.
sizeof(EFI_MEMORY_DESCRIPTOR) is 40 bytes,
but the descriptor size returned by GetMemoryMap is 48.
The returned descriptor version and the definition in the header are both 1.

Here's the fixed code. As mentioned above, I just cast to char* and increased by descriptor_size bytes before casting back to EFI_MEMORY_DESCRIPTOR for the result.
The output makes much more sense now. The physical addresses are finally 4KiB aligned as per spec, and virtual addresses are 0.
Code:
EFI_STATUS
EFIAPI
efi_main (EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable) {
    InitializeLib(ImageHandle, SystemTable);

    EFI_STATUS status = EFI_SUCCESS;

    EFI_MEMORY_DESCRIPTOR *memory_map = NULL;
    UINT32 version = 0;
    UINTN map_key = 0;
    UINTN descriptor_size = 0;
    UINTN memory_map_size = 0;

    uefi_call_wrapper(ST->ConOut->ClearScreen, 1, ST->ConOut);

    status = uefi_call_wrapper(BS->GetMemoryMap, 5, &memory_map_size, memory_map, &map_key, &descriptor_size, &version);
    // descriptor size and version should be correct values.  memory_map_size should be needed size.

    if (status == EFI_BUFFER_TOO_SMALL) {
        UINTN encompassing_size = memory_map_size + (2 * descriptor_size);
        void *buffer = NULL;
        status = uefi_call_wrapper(BS->AllocatePool, 3, EfiLoaderData, encompassing_size, &buffer);
        if (status == EFI_SUCCESS) {
            memory_map = (EFI_MEMORY_DESCRIPTOR*) buffer;
            memory_map_size = encompassing_size;

            status = uefi_call_wrapper(BS->GetMemoryMap, 5, &memory_map_size, memory_map, &map_key, &descriptor_size, &version);
            if (status != EFI_SUCCESS) {
                Print(L"Second call to GetMemoryMap failed for some reason.");
            } else {
                Print(L"Second call to GetMemoryMap succeeded.\n");
                char *mem = (char*) memory_map;
                Print(L"sizeof(EFI_MEMORY_DESCRIPTOR):\t%d\n", sizeof(EFI_MEMORY_DESCRIPTOR));
                Print(L"descriptor_size:\t%d\n", descriptor_size);
                Print(L"EFI_MEMORY_DESCRIPTOR_VERSION = %d", version);
                for (int i = 0; i < (memory_map_size / descriptor_size); ++i) {
                    EFI_MEMORY_DESCRIPTOR desc = *( (EFI_MEMORY_DESCRIPTOR*) mem);
                    Print(L"\n");
                    Print(L"Physical Address of i-th memory descriptor:\t%x\n", desc.PhysicalStart);
                    Print(L"Virtual Address of i-th memory descriptor:\t%x\n", desc.VirtualStart);
                    Print(L"Memory Type of i-th memory descriptor:\t%d\n", desc.Type);
                    uefi_call_wrapper(BS->Stall, 1, 4000000);

                    mem += descriptor_size;
                }
            }

        } else {
            Print(L"AllocatePool failure.");
        }
    } else if (status == EFI_SUCCESS) { // shouldn't happen.
        Print(L"First call to GetMemoryMap should never succeed... ???");
    } else {
        Print(L"(First) GetMemoryMap usage failure.");
    }


    return EFI_SUCCESS;
}


Top
 Profile  
 
 Post subject: Re: UEFI GetMemoryMap success with odd results
PostPosted: Wed May 23, 2018 4:55 am 
Offline
Member
Member
User avatar

Joined: Fri Feb 17, 2017 4:01 pm
Posts: 389
Location: Ukraine, Bachmut
Thanks for your reply.
Now it's clear - the UEFI implementation you are working with has screwed up the descriptors. :evil: And this is really pissing off.
Indeed, the size of the descriptor version 1 should be 40 bytes.
Code:
/*
* typedef UINT64   EFI_PHYSICAL_ADDRESS;
* typedef UINT64   EFI_VIRTUAL_ADDRESS;
*/
typedef struct _EFI_MEMORY_DESCRIPTOR{
   UINT32         Type;
   UINT32         _Pad;
   EFI_PHYSICAL_ADDRESS   PhysicalStart;
   EFI_VIRTUAL_ADDRESS   VirtualStart;
   UINT64         NumberOfPages;
   UINT64         Attribute;
}EFI_MEMORY_DESCRIPTOR, *PEFI_MEMORY_DESCRIPTOR, **PPEFI_MEMORY_DESCRIPTOR;

_________________
future big goal: ANT - NT-like OS for mips, arm and x86.
current smaller goal: efify - UEFI for a couple of boards (mips and arm).


Top
 Profile  
 
 Post subject: Re: UEFI GetMemoryMap success with odd results
PostPosted: Wed May 23, 2018 5:57 am 
Offline
Member
Member

Joined: Sat Jul 02, 2016 7:02 am
Posts: 113
Andrew's response here.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 12 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: Google [Bot] and 8 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group