OSDev.org

The Place to Start for Operating System Developers
It is currently Sun May 31, 2020 1:03 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 20 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: Multiple CPUs on UEFI
PostPosted: Mon Dec 02, 2019 11:15 am 
Offline

Joined: Sat May 11, 2019 2:39 pm
Posts: 1
This ancient post is the first and the only thing that Google pulls out on this topic. I'm not really surprised - utilizing multiple CPUs is an extremely rare feature in hobby OSes.

Anyway, that thread left two of my questions unanswered.
    1. With UEFI, the bootloader starts off in protected mode with A20 enabled and 4 KB identity paging, as well as a system table pointer bundle that includes ACPI which can be used to discover other cores/CPUs. When I wake up a core myself, does the firmware boot it in protected mode with A20 open, or do I have to do GRUB's work myself for the rest of the cores?
    2. How do I place the entry point and the GPT in the first 1 MiB of memory without trespassing into UEFI land?

Any suggestions or example code of how to find and wake up other cores from EFI_Main()?


Top
 Profile  
 
 Post subject: Re: Multiple CPUs on UEFI
PostPosted: Mon Dec 02, 2019 12:21 pm 
Offline
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 1967
You're not finding any results because UEFI doesn't really change how you initialize multiple CPUs. You'll want to read the Multiprocessor Specification version 1.4 if you haven't already.


Top
 Profile  
 
 Post subject: Re: Multiple CPUs on UEFI
PostPosted: Mon Dec 02, 2019 1:43 pm 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 457
To expand upon Octo's answer: If you have access to UEFI, then you have access to the System Table. And with that, access to ACPI. And with that, access to the MADT, which tells you the APIC IDs of all enabled CPUs. And how to proceed from here is something you can read in the Intel SDM, volume 3A, chapter 8.4. Or else in the AMD APM volume 2 somewhere. In all brevity, you are expected to send an INIT IPI to the target processor, wait for 10 ms, send a SIPI to it, wait for 200 µs, check to see if the CPU started, if so, assume success, else send a second SIPI and wait up to 100 ms. If the processor still is not up, it's not coming. (I would send it another INIT IPI on failure, just in case the processor is somehow up and running something I didn't ask it to. INIT IPI should stop it from executing any instructions)

Initial state of the AP will be 16-bit mode, with most registers initialized to 0, including IP. CS will be xx00, where xx is the vector of the SIPI. From there you execute the usual trampoline to end up in your OS.

Since there is only one A20 gate in the system, it will be necessarily enabled already. However, MSRs are processor specific, and thus must be reloaded with their values as needed. This includes MTRRs, if you reprogram those. I have no idea what you want with the GPT in the first MB of memory. I have a normal and a debug trampoline, and even the debug trampoline is less than 3000 bytes. Since the initialization code takes at least one page, anyway, this is all you'll need.


Top
 Profile  
 
 Post subject: Re: Multiple CPUs on UEFI
PostPosted: Tue Dec 03, 2019 2:14 am 
Offline
Member
Member
User avatar

Joined: Mon Sep 03, 2018 2:25 am
Posts: 57
There is another way, UEFI also provides a service to boot up the APs. The MpService

I don't know if its widely supported, and if it doesn't exist you have to fallback to parsing the MADT or MP tables i guess


Top
 Profile  
 
 Post subject: Re: Multiple CPUs on UEFI
PostPosted: Tue Dec 03, 2019 7:22 pm 
Offline
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 1967
I don't think you can use that service as a substitute for the ACPI MADT, it doesn't give a whole lot of information about the APs.

You certainly can't use it to boot the APs:
UEFI Platform Initialization Specification wrote:
The Protocol is available only during boot time.
...
Modules that use this protocol must guarantee that all non-blocking mode requests on all APs have been completed before the UEFI event EFI_EVENT_LEGACY_BOOT_GUID or EFI_EVENT_GROUP_EXIT_BOOT_SERVICES is signaled.


Top
 Profile  
 
 Post subject: Re: Multiple CPUs on UEFI
PostPosted: Fri Dec 06, 2019 6:24 am 
Offline
Member
Member
User avatar

Joined: Sun Oct 18, 2009 5:47 pm
Posts: 208
Location: Alexandria, Egypt | Ottawa, Canada
Octocontrabass wrote:
I don't think you can use that service as a substitute for the ACPI MADT, it doesn't give a whole lot of information about the APs.

You certainly can't use it to boot the APs:
UEFI Platform Initialization Specification wrote:
The Protocol is available only during boot time.
...
Modules that use this protocol must guarantee that all non-blocking mode requests on all APs have been completed before the UEFI event EFI_EVENT_LEGACY_BOOT_GUID or EFI_EVENT_GROUP_EXIT_BOOT_SERVICES is signaled.

I think you still can use it at boot time in order to bring the APs to a known state (protected mode with your code running on the AP and waiting for an OS-specific IPI).

If I understand correctly, I would expect the boot-loader to use MpServices in order to execute a 'dispatching user-provided function' as explained in the header file. The user function could be a simple 32-bit or 64-bit piece of code that sends EFI_EVENT to the BSP then waits for an IPI that is specific to the OS.

The BSP (running the UEFI boot-loader) waits for the event to make sure that the AP is initialized. the bootloder then exits boot services and boots into the OS, which communicates the IPIs to the APs, executes the scheduler on them, then the APs have fun together.


Top
 Profile  
 
 Post subject: Re: Multiple CPUs on UEFI
PostPosted: Sat Dec 07, 2019 1:53 am 
Offline
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 1967
According to the spec, the firmware is allowed (and perhaps expected) to reset the APs when you call ExitBootServices().

UEFI Platform Initialization Specification wrote:
After the UEFI event EFI_EVENT_LEGACY_BOOT_GUID or EFI_EVENT_GROUP_EXIT_BOOT_SERVICES is signaled, all the APs must be placed in the OS compatible CPU state as defined by the UEFI Specification. Implementations of this protocol may use the UEFI event EFI_EVENT_LEGACY_BOOT_GUID or EFI_EVENT_GROUP_EXIT_BOOT_SERVICES to force APs into the OS compatible state as defined by the UEFI Specification.

The UEFI specification says the OS compatible state for x86 is uniprocessor, which means firmware will place all APs in the wait-for-SIPI state.


Top
 Profile  
 
 Post subject: Re: Multiple CPUs on UEFI
PostPosted: Sat Dec 07, 2019 5:51 pm 
Offline
Member
Member
User avatar

Joined: Sun Oct 18, 2009 5:47 pm
Posts: 208
Location: Alexandria, Egypt | Ottawa, Canada
Octocontrabass wrote:
According to the spec, the firmware is allowed (and perhaps expected) to reset the APs when you call ExitBootServices().

UEFI Platform Initialization Specification wrote:
After the UEFI event EFI_EVENT_LEGACY_BOOT_GUID or EFI_EVENT_GROUP_EXIT_BOOT_SERVICES is signaled, all the APs must be placed in the OS compatible CPU state as defined by the UEFI Specification. Implementations of this protocol may use the UEFI event EFI_EVENT_LEGACY_BOOT_GUID or EFI_EVENT_GROUP_EXIT_BOOT_SERVICES to force APs into the OS compatible state as defined by the UEFI Specification.

The UEFI specification says the OS compatible state for x86 is uniprocessor, which means firmware will place all APs in the wait-for-SIPI state.


Thanks Octocontrabass! In that case I would rather prefer to implement a trampoline that brings the AP from real-mode to protected/long mode (just like what all people do), and never touch the APs before Exit Boot Services. I think this is the safest way.


Top
 Profile  
 
 Post subject: Re: Multiple CPUs on UEFI
PostPosted: Sat Dec 28, 2019 8:52 am 
Offline
Member
Member

Joined: Wed Oct 01, 2008 1:55 pm
Posts: 2249
Octocontrabass wrote:
You're not finding any results because UEFI doesn't really change how you initialize multiple CPUs. You'll want to read the Multiprocessor Specification version 1.4 if you haven't already.


Is there really ANY reason at all why you would bother with the Multiprocessor Specification (unless you run on really old hardware, but that is typically single-core anyway)?

My approach is that it is ACPI that provides the relevant information for how many cores & processors a machine has, and how they connect to each other and to PCI devices.


Top
 Profile  
 
 Post subject: Re: Multiple CPUs on UEFI
PostPosted: Sat Dec 28, 2019 9:26 am 
Offline
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 1967
rdos wrote:
Is there really ANY reason at all why you would bother with the Multiprocessor Specification (unless you run on really old hardware, but that is typically single-core anyway)?

Last I checked, the ACPI specification doesn't explain how to bring APs online.


Top
 Profile  
 
 Post subject: Re: Multiple CPUs on UEFI
PostPosted: Sat Dec 28, 2019 10:00 am 
Offline
Member
Member

Joined: Wed Oct 01, 2008 1:55 pm
Posts: 2249
Octocontrabass wrote:
rdos wrote:
Is there really ANY reason at all why you would bother with the Multiprocessor Specification (unless you run on really old hardware, but that is typically single-core anyway)?

Last I checked, the ACPI specification doesn't explain how to bring APs online.


That's only for informational purposes. I doesn't add anything to how to decide about how a particular system is configured.


Top
 Profile  
 
 Post subject: Re: Multiple CPUs on UEFI
PostPosted: Sat Dec 28, 2019 10:16 am 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 457
rdos wrote:
Is there really ANY reason at all why you would bother with the Multiprocessor Specification (unless you run on really old hardware, but that is typically single-core anyway)?
Perhaps, if you don't know both the Intel SDM and the AMD APM tell you how to initialize an AP if you know its local APIC ID? That used to be standardized in the MP spec only. And to this day it contains information considered relevant by historians, such as how to initialize secondary CPUs if those are 486s, which was that stuff with the INIT level de-assert, which I didn't fully absorb, either, since it is about as relevant today as a punch card.


Top
 Profile  
 
 Post subject: Re: Multiple CPUs on UEFI
PostPosted: Sat Dec 28, 2019 10:28 am 
Offline
Member
Member

Joined: Wed Oct 01, 2008 1:55 pm
Posts: 2249
The AP initialization sequence is basically covered in the APIC description (which is included in the Intel & AMD manuals). Maybe specific delays are not there, but then I'm not sure using exactly those is really necessary. Of course, that assumes the processor use the APIC, and not the legacy PIC, which has no support for AP initialization. AFAIK, 486 processors always came with PIC, and so would need some other initialization process. The inter-processor communication also is different since modern processors use the APIC for this too, and the legacy PIC doesn't support this.

So, this code should be part of the APIC driver in an OS. If some other interrupt controller comes along, then a new method can be implemented in that controller driver instead. The 486 logic will need some specific driver that knows how that worked.


Top
 Profile  
 
 Post subject: Re: Multiple CPUs on UEFI
PostPosted: Mon Dec 30, 2019 8:59 am 
Offline
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 1967
nullplan wrote:
Perhaps, if you don't know both the Intel SDM and the AMD APM tell you how to initialize an AP if you know its local APIC ID?

The Intel SDM is very large, so it's possible I missed it, but I was under the impression that it only explained how firmware should bring APs online, and didn't cover how the OS should do so after firmware has initialized everything.

nullplan wrote:
And to this day it contains information considered relevant by historians, such as how to initialize secondary CPUs if those are 486s, which was that stuff with the INIT level de-assert, which I didn't fully absorb, either, since it is about as relevant today as a punch card.

Before the APIC was integrated into the CPU, those IPIs were used to turn the reset signal on and off. You had to leave the signal on for some minimum amount of time for the CPU to reset properly, and the APIC didn't know how long that should be.

rdos wrote:
AFAIK, 486 processors always came with PIC, and so would need some other initialization process.

The 486 didn't come with any interrupt controller. It was up to the motherboard to provide APICs if it wanted to support two or more 486 processors.


Top
 Profile  
 
 Post subject: Re: Multiple CPUs on UEFI
PostPosted: Mon Dec 30, 2019 2:03 pm 
Offline
Member
Member

Joined: Wed Oct 01, 2008 1:55 pm
Posts: 2249
486 systems with APICs simply would not run most operating systems or applications of the time. This was when PIT & PIC were standard and must-haves in every PC, and programs used them extensively too, not just BIOS and the OS. When the APIC was introduced, it had a PIC emulation mode that was default after boot. More capable OSes that wanted to use multiple cores had to abondon the PIC and start using the APIC. However, when 486 chips were sold, the multiprocessor market was a small niche, and so mainstream systems contiuned to use the PIC. I still use the PIC when booting with an old single core processor regardless if it has an APIC or not. I also ignore ACPI on those systems.

There is a peculiarity about booting an AP core on a PC. Since it is the BIOS that gets control when you boot the AP, you need to tell the BIOS to jump to another location rather than the normal boot location. You do this by setting a byte in the CMOS RAM.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 20 posts ]  Go to page 1, 2  Next

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 16 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group