UEFI stuff

[scippie]

I would like to start writing my own OS again. Just for fun (although I am quite certain it will take over the world ). So I came back to this place and read some more on the wiki.
One thing I have never looked at in the past is UEFI, so now I took the time to read about it.

I am a little bit confused:
1/ I saw a tutorial for creating your own UEFI tool. I tried it and it worked fine. But: do I need this? Do I actually need to do something in this UEFI partition when I want to have my own OS?
2/ I also saw that UEFI offers functionality to query the hardware, to even do some graphical and networking stuff. How do I access this functionality? Can I access it outside of the UEFI executable? Am I correct that this functionality will ease my 'getting started' part a bit?
3/ I would like to have my OS start from USB: I installed Linux on my computers via a USB and I think I saw a UEFI structure on it. But it didn't have other partitions on it. Does this mean that if I want to create an OS that can boot on USB, that I need to write an UEFI executable that loads my OS from the same FAT-partition? Does this mean that my OS will then need to understand FAT if I want to have access to files?

[ronsor]

2. Graphics and filesystem are "boot services" -- you cant use them in your OS, unless you want your os to just be a uefi app.

3. Your os doesn't need to understand FAT, UEFI only needs a partition to boot from, you can have a seperate one without FAT

[BenLunt]

In my opinion, and this is just me, I don't like UEFI, and think it is way over-kill for what it actually should do.

A BIOS, whether it be the Legacy BIOS or the UEFI, should have little to do with the OS, just enough to access what needs to be accessed, before the OS takes control. For example, you must be able to read from the disk. If not, how are you going to read from the disk to read your driver that reads from the disk? The same goes for the screen (terminal) output. There are other things too, network booting, etc.

However, for the hobbyist like you and I, console in, console out, and disk access is really all we need. The rest should be done via the OS.

My UEFI boot code does the following using UEFI:

• 1. loads the loader which in turn, loads the kernel, and other system files.
  2. retrieves the "default" video screen modes and stores them for the kernel.
  3. switches to one of these "default" modes allowing the user to interrupt the process and choose a mode.
  4. gets the memory map.
  5. stops the UEFI services and jumps to the kernel.

That's it.

Now for a few notes.

• 1. UEFI has no text mode as you know it. For example, you can't write to memory at 0xB8000 and see text. You must use the Console out services and then your kernel *must* have a graphical text-out system already in place when it takes over. You cannot print debug messages using the Legacy style text-out via writes to screen memory at 0xB8000. (Imagine debugging your kernel's terminal/console out driver with no console out services...)
  2. To be able to get to the kernel, you must stop the UEFI services. However, once you stop the UEFI services, you no longer have any access to these services including printing to the screen and disk access.
  3. Your UEFI code is not in realmode. It is already in 32- or 64-bit protected/long mode.
  4. You already have access to all physical memory via default selectors.
  5. You can, however, load and execute (UEFI) PE files.
  6. You have to find a UEFI service before you can use that service. Therefore, you have to use services to find services to use services. uuuggg.

Requirements:

• 1. You need a compiler that will output the correct binary, linking with the correct library. I use a modified version of Alex's who is a frequent visitor here. http://www.fysnet.net/nbc.htm
  2. You need a GDT formatted drive that *must* use a FAT partition as your boot partition.
  3. (Most of) your loader/kernel files must be on this FAT partition.

Most UEFI systems will partition the drive to have a small FAT partition, hidden from other partitions, used to boot the OS. Once it boots the OS, this partition is usually not touched anymore. As for if your OS will need to know how to read from a FAT partition, most likely yes. The UEFI will take you only so far. You will need to read from the booted partition or another non-fat partition to continue loading drivers and OS items.

I am not trying to say that UEFI is bad, nor am I trying to convince you to stay with the Legacy BIOS. I just want to make sure that you understand that it is a lot of work to get to know how to use the UEFI. Something I don't recommend a newbie uses, though there have been quite a few comments on this forum about using the Legacy BIOS is a sin.

Hope that helps,
Ben
http://www.fysnet.net/osdesign_book_series.htm

[ronsor]

In my opinion, and this is just me, I don't like UEFI, and think it is way over-kill for what it actually should do.

A BIOS, whether it be the Legacy BIOS or the UEFI, should have little to do with the OS, just enough to access what needs to be accessed, before the OS takes control. For example, you must be able to read from the disk. If not, how are you going to read from the disk to read your driver that reads from the disk? The same goes for the screen (terminal) output. There are other things too, network booting, etc.

However, for the hobbyist like you and I, console in, console out, and disk access is really all we need. The rest should be done via the OS.

My UEFI boot code does the following using UEFI:

• 1. loads the loader which in turn, loads the kernel, and other system files.
  2. retrieves the "default" video screen modes and stores them for the kernel.
  3. switches to one of these "default" modes allowing the user to interrupt the process and choose a mode.
  4. gets the memory map.
  5. stops the UEFI services and jumps to the kernel.

That's it.

Now for a few notes.

• 1. UEFI has no text mode as you know it. For example, you can't write to memory at 0xB8000 and see text. You must use the Console out services and then your kernel *must* have a graphical text-out system already in place when it takes over. You cannot print debug messages using the Legacy style text-out via writes to screen memory at 0xB8000. (Imagine debugging your kernel's terminal/console out driver with no console out services...)
  2. To be able to get to the kernel, you must stop the UEFI services. However, once you stop the UEFI services, you no longer have any access to these services including printing to the screen and disk access.
  3. Your UEFI code is not in realmode. It is already in 32- or 64-bit protected/long mode.
  4. You already have access to all physical memory via default selectors.
  5. You can, however, load and execute (UEFI) PE files.
  6. You have to find a UEFI service before you can use that service. Therefore, you have to use services to find services to use services. uuuggg.

Requirements:

• 1. You need a compiler that will output the correct binary, linking with the correct library. I use a modified version of Alex's who is a frequent visitor here. http://www.fysnet.net/nbc.htm
  2. You need a GDT formatted drive that *must* use a FAT partition as your boot partition.
  3. (Most of) your loader/kernel files must be on this FAT partition.

Most UEFI systems will partition the drive to have a small FAT partition, hidden from other partitions, used to boot the OS. Once it boots the OS, this partition is usually not touched anymore. As for if your OS will need to know how to read from a FAT partition, most likely yes. The UEFI will take you only so far. You will need to read from the booted partition or another non-fat partition to continue loading drivers and OS items.

I am not trying to say that UEFI is bad, nor am I trying to convince you to stay with the Legacy BIOS. I just want to make sure that you understand that it is a lot of work to get to know how to use the UEFI. Something I don't recommend a newbie uses, though there have been quite a few comments on this forum about using the Legacy BIOS is a sin.

Hope that helps,
Ben
http://www.fysnet.net/osdesign_book_series.htm

I do not like UEFI personally -- it is over complicated -- but it has some appeal in it's boot services. Still it's Microsoft-centric design and toolchain makes it a pain.

[scippie]

In my opinion, and this is just me, I don't like UEFI, and think it is way over-kill for what it actually should do.
...
I am not trying to say that UEFI is bad, nor am I trying to convince you to stay with the Legacy BIOS. I just want to make sure that you understand that it is a lot of work to get to know how to use the UEFI. Something I don't recommend a newbie uses, though there have been quite a few comments on this forum about using the Legacy BIOS is a sin.

So if I understand you correctly: learning to use UEFI can be as much work as just talking to the hardware directly, which I will need anyway. In my previous attempts, I have always tried to avoid using the legacy BIOS as much as possible. The only thing I couldn't do myself was getting the memory map I believe (I has been a while so I am not certain).

Is there documentation about a non-bios/non-grub/non-uefi way to fetch the memory map?
Because the thought of doing it over but now completely without using L-BIOS/UEFI appeals to me, having an OS that boots from any past/present/future architecture.

Is the UEFI lib source code open? I haven't found it. Because it should show me how it is done, no?

[Octocontrabass]

Is there documentation about a non-bios/non-grub/non-uefi way to fetch the memory map?

It's impossible to get an accurate memory map on a PC without using the firmware.

[scippie]

Is there documentation about a non-bios/non-grub/non-uefi way to fetch the memory map?

It's impossible to get an accurate memory map on a PC without using the firmware.

You mean it's design dependent and the only way to retrieve it is asking the devices' implementation (firmware) for it?
So, on L-BIOS, that's through interrupt 0x15.
Through UEFI, that's a certain function call.

So I really DO need to install my own executable (preferably as small as possible) on the UEFI partition (if I don't use L-BIOS of course) to get my OS started, right?

In the 80's I would have loved to have a HD as big as the UEFI partition. What a waste

[Octocontrabass]

You mean it's design dependent and the only way to retrieve it is asking the devices' implementation (firmware) for it?

Yep. That is correct.

So I really DO need to install my own executable (preferably as small as possible) on the UEFI partition (if I don't use L-BIOS of course) to get my OS started, right?

How were you planning on booting your OS without doing that?

In the 80's I would have loved to have a HD as big as the UEFI partition. What a waste

The relative size of the EFI system partition today is comparable to a few kilobytes in the eighties. It's not so wasteful when you look at it like that.

[BenLunt]

As Octocontrabass has stated, you must use the BIOS, whether it be the Legacy or the UEFI to boot your kernel. There has to be some form of media access already in place. The chicken before the egg.

The Memory map is a must for the BIOS. Your kernel code cannot get an accurate account of the memory. The memory map is created via various forms: Boot up, BIOS code, and even hardwired at manufacture time.

As I stated before, I think the BIOS (Legacy or other) should be used as little as possible, but it is a necessity. You must use it to load from the media as well as get the memory map and a few other things.

One thing to remember, manufacturers, (no names please) are pushing to get rid of the Legacy BIOS. Once this happens, your Legacy BIOS boot will not boot. Therefore, you must have a UEFI bootable boot code and loader. However, I think this will be quite a ways down the road. UEFI machines are and have been available for some time, though these machines still have a Legacy Boot option...

Ben

[dhoelzer]

I've actually been doing quite a bit of work in the handoff from UEFI. While it is true that almost all current hardware does have a compatibility boot mode, more and more hardware is coming with that feature disabled. To avoid the user from having to work his way through setup menus that are confusing and inconsistent, I'm simply booting directly from UEFI.

The good news is that it does provide some basic FS access and direct disk block access, in addition to an interface into the graphics hardware. The bad news is that, especially on the graphics side, you will find significant inconsistencies in how UEFI has been implemented. Apple hardware, for instance, will never start in a text mode, requiring some work to figure out what's what. I've dealt with this by implementing a back-buffered frame buffer using SIMD for screen updates to keep it as device neutral as possible and have implemented a text console on top of the high-res graphics screen.

For me, my goal is to be as device/hardware agnostic as I can be. Thus far, this hasn't been too difficult to accomplish through UEFI, though figuring out the documentation can be a pain.

[rdos]

UEFI is overcomplicated, and also need a cross-compiler to even be able to produce an EFI file. It's also pretty bad that you have to support both 32-bit and 64-bit UEFIs, although today almost all UEFIs run in 64-bit mode (except some Intel Atoms that don't even have 64-bit support).

The disc access interfaces are also buggy on some platforms, not reading files as it should, and not even returning errors that something went wrong. Still, I assume this is something that will improve.

The worse problem, however, is how Windows can lock-out alternative boots, something that can be extremely hard to fix. And the hidden firmware strings should have been openly documented and possible to always edit or delete in the BIOS interface. Because of this, some laptops are forever locked-up to Windows, and not even replacing the drives will fix it.

[mikegonta]

As Octocontrabass has stated, you must use the BIOS, whether it be the Legacy or the UEFI to boot your kernel. There has to be
some form of media access already in place. The chicken before the egg.
The Memory map is a must for the BIOS. Your kernel code cannot get an accurate account of the memory. The memory map is
created via various forms: Boot up, BIOS code, and even hardwired at manufacture time.
As I stated before, I think the BIOS (Legacy or other) should be used as little as possible, but it is a necessity. You must use it to
load from the media as well as get the memory map and a few other things.
One thing to remember, manufacturers, (no names please) are pushing to get rid of the Legacy BIOS. Once this happens, your
Legacy BIOS boot will not boot. Therefore, you must have a UEFI bootable boot code and loader. However, I think this will be
quite a ways down the road. UEFI machines are and have been available for some time, though these machines still have a
Legacy Boot option...

As long as it was not a Class 3 machine (server hardware), originally UEFI required a compatibility support module for transition support.
This Classic PC BIOS CSM is still available on most if not all current PC's. Additionally there are brand new late model PC's which
either cater to DOS / Linux as well as PC's such as the intel NUC which for the most part are sold as barebones with no OS which have
the CSM (recently a few models have been released as complete PC's with WIN 10 installed). Even though intel is out of the motherboard
business the NUC is here to stay. The nice thing about the NUC is that it has the intel Visual BIOS. Plug the USB flash drive in, shut down
the machine in hibernate mode, restart the PC and enter the BIOS, click on the legacy device tab and double click on the listed USB
device to boot and run without changing any settings. CTRL-ALT-DEL and the PC restarts exactly we you were before (thanks to hibernate -
which of course is not necessary but rather an added convenience).
Works just like an emulator except it's bare hardware.

[scippie]

So I really DO need to install my own executable (preferably as small as possible) on the UEFI partition (if I don't use L-BIOS of course) to get my OS started, right?

How were you planning on booting your OS without doing that?

Well, before yesterday, the UEFI partition was something Windows specific in my mind. In my mind, a system still booted from sector 0 and Microsoft had this crazy idea to have an extra partition.

So after what I read about it here, I thought that it was more something like a new sector 0 that will find the partition (I know that GPT is used now) marked as bootable and boot it automatically. That is, unless you install a bootloader like GRUB of course.

So it wasn't such a strange thought in that way.

But now I am starting to understand that it 's something different.

I would have expected that now that it 's a partition with a file system, different OSses would be able to share that UEFI partition, but if I understand correctly, it 's still one OS that decides what will start, right?

My laptop has a dual boot with Linux and GRUB. So before that was the case and only Windows was on it, Windows had installed something on my UEFI partition? And since I installed Linux, GRUB has installed something on the UEFI partition and has replaced that Windows specific thing? GRUB then just chooses what partition's first sector to load & run?