OSDev.org

Hi,



GRUB and multiboot has many serious design flaws. GRUB2 and multiboot2 fixes a small number of those flaws, and adds a lot more new flaws.

They're only useful for large OSs (Linux, FreeBSD, Solaris, etc; where GRUB developers customise the boot loader/manager to suit the OS) and small/limited OSs that are willing to accept the limitations. It's not a realistic option for anything in between.


Cheers,

Brendan

I'm afraid that starting from scratch without even having a reference to already made mistakes just asks for making these same mistakes and some more for shure.

I don't know why you have such an idea. I'm sure the opposite, everyone here understand what they want to make and perform necessary research. It's unfair to assume they has not done any research and thus naively thinks that they must doom to failure.

These are not one would use to differentiate different products. modules/plugin, configurations, are just mechanism for extensions, just because they may use a similar mechanism to do some work does not automatically induce they are the same product - it's the work(or feature, goal, etc) that differentiate them.

The specification is not ready yet, I'm not sure at the end how to support different file systems. It may be the VBR that handle the booting FS, then pull in drivers for other FS, I'm not sure.
Anyway, what do you expect if not (1)load FS module or (2) built the FS module into the boot loader - which in turn exceed 512B and require some kind of loading modules anyway?


Come on, it's either single stage or multiple stage, you can't have 2i +1 stage.

Wouldn't "fixing" grub then work better?

There's a difference between attempting to fix something that's mildly broken (think the chain on a bicycle) and attempting to fix something that's horribly mangled (think an entire bicycle that was run over by an 18-wheeler). One is a potentially valuable endeavour that would leave you with a nice, working bicycle. The other is fixing GRUB.

Count me in,
A bootloader shouldn't be larger than the kernel it's loading...
My only request is that we make this BSD-License Compatible.

Hi,



For my boot loaders:

• Floppy boot loader uses "starting LBA of each file" where each file is in sequential sectors starting from its starting LBA. A floppy is too small to bother trying to store more than just the required files (boot code, kernel/s, boot image, etc) and there's only a small number of these required files. An actual file system (even just FAT) is a waste of time and space.
• For PXE/network boot; you have to use TFTP and therefore there's no file system code in the boot loader (the TFTP server can use any file system it likes and it doesn't matter).
• For "no emulation El Torito", the only sane choice is ISO9660. There are extensions to ISO9660 (Joliet, Rock Ridge) but these are compatible with ISO9660 and don't provide anything a boot loader would care about (e.g. you wouldn't want to mess with Unicode file names or *nix style file permissions).

That only leaves hard drives and "hard drive like devices". For these I keep changing my mind. One alternative is to have a special boot partition, which mostly just does the "starting LBA of each file stored in sequential sectors" thing.

The other alternative is to use "n reserved sectors" at the beginning of the partition to store things needed for boot; and mostly just do the "starting LBA of each file stored in sequential sectors" thing. For this case boot code doesn't need to care about the file system itself (it only uses those reserved sectors at the start of the partition), and I can guarantee it'll work fine for any native file system I design for my OS; which is all that matters to me.

Note: I refuse to allow my OS's boot code to be stored on something like FAT where anyone can change anything using any other OS; but do plan to support these file systems (for the purpose of exchanging files between different OSs). I have no intention of supporting file systems that are designed for other OS's security models (e.g. ext2/3/4, NTFS, etc); because I don't want my OS used to violate another OS's security and it's nearly impossible to honour another OSs security model (e.g. make sure that only user "foo" on my OS can access files belonging to user "foo" on the other OS).

Basically; for how I do things, the only file system any boot loader for my OS will ever need to care about is ISO9660.

I would suggest a similar approach for a generic boot loader. If you start messing about trying to support arbitrary file systems (including file systems that haven't been designed yet) then you'd probably have to support some sort of "file system modules" (including finding a suitable executable file format for those modules, defining APIs for the modules to use, etc); and then instead of figuring out where to load the kernel from you need to figure out where to load file system module/s from (which is the same problem you were trying to fix to begin with, just in a different place).


Cheers,

Brendan

Hi,

I've been faced by the chicken&egg problem. If I try to write a specification, I think I need to reply & refer to all the posts. If I try to reply to all the posts, I keep on referring to the non-existent specification. I'll therefore draw up a rudimentary version of the specifications, over here.



That's the plan.



I have a similarly based design, and, without any further ado, here it is:

There exists roughly two stages. The general idea is that the first stage (hence called "Stage 1") would be firmware-dependent and boot-device-dependent. The next stage (hence called "Stage 2") would be both firmware-independent and boot-device-independant.

Stage 1 would first start of by doing everything which is firmware-dependent - this includes enabling A20, getting a memory map, getting VBE information, getting EDID information, finding the ACPI & MPS tables, switching to an identifiable text mode, and anything else I haven't thought of. It would then expose an API which would allow Stage 2 to access all file related functions (Open, Read, Close) and other firmware related functions (SwitchVideoMode, anything else I haven't thought of).

Stage 2 would first parse the config file. It would then do everything common - it would clean (is sanitate a word?) the memory map, find the best video mode and switch to it, make some clean & neat table to store all the data. It would then read the kernel & modules, switch on paging (or perhaps long mode), and jump to the kernel.

That is mostly an overview. The actual implementation details would be something like the following:

1. Stage 1. The Stage 1 would have the purpose of doing all things firmware-specific and boot-device-specific. There'd be two versions of this:
   
   1. BIOS.
      
      • For all things NOT-hardisk, I'll split Stage 1 into two stages (d'oh!) - Stage 1 & Stage 1.5. Stage 1 should do everything boot-device-specific, while Stage 1.5 would do everything firmware-specific. Stage 1 would provide an API (most probably in some register, so that implementation can vary) which would allow the later stages to Open, Read & Close files. This API should be generic. The idea is that different Stage 1s can be used with the same Stage 1.5. The location for these would be as follows:
        
        
        • Floppy. For floppies, we'd support FAT 12/16/32 and the no-filesystem idea described by Brendan. In case a filesystem is present, FAT supports having some number of reserved sectors. All the stages (1, 1.5 & 2) should be stored in these reserved sectors. Since we can load the rest of the sectors for Stage 1 from these reserved sectors, the size of the Stage 1 isn't limited to 512 bytes.
        • PXE.For PXE, the location of the file on disk shouldn't really matter, and it's size isn't limited either.
        • CD.For CDs, if I recall correctly, ISO9660 doesn't put the bootsector at LBA 0, but somewhere around. Moreover, while the size of the boot file can be as large as you want, most firmwares support 2KiB best. The rest of the files would be in a BOOT directory.
        Stage 1 would be therefore filesystem (err, whatever) aware. It would load Stage 1.5, pass the API and jump to it. Stage 1.5 would do everything firmware specific (set A20 pin, get memory map, etc). These two stages would operate in real mode.
        
      • For all things harddisk, things start getting a bit confused (darn you to hell, filesystem designers). The most generic idea I could gather from all the posts, is still filesystem aware (can't do without it) and limited. The idea is that the MBR & VBR would both be custom. The MBR (512 bytes) would provide all hard disk specific functions (ReadSector, etc) and good enough error detection & reporting.
        
        It'd then pass control to the VBR (512 bytes) which would provide all filesystem related functions (Open, Read, Close), and would be different for each filesystem. These both, together, form the Stage 1. These would load Stage 1.5, which would do everything firmware related. Obviously, this is severely limited. Doing disk-access in 512 bytes, and filesystem-access in 512 bytes - I'm not too sure how plausible this is (especially to do it at a good level, not something for a 512-byte competition or something) Again, everything's in real mode, here.
   2. UEFI. For UEFI, Stage 1 would exist as a single binary. As far as I've heard, if it's a 32-bit PE, then it'd be loaded in 32-bit protected+paging mode. This is what we'd do, then (for reasons described later). This shouldn't be tough - again, I'm not entirely familiar with UEFI, but I think it provides a FAT partition in every case (even CDs), so it should be generic. If anyone has more knowledge on this, please share.
2. Stage 2. Stage 2 would do everything not related with the firmware. It'd be in 32-bit flat mode (no paging), which is why we prefer 32-bit protected+paging in the UEFI loader. Obviously, there'd be change in modes a lot - from 32-bit to real mode, or from no-paging to paging. These switches would be handled by lower modes (Stage 1), and higher modes shouldn't care about them. There'd be a minimalistic configuration file, since most of the information should be specified by some sort of a binary header in the kernel. A file is still needed for things like module list, etc. Please note that the Stage 2 would be same for UEFI & BIOS. It'd load the kernel, parse everything (and stuff), switch modes finally, and jump to the kernel.

I've missed a lot of stuff in there. I haven't covered GPT at all, since I lack intimate knowledge on it. I guess someone could assist. Stuff on UEFI is lacking, and I've assumed a lot for it. Again, this should be covered as we progress.

I'll now reply to the rest of the posts (and hopefully, it should be easier). Let the criticism flow!

Regards,
Shikhin

Hi,



That... really shouldn't be required. The disk-specific code would be all same (one binary to rule 'em all), while the fs-specific code would be all different (so no need for any macros there). Unless I really missed your point again, and you meant something different...



Those would be okay - but for things like a module list, you would need a configuration file.

Yes, PIC. We might look into IOAPIC. As for dropping MP support - I don't believe we should drop support for anything. We should try to support as archaic a system as possible, so that it can fit the demands for everyone (Want support for a 486? No worries, GANDALF can do that! or Want support for my latest 512-core machine, with Microsoft's new PUFU firmware? No worries, GANDALF can do that!).



Exactly my point (please refer to design explained above).

/me skips replying to the next ~10 posts - those should be well answered by my previous design post.



I'm afraid we aren't designing GRUB here. Sorry, GNU.

Anyway, we do have some sort of a reference - multiboot. We know it's mistakes, and what it does correctly. We're learning from it, and we think we'll do everything much better.





Anyone?

Regards,
Shikhin

Hi,



What I've found is that "my boot loader" != "generic bootloader".



For PXE & "no emulation El Torito", our ideas match. On the other hand, for a floppy disk, one might or might not require a file system. I thus plan to support both (described above, in different post).



While I'd really love that idea (simple, and no hacky method), we're talking about a generic boot loader here.



My idea roughly attempts to fix that, though it still lacks. I'm not too sure how to do it with something which doesn't care about filesystem (yet works with every filesystem). Perhaps the community can think up of something ingenious.

Regards,
Shikhin

Hi,

As a quick update, I'll list down some of the things which we need to decide upon (apart from the design). I'll list down my preferences. If anyone has any other alternative, please list, and I'll surely consider it. Where there's a "??", we'll decide upon that later.

• Language choice. C and intel-style assembly.
• Code style. ??
• Tool choice. GCC, nasm and make.
• Revision control. Git & GitHub.
• License. WTFPL? However, IANAL so maybe new BSD might suit better? Surely not GPL, though. ??
• TABS vs SPACES! ??

Since there was no opposition to the name GANDALF (and it's really enticing), I think we'll go with it. #gandalf on freenode is ours to play with, and Gandalf-Team on GitHub as an organization is owned by us. If you want to be a member of that (you can either be an owner - full responsibility - or a contributor), please reply with your GitHub handle. The email gandalfgrey.team@gmail has also been registered by me, and once I've verified that you really want to participate, the password would be shared.

If there's ambiguity in anything else, please say so.

Let the game begin.

Regards,
Shikhin

In my new stage 2 boot loader I use this way. Device and FS specific stage 1 loads stage 2 kernel and device and FS specific driver from root directory. I do not care about loading other drivers because my general aim is to organize choice between "multiple versions on single volume", not more. If you need to load something from other device/FS, put into boot menu a command to load other driver or to reload boot loader with other driver.

Hi,

Another idea that everyone (including) egos seems to use. The VBR would just have enough code to load the filesystem driver, which would then, along with the MBR, form Stage 1. However, I'm not too sure if the VBR wouldn't really just become the filesystem driver (since it's got to have enough code to load the fs driver anyway). Is it worth it? What's your views?

Regards,
Shikhin