OSDev.org

I've been thinking about how to make boot loaders more flexible, and I came to a conclusion that can be the solution, although it still has some faults...

My idea is to represent bootable "objects" as a pipeline of device drivers / device mappers / filesystem drivers / network drivers / etc.


For example, if you want to refer the file /boot/kernel on the second floppy drive, who is formatted as minixfs, you would write:


If you want an uncompressed version of the file /home/someone/programming_projects/myOS/kernel-latest.img.gz from the seventh logical partition (the respective extended partition is the fourth), who is formatted as nilfs, from the first SATA disk (who has a regular PC partition table), you would write:


Even more complex: suppose you want to read a file that is inside a 7zip compressed HD image (to import through nfs from 123.456.789.012 (please pretend that this IP address is possible!)), with a BSD disklabel. The file is inside the partition number 1 of the disklabel, and is formatted as UFS. The file name is /floppy.mod:




Of course this are some very complex examples, but they show some flexible properties.

Some parts of it aren't particulary difficult to implement. We just need to represent the pipeline as program data (an array of structures, each with a callback and an argument string is enough) and write a function that builds this representation based on a string passed to it (in the form shown above). Then, the pipeline is called. To do this, we call the topmost "driver", passing the argument string and the pipeline in which it has to operate (which is the same as the original pipeline, but without the former topmost item). The process of calling the pipeline is repeated until the "driver" we are calling is a device driver who doesn't read from a pipe, but from the hardware instead...

There are cases where the thing fails a bit...
For example, to parse a PC partition table, you need to know the underlying device's geometry. But it isn't easy to get this information. A workaround could be to transform the concept of "reads from a pipeline" in "messages sent to a pipeline". This way, the interface can be easily extended in order to allow writes and ioctl-like requests. (This would be needed to support network filesystems too, because they work over a protocol, not over an array of bytes/sectors)

Another layout that I don't know how we can handle is logical volume managers. With support for ioctl-like requests, perhaps there could be created a driver that would serve as the lowlevel interface to the physical volumes. This layer would take, as arguments, the "candidates" to be physical volumes, and would reject any who isn't one (or even kill the pipeline...), and match the random keys needed to group the physical volumes in logical volumes. Then, another layer (that could be used for all LVM formats) would act as a wrapper, forwarding the reads requests to the low level layer. For example:

Or... the pipeline Syntax itself could support the concept of "lists of drivers"... But one step further we would create a programming language!

This reflection only takes into account Linux LVM, as I don't know how other LVM schemes work...



I think I will implement sort of this thing in my boot loader. There is, however, a very special reason for this not being practical for an operating system file IO interface: it does not take into account things like caching or mounts.
It could, however, be a valid option on how to organize the IO subsystem layers on the inside of a kernel, provided that the kernel inserts the cache layers in the correct places of the pipeline. (Or if the kernel doesn't want to support caching at all).

JJ

Looks like nobody liked my idea....
Or perhaps it is soooo old that nobody cares about it!

JJ

My first thought when I read it was "overkill".

I think some more background on what problem you're trying to solve would help.

You think a boot loader doesn't need to be so flexible?

JJ

I think if you want to sell your idea you have to explain why the status quo isn't good enough first...

I don't spend a lot of time thinking about bootloaders, but you wanted feedback, so there it is.

As far as I can see, all you've done is taken the GRUB command shell and put it all one one line.

GRUB:


Yours:


I don't see the difference... and personally... I prefer GRUBs. It tells it by what I want it to do, not by how to do it.

using piping in a bootloader wouldn't be very useful, but if u implemented it into your kernel would be something else. good thought tho. kudos.

1. i don't think that "sata(0) | msdospt(3) | extmsdospt(6) |" is something i'd like to type instead of "sda6", quite frankly

2. piping is useful when you are capable of providing very generic tools (gzip being one, true) and that you may want to use them in a large collection of cases. Still, there's virtually noone calling "gunzip -C stuff.tar.gz | tar xvf -" nowadays ... People know that 'tar' can work with tar -zxvf stuff.tar.gz and live happily with that. Seeing "kernel.gz" should be sufficient to guess that prior gunzipping is required, and if you really want it that way, you still could do magic signature checking as well.

Maybe it's a good thing to have separated software components that you can chain internally, but quite frankly, i think the pipe chains you mentionned (which all exclude the final interpretation such as ELF, i noticed) can be built by the "boot line interpreter"...

The motto when doing a boot system is "KISS". most people won't care if you need the whole first cylinder, but opting for NFS rather than TFTP don't make any sense as far as i'm concerned.

No? But I think.
- Sda6 doesn't give information about the format of the partition table in sda.
- Sda6 doesn't say what filesystem is in the device.
- By saying "sda", you can't say whether it refers to a USB Mass Storage, a SCSI device, a SATA or a PATA device. More or less like the BIOS, who fakes a floppy if you boot from USB..........

Why do you type /mnt/floppy/document.txt if you could type A:\document.txt?



Well, in some cases, combining block device concepts can be useful in a bootstrapping environment. Why not?



But in an operating system you are doing this sort of things many times a day. Clearly, it's not the case of boot loaders (or would you reject a boot loader who doesn't include a playable Tetris!).



Of course this could be done in the command line interpreter. And additionally you could have some syntactic sugar, if you still want to play Tetris in the boot loader and lose many times (and thus need to re-execute the game many times) .

But there is no need to do like that, cause people don't type boot command every time they want to boot their computer, or do they?...
Or didn't you know that GRUB had a configuration file? I don't believe!



The purpose of this syntax doesn't include choosing the loading method. It only aims to refer to something which can be viewed/read as files. It could be a kernel. It could be the configuration file. It could be a loadable module. It could be an initrd. It could be.........



Right. But then don't blame me if the boot system happens to be so SimpleStupid that it is unable to load a kernel which resizes in some crazy place, where the kernel you're developing happens to be...



I don't know TFTP, so I don't know whether it's better or worse than NFS (anyway, it doesn't matter, as it was only an example).
Still, if you don't like NFS and want to use TFTP instead, you can:
eth(0, 123.211.34.01) | tftp(/system/kernel.img)

Which I don't think can be easily done in GRUB...



Now it's my turn to ask you. Why the hell people need dynamically loadable modules on boot loaders, if their are to follow the "KISS" principle?
Well. GRUB2 supports (or plans to support... I don't know) loadable modules!

JJ

You still fail to present your business case clearly. What problem that isn't properly addressed in existing bootloaders can be solved elegantly by your proposed "bootloader piping"?

On your average system, a bootloader is something that is installed and then mostly (on a multiboot system) or completely (on a single boot system) forgotten.

I see the GRUB command line only about 2-3 times a year (because I set up Gentoo machines for other people, and work on various Gentoo installation scripts). It could be in suaheli for all the problems that would give me.

Even when I was actively working on a kernel project, the nitty gritty of the bootloader was handled by the Makefile. I don't see what a proposed "new abstraction" does for me. Please explain.



Attacking people doesn't help.

I don't see any sense in this "bootloader piping" stuff either. As solar says: It's installed and as long as it works, it is forgotten. I for one don't care anymore about bootloaders -- Grub does it nicely for me,so I don't need the kludgy own one that I've scrabbled together anymore.

KISS it is really in that department.

If I were you (yeah, I'm pretty well aware that I ain't.), I'd put my efforts in kernel development, in file system stuff, in a network stack or anything else which is way more interesting than bootloaders. By the way --- writing a bootloader 'd bore the bleeding bejeezes outta me. (as does the transformation of an ERD into the relational Model *rofl*)

Just my 2 $. Oh, & btw, JJ, you may want not to get in trouble, ain't you? So keep calm.

Stay safe

Well, perhaps you are right and I'm concentrating an excess of efforts in the bootloader.

I think I'll use a simpler design, but just because I want to advance to the kernel itself...

I still think that present bootloaders aren't flexible enough to cover certain specific schemes. In fact, GRUB seems to consider flexibility a reasonable goal for bootloaders, so I'm not the only one who thinks like that.
And quoting the OSDEV Wiki:


But, as no one seems to be concerned about it (although everyone uses grub........), I'll leave my ideas for now...

JJ

I have a thing for linuxBIOS.

See, the BIOS runs as soon as the machine is turned on, NOT the bootloader. The MBR is then loaded by that, and the MBR loads the bootloader, which loads the OS.

I say cut out all the middle men, use the linux hardware drivers, and load an OS in 32 (or 64) bit mode straight from the BIOS.

The most perfect way to optimize a system when it's not necessary is to remove it altogether.