OSDev.org

Hello,Resources:
http://starman.vertcomp.com/asm/mbr/DOS50FDB.htm
http://fileadmin.cs.lth.se/cs/Education ... erview.pdf
https://staff.washington.edu/dittrich/m ... gen103.pdf
http://www.eit.lth.se/fileadmin/eit/cou ... iption.pdf
http://wiki.osdev.org/FAT
https://github.com/alexfru/BootProg/blo ... boot12.asm

I suggested GrUB for the following reasons:

1. I am not convinced there is any good reason for you to not use it.
2. You can always go back and write a new boot loader after getting experience.
3. Writing a file system parser is a lot harder then reading sectors.

If you still want to proceed - then the first step is formatting the disk image to fat12, mounting it, and copying your kernel to the root directory. Then you can start writing the code for it in your boot sector. I personally use ImDisk. If using Linux, see the Wiki here. You need a properly formatted fat12 disk image before anything else.

I'm lazy to use GRUB. There. Perfect. Happy?

OK, how do I implement FAT? Is there a small example to help? Is there anything to give me a kickstart? This is what I want to know. I can leave if you want, at least for a while. I want the answer, no recommendations. That's how the forums these days can be better.

You say you don't want code, yet you want examples? Which is it? Read the FAT specification, write the code to interpret FAT and then profit.. What exactly is it that you need help with?

PS. I'm curious, which search engine is down? How many days/weeks has it been down to prevent you from searching basic stuff yourself? Why can't you use one of the other search engines while your favorite one is down?

That is completely contradictory.
You are "lazy" to use GRUB, but not lazy to code a fully fledged bootloader.
Many people (even myself) gave you a lot of examples and you're asking for it again as if there weren't any given...
"How do you implement FAT?" Again, what about those billion links you were given.
Are you actually joking or you just don't understand English well enough? I am serious.

It seems to be a common misconception amongst beginners that it's easier to write your own bootloader than to use GRUB. This is incorrect. As we've already seen here, writing your own bootloader involves not simply reading sectors from a disk, but being able to parse the filesystem, find the location of the kernel, and then load the code. Plus you then have to set up the GDT and switch to protected mode. (You have to so that first if you want to load your code above the 1MB point.) And then you have to determine how much RAM you have and where it is located.

GRUB does all that for you and more. The lazy man will choose it every time. What's not to like?

Hello,

Have you already created and formatted your disk image to fat12? You can't really do anything else until you do that. This isn't something that you can "just complete" - you need to work on it incrementally and test it every step of the way. If you need to, get some experience with bochsdbg and hex editors before attempting to write the boot loader. Use the hex editor on your disk image to see, visualize, and understand the fat12 file system disk structure since that is what you will need to parse.

So format your disk image to fat12 and copy the kernel file to the root directory. Let us know when you are done or if you are not sure about anything.

(I would typically encourage to write a simple file system parser in C on the disk image so that you can get an idea of how it works and experience with it before writing it in the boot code, but you don't appear to like C.)

I want an answer.

Hello,

The smallest example of reading a file with FAT12 would still be about 300 lines of code. You won't find anything smaller.

We provided some actual boot loaders for you to look at. We also provided lots of links to resources that describe the file system in depth. I even have tutorials that cover it (a, b, c). We can't really provide anything else. I certainly don't want to discourage, but it does not get any easier then this.

If you still want to continue with your boot loader despite the recommended advice, I applaud you for not wanting to give up and even encourage you to continue learning. But take it slowly and one step at a time. I already gave you the first step.

Yet don't provide any answers yourself... cool.

Hey @neon.
I noticed you know a lot about filesystems.
What is a filesystem actually needed for (when talking about bootloaders)? Is the answer because that is the only way to find a name of our file, or because that is the only way to determine the kernel size.
If we were to physically put a flat binary on the disk and then read it we wouldn't know how many sectors to load right? We could only assume and change it every time we compile it.
Also what about linking? I don't know if it is possible to load a pure flat binary code without using something like elf. I am talking about linking assembly bootloader with c kernel code.

Recent boot process discussions seem to have an underlying theme about the cohesion between the implementation of the filesystem services in the OS and the facilities for booting. I may be confused about that, but a lot of the opinions seem to implicitly differ on the division of responsibilities and degree of coupling between the OS tools, filesystem specification, filesystem driver implementation, and boot sector code.

I mean, is coupling the correct design strategy given the portability goals of fs storage. Shouldn't the filesystem define its storage representation in a manner which maintains foreign allocations in any kind of conformant host. And furthermore, when defining such contract (or addenda) wouldn't it be better to use the most minimalist approach, such as array of lbas in a chained list of sectors, etc, to relieve the boot code from the responsibility of supporting fs mounting and the complexities that this may impose.

Entertain the following scenario. Imagine a Windows host restores a Linux ext3 volume from backup. The Windows ext3 driver tries its best to be conform to the portable aspects of the filesystem. At the end, the disk may not be bootable, because update-grub has not run. The volume's contents are logically restored, but the observable behavior has changes, because the OS that lives on the volume has not had the chance to self-preserve. The argument can be made that ext3 doesn't have specification, but reference implementation, and thus update-grub had to be mimicked by the Windows host as well, but this is not addressing the main issue, which is the need for portability.

To illustrate the other point, lets imagine that the machine crashes. The superblock may have to be restored from a backup superblock, the journal may have to be replayed, the fs may require fsck. Would the boot code support all of those consistency features in the limited execution environment in which it operates? Shouldn't it be using different means to access the last stable version of the kernel storage on the device directly, bypassing the mounting process.

Hello,

The topic of boot loaders is independent of file systems. (That is, it is only "needed" if the operating system "needs" it.) However, the boot loader needs to know how to locate and load the different components of the operating system. If these components are stored as files, then the boot loader needs to know how to locate and load these files -- so it needs to be able to understand the file system used. Likewise, if these components are stored as raw sectors or in some other format (like a ramdisk), the boot loader would need to know how to locate it some other way.

Files systems give the boot loader (1) a consistent way to locate files on disk; (2) to obtain metainformation about the file; (3) obtain the size of the file; (4) load fragmented files; (5) debug logging; and (6) supporting virtual files & file streams. However the files themselves can provide a lot of useful information as well. I.e. you can perform dynamic linking and relocation in the boot loader if you wanted.If you just put the file across raw sectors on the disk, then it ceases to become a file. You can have "header" information in the start sector to tell the boot loader its size, however there is nothing that can prevent the header from being wrong as it would need to be consistently updated.Flat binary files don't have export/import/relocation/symbol information so linking becomes impossible. You can have a call/jump table instead of linking, but its not quite the same. You would want to use ELF, PE, or some other format that your linker supports (both PE and ELF support dynamic linking as well.)

This is sort of the crux of the argument. The real question, with regards to the boot loader, lies in how the boot sector determines which sectors to read.

In most hobbyists first attempts at a boot loader, the values are hard-coded in; if the number or location of the sectors changes, the boot sector itself has to change. It is a workable solution, and one I use in my demonstrator, but not a scalable or stable one, especially if the developer is manually updating the boot sector and/or the stage it loads.

Part of the solution, regardless of whether the disk is formatted in any particular manner or not, is to automate the process of installing the images. Writing a utility - or at least a script or a makefile - that manages the installation of both the boot sector and the next stage (regardless of what the stage is, and how many other stages may follow it), and can automatically patch any changed values into the boot loader, is an oft-overlooked aspect which, realistically, we need to emphasize more in the Wiki. You really do need to write a disk writing and formatting utility of your own, no ifs, ands, or buts, especially if you aren't using an existing file system.

(I think I will need to add something more about this to the wiki, because it is rather dishonest of us to not explain that today. In the past, it wasn't as clear, but now we have no excuses on this point.)

The question becomes, then, how does the utility know how to set or change the constants in the boot sector? The usual solution is to set aside a header (or, I suppose, a footer) at a fixed location in the boot sector itself, with information on the disk geometry and the locations and size of the sectors it next needs to load. This is exactly what the BIOS Parameter Block in the boot sectors for FAT-formatted disks is; it starts at exactly byte 4 in the boot sector of all FAT disks (the first three bytes have to contain a jump past the end of the BPB) and describes the disk in sufficient detail for the boot sector to find the next part.

For example, the VERBUM BPB is:



(And yes, this is more than just shameless self-promotion; I had it on hand and knew the names I used for the constants, so it was just the easiest one to use. This discussion does make me think I need to overhaul and extend that demonstrator, however.)

For this discussion, the most relevant constants are

• Heads for the number of read heads (always two in a floppy disk, one for each side of the 'platter', but possibly 16 or more in some hard drives);
• Sectors_Per_Track for the number of sectors per track (or cylinder, for a hard drive, but I am keeping this discussion specific to floppy disks for now); and
• Media_Descriptor for the type of disk (e.g., 5 1/4" 360KB, 3 1/2" 1.44MB), which determines both the size and the number of tracks for floppy disks.

Any boot sector looking to read an OS from a floppy disk will need these details of the drive, whether hard-coded or set when the disk is formatted - even if the disk is just a file read by the emulator. In addition, an OS trying to access a disk, bootable or otherwise, will need these data in order to determine how to read the disk, how larger it is, etc. It gets more complicated when dealing with hard drives, but the same data (and more) is needed.

One more datum that would be needed in many cases is, which sectors which are not used for files. In FAT, this is held in the Reserved_Sectors word, which is how many sectors - including the boot sector - are set aside for things like a second stage of the boot loader (or, if you want to really get crafty, the rest of the OS, but I'll discuss later why this may not be a good idea for longer-term concerns) and the File Allocation Tables.

Note that this a count - only the first n sectors can be reserved in FAT. Thus, if Reserved_Sectors is set to 16, then there are 15 reserved sectors following the boot sector.

Note also that these values are, for the most part, known when the disk is formatted and the system is installed. Historically at least, most disks didn't have a working boot sector installed, though they did have a BPB, and the first sector was still reserved.

As I will explain shortly, actual MS-DOS FAT12 disks only used the reserved sectors for the FATs; no code was stored in them. the boot sector used a bit of a trick to find the second stage file, IO.SYS, without having to parse the whole FAT. The practical upshot of this is that for FAT12,

Reserved_Sectors = 1 + (2 * Sectors_Per_FAT_Short)

and the first FAT started at sector 2.

Continuing with FAT specifically, the File Allocation Tables - of which there are usually two (with the second being a mirror of the first used if the first became corrupted) - follow immediately after the reserved sectors. This means that to find and read the FAT, you need all of the previous values, plus Sectors_Per_FAT_Short (the Sectors_Per_FAT_Long entry doesn't apply to FAT12) and Root_Entries, which indicates how many directory entries (as opposed to file entries) the root directory has allocated.

A number of the other BPB values are either specific to MS-DOS, provided for informational reasons (e.g., the disk label), or not applicable to most floppy disks or simply don't change in general (e.g. the number of sectors per cluster, which is always 1 for most floppies).

Now, I said that MS-DOS had a trick for getting the location of IO.SYS without having to parse the whole FAT. The trick was that for a bootable disk, the hidden IO.SYS (second stage boot loader and basic drivers) and MSDOS.SYS (the closest thing MS-DOS had to a kernel) files had to be the first and second entries in the FAT, and the first three sectors of IO.SYS had to be contiguous (originally, all the sectors for both had to be contiguous, and the first sector of MS-DOS.SYS had to be directly after the end of IO.SYS). This meant that the boot sector could always find the first three sectors of IO.SYS just by loading the address in the first FAT entry, and those three sectors contained the second stage boot loader.



they also allow the boot loader proper (as opposed to the boot sector) to be anywhere in the file system, provided the boot sector can find at least enough of the boot loader to load and execute it - as seen with MS-DOS and FAT. So long as the FAT boot loader for an MS-DOS system can find the first entry of the FAT, it can find the start of IO.SYS - and that's all it needed to do. There are several other ways to do this, such as actually having other reserved sectors containing a second stage boot loader capable of actually parsing a FAT, so how you do it is a matter of choice.

I've got a BPB. Does this mean I have implemented the FAT12 filesystem or a part of it?

You really want me to use GRUB...... no.
"(GASPS)"
I am not using GRUB or C.

I haven't made GRUB. GRUB is not my code. I want my own code.

OK, I can make a FAT12 image. How do I load a kernel that is more THAN 512 BYTES?

Part of what one needs, yes. The other big part you would need is the File Allocation Tables, which, if you follow the MS-DOS model, would be in the sectors immediately following the boot sector.



Fair enough. We've made the case for using an existing boot loader (of which GRUB is not the only option, BTW), and you say you don't want that. We can move past that, or at least I can - I can't make promises for the others on this group. As you might have noticed, we're a pack of stubborn SOBs, here.


Well, as I explained in that previous post, one way is to place the first FAT in the second sector on the same head and track as the boot sector, and in the first entry of that table, put a listing for the second part of the boot loader in the first entry.

You would read sector two into some arbitrary address in usable memory - it could be anywhere as long as it doesn't overwrite the boot loader or the stack - and then read the two bytes at address 0x1A (that is, bytes 26and 27 decimal) in the loaded sector to find which sector(s) to load next.

Finally, you would read however many sectors starting with the one at the location you just got, into some arbitrary location in RAM - again, anywhere works as long as it isn't overwriting what you are already using and isn't mapped to some device or ROM. Then finally you would jump to that location to run the loaded code.

This puts you back where you were, finding out how to read in more than one sector, but it hopefully gives you a lot more context for this. I don't know if it is really enough information - it is really difficult to gauge that when you aren't communicating in person - but hopefully it is.

I think I will spend some time working on VERBUM, adding full FAT12 support and documenting it in detail. I have been meaning to do more with it for some time and this is a good reason to do so.

EDIT: Wholly Carp, I didn't realize how badly I blew things with the current version of VERBUM. It's going to need a total overhaul. Sorry about that.