Making a bootable image

[Bonfra]

I'm not really sure on what I have to do, is my first time doing a bootloader my own
I've followed the new wiki:

Code: Select all

$ dd if=boot.bin of=diskimage.dd conv=notrunc bs=446 count=1
$ dd if=boot.bin of=diskimage.dd conv=notrunc bs=2 count=1 skip=510 seek=510

But is just like before, qemu and bochs boot the image but i can't explore the image with poweriso and if i burn the image to a usb, the stick is no more recognized.

This is my bootloader simplified:

Code: Select all

org 0
bits 16

jmp 0x7C00 : boot

BiosPrint:
    pusha
    .loop:
        lodsb
        or al, al
        jz .done
        mov ah, 0x0E
        int 0x10
        jmp .loop
    .done:
    popa
    ret

%macro BiosPrintMacro 1
    mov si, word %1
    call BiosPrint
%endmacro

boot:
.init:
    cli ; Disable interrupts

    ; All data segments (except es) are initialized to use the code segment.
    mov ax, cs
    mov ds, ax
    mov fs, ax
    mov gs, ax

    ; Set up a temporary stack.
    xor ax, ax
    mov ss, ax
    mov sp, 0x00007C00

    ; The es segment is 0, which is useful for absolute addressing of the first 64KiB of memory.
    mov es, ax

    sti ; Re-enable interrupts.

    BiosPrintMacro Message

hang:
    cli
    hlt
    jmp hang

Message db "Hello World!", 13, 10, 0

times 510-($-$$) db 0
dw 0xAA55

[PeterX]

1.)

Code: Select all

jmp 0x7C00 : boot

This is wrong.
It must be 0x7C0:boot or 0:boot. (If you use the latter, use "org 0x7C00")

2.) a) Have you setup a valid partition table on the USB stick?
b) What is the command/tool you use for writing to the USB stick?

3.) According to this:
https://www.computerhope.com/unix/dd.htm
the dd-example with "bs=2 count=1 seek=510 skip=510" is wrong.
I think it must be "bs=1 count=2" (because otherwise it seeks/skips 510x2 bytes.)

[Bonfra]

It must be 0x7C0:boot or 0:boot. (If you use the latter, use "org 0x7C00")

Yes actually in the full bootloader i have in another file with a full memory layout

Code: Select all

 Mem.Loader1 equ 0x00007C00 

And then in the actual bootloader:

Code: Select all

jmp Mem.Loader1 >> 4 : boot

So i just simplified it wrongly.

What is the command/tool you use for writing to the USB stick?

I Use Rufus to write the image to the usb.

the dd-example with "bs=2 count=1 seek=510 skip=510" is wrong.
I think it must be "bs=1 count=2" (because otherwise it seeks/skips 510x2 bytes.)

I've tried both and both gave the same result, i tried to explore the image before running the second command and it still breaks so I think that is this one that overrides:

Code: Select all

dd if=bin/boot/boot.bin of=diskimage.dd conv=notrunc bs=446 count=1

[PeterX]

So the question remains: Have you setup a proper partition table? Or are you using the whole disk as a FAT partition? Is your disk image CD/DVD, HD or floppy disk?

I forgot this: Maybe you use UEFI? If so, in Legacy mode?

[Bonfra]

I create only one partiton as the wiki shows:

Code: Select all

$ fdisk diskimage.dd
 
Welcome to fdisk (util-linux 2.30.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
 
Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0xfa00b86e.
 
Command (m for help): g
Created a new GPT disklabel (GUID: E6B4945A-8308-448B-9ACA-0E656854CF66).
 
Command (m for help): n p
Partition number (1-128, default 1): 1
First sector (2048-262110, default 2048): 2048
Last sector, +sectors or +size{K,M,G,T,P} (2048-262110, default 262110): +8M
 
Created a new partition 1 of type 'Linux filesystem' and of size 8 MiB.
 
Command (m for help): t 1
Selected partition 1
Partition type (type L to list all types): 1
Changed type of partition 'Linux filesystem' to 'EFI System'.
 
Command (m for help): w
The partition table has been altered.Syncing disks.
$

then add a Fat file system with:

Code: Select all

mkfs.vfat -F 16 -n "EFI System" diskimage.dd

[PeterX]

$ fdisk diskimage.dd

then add a Fat file system with:

Code: Select all

mkfs.vfat -F 16 -n "EFI System" diskimage.dd

I think that's a mistake because you add the FAT FS to the whole "disk" (image) not the partition. Then the MBR (the first sector) has a partition table AND a FAT fs BPB (BIOS Parameter Block).

The command for writing the bootcode to the MBR will indeed overwrite the BPB.

You should decide for either partition table or BPB. In other words for partitioned disk or un-partitioned disk. Floppy disks are traditionally un-partitioned and HDs are traditionally partitioned. USB sticks can normally be bootable like floppy or like HD, but I guess that depends on what the BIOS supports. Or do you want to write a CD/DVD image?

You ignored several times my questions. So again: Do you use Legacy BIOS or UEFI? Do you use Legacy boot mode (CSM) or UEFI mode? Do you use OVMF in QEMU?

[Bonfra]

I think that a floppy beeing really small will not be enough for a fully developed kernel, so I want to go with HDs.
Being that HDs are partitioned I need to create a file system in the first (and for now only) partition. The wiki uses losetup but I'm using WSL and it does not implement loop module as far as I'm aware so I wrongly tried to do it with mkfs.vfat on the whole disk.
Is there a way to create the filesystem in the partiton without using losetup?

You ignored several times my questions. So again: Do you use Legacy BIOS or UEFI? Do you use Legacy boot mode (CSM) or UEFI mode? Do you use OVMF in QEMU?

I'm sorry but I don't have the slightest idea, how can i check?

Here is the repo: https://github.com/DefEnge/test-kernel
the bootloader part is in the boot direcotry
In this repo the bootloader is still the one that boot with the floppy kinda wrongly, the bootloader that i've used for testing is kinda the one I've poster before

[PeterX]

I quote https://itsfoss.com/check-uefi-or-bios/

On Windows, “System Information” in Start panel and under BIOS Mode, you can find the boot mode. If it says Legacy, your system has BIOS. If it says UEFI, well it’s UEFI.

Tell us the result of the check.

You need the file "OVMF.fd" from the Tianocore/OVMF project to emulate UEFI! Since WSL is Ubuntu AFAIK, you can install it with

Code: Select all

sudo apt install ovmf

And you use the floppy option "-fda" in the run-script. Use the HD option "-hda" instead. [EDIT: -hda isprobably wrong.]

Try this page for mounting a HD image file:
https://www.howtogeek.com/howto/windows ... ows-vista/

[Bonfra]

Following https://itsfoss.com/check-uefi-or-bios/ I can say that my physical machine does not support UEFI.

You need the file "OVMF.fd" from the Tianocore/OVMF project to emulate UEFI! Since WSL is Ubuntu AFAIK, you can installe it with

If I undertood correctrly this program is used to enable UEFI support on virtual machines but since my physical machine is no UEFI I don't need it right?

[BenLunt]

Hi,

I think what Peter is trying to express to you, since your USB is not booting on the Windows machine, he is trying to ask if the Windows machine is Legacy BIOS or UEFI firmware.
- If the Windows machine is Legacy BIOS, to boot the USB, you *must* have a MBR and then at least one partition.
- If the Windows machine is UEFI Firmware (without CSM), you *must* have a GPT and then at least one partition.

Since you have declared that the Windows machine is Legacy BIOS, let's say that you *must* have a MBR and at least one partition.

This MBR must have at least one entry marked BOOTABLE (80h) and point to the first sector of the bootable partition. This MBR must be at LBA 0.
Now let's say that your bootable partition starts at LBA 63 (legacy default, but you can have it at any LBA).
Your Bootable FAT partition, with a valid BPB, must be written to the media starting at LBA 63.
Your FAT BPB must also have the Hidden Sectors member set to 63.

It looks like you are using a *nix platform to do this, since you are using DD and other *nix utilities. I have a utility (sadly for your sake it is Windows only) that can manipulate image files with ease. http://www.fysnet.net/ultimate/index.htm

Does this help?

Ben

[Bonfra]

It looks like you are using a *nix platform to do this, since you are using DD and other *nix utilities. I have a utility (sadly for your sake it is Windows only) that can manipulate image files with ease. http://www.fysnet.net/ultimate/index.htm

Luckily I use windows, I'm able to use *nix utils thanks to WSL(windows subsystem linux) witch is a natively integrate linux terminal in windows.

Ok so to add a MBR to my bootloader reading trough this I can add at the end of my bootloader, before the magic numer and with the right padding, this:

Code: Select all

UID times 10 db 0             ; Unique Disk ID
PT1 times 16 db 0             ; First Partition Entry
PT2 times 16 db 0             ; Second Partition Entry
PT3 times 16 db 0             ; Third Partition Entry
PT4 times 16 db 0             ; Fourth Partition Entry

Can I leave PT2 PT3 and PT4 as 0 if I want only one partition?
Then I can implement the partition table as a struct:

Code: Select all

struc PartitionTable
    .Drive   resb 1
    .CHSf    resb 3
    .PType   resb 1
    .CHSl    resb 3
    .LBA     resb 4
    .NSec    resb 4
endstruc

And substitute PT1 with the implementation of the struct.
Finally i can write the MBR to the first sector of the hard disk image and it should work right?

Some final questions:
How do I create the partition in the image? the wiki creates a GPT, witch is not what I want as far as i understood. Can i just create the file system on the image with mkfs and as so create only one partition?
Do i really have to poulate the partition table entry my self or can i make them be populated by the code above and so burn a bootloader only in the first part of the MBR leaving the table there?

[bzt]

Luckily I use windows, I'm able to use *nix utils thanks to WSL(windows subsystem linux) witch is a natively integrate linux terminal in windows.

BenLunt's tool is really cool. For writing the image to an USB stick, you can also use USBImager.

Can I leave PT2 PT3 and PT4 as 0 if I want only one partition?

Yes, of course!

Finally i can write the MBR to the first sector of the hard disk image and it should work right?

Yes

How do I create the partition in the image? the wiki creates a GPT, witch is not what I want as far as i understood.

That depends. GPT is the de facto standard so I suggest to use it. You can also create a legacy MBR with fdisk (don't use the "g" command).

Can i just create the file system on the image with mkfs and as so create only one partition?

No. You have to create the file system at the starting LBA where the partitioning table (either MBR or GPT) tells to. That's why the wiki creates a /dev/loop device. For example, if that single partition starts at sector 2048, then you have to create a file system starting from offset 1M (2048*512).

Do i really have to poulate the partition table entry my self or can i make them be populated by the code above and so burn a bootloader only in the first part of the MBR leaving the table there?

You don't need to create it manually. You can use fdisk to create a GPT and take a look at my tool that I've linked which maps the ESP in GPT into the MBR. (Or as I've already said, you can use fdisk to create an MBR table in the first place.) Filling the partitioning table from code also possible and works perfectly. As a matter of fact, you can create both the MBR and the GPT table from code (however the CRC calculation requires some very crafted macro assembler like fasm).

I've created a simple tool that writes disk images (with both MBR and GPT, and also creates a FAT16/32 partition). You can use that as a skeleton for your own disk image creator if you want to. See mkbootimg. I recommend this approach as it takes into consideration how large the disk image is.

Cheers,
bzt

[Bonfra]

No. You have to create the file system at the starting LBA where the partitioning table (either MBR or GPT) tells to. That's why the wiki creates a /dev/loop device. For example, if that single partition starts at sector 2048, then you have to create a file system starting from offset 1M (2048*512).

Ok, is there another way of doing it? WSL does not have loop module and can't mount devices, so I must find another way...

[PeterX]

No. You have to create the file system at the starting LBA where the partitioning table (either MBR or GPT) tells to. That's why the wiki creates a /dev/loop device. For example, if that single partition starts at sector 2048, then you have to create a file system starting from offset 1M (2048*512).

Ok, is there another way of doing it? WSL does not have loop module and can't mount devices, so I must find another way...

Why not double-click on the image? Then AFAIK Windows mounts all partitions. I don't use Windows anymore so I don't know Win10 very well.

[BenLunt]

No. You have to create the file system at the starting LBA where the partitioning table (either MBR or GPT) tells to. That's why the wiki creates a /dev/loop device. For example, if that single partition starts at sector 2048, then you have to create a file system starting from offset 1M (2048*512).

Ok, is there another way of doing it? WSL does not have loop module and can't mount devices, so I must find another way...

Since you are using Windows, why not give my utility a try? Creating a new image is very simple. (I just updated the instructions to be more detailed)

Once the image is created, you can easily modify it for your needs.
1) You can adjust the MBR partition entries
2) You can update the boot code for each partition
3) You can insert/delete files on the partition (Currently only FAT partitions are guaranteed to work. Other file systems are experimental)

For example, you only need to create the image once. If you use my utility, it will create the MBR and place the specified image at a LBA of your choosing.
You will need to update the MBR code, but there is an option to do that.

Once you have the image file created, if you need to update a file on the bootable partition, you simply delete the old file and insert the new file.

This utility will allow you to create MBR or GPT style images. I have a detailed example for GPT images.

A few notes:
1) The examples shown at the URLs above all show a Windows XP GUI. This is simply because I think the WinXP theme is much more elegant than the new Win10 theme. However, the utility comes in both WinXP (32-bit) and Win10 (64-bit) versions.
2) Whether you choose a MBR or GPT style image, you will need to place code within the first few sectors to boot and parse the partitions. If you have a Legacy BIOS boot, both styles require some form of code to parse these partitions. If you use a UEFI boot, only the MBR requires code, though the UEFI may not recognize the partitions since it will be looking for a GPT.
3) It is perfectly acceptable, and I think it was bzt that mentioned this elsewhere, to have an image that has both a MBR and a GPT allowing for both firmware styles to boot the same image. Simply have the partition tables in the MBR point to the same partitions the GPT points to. However, this has the one assumption that all partitions are within the 32-bit sector range.

Ben

Update:

I have added detailed instructions on how to create an image file.

Start at the top, then scroll down to the type you wish to create. For the purpose of this thread, you want to scroll down to the Master Boot Record type. Once you follow those instructions, scroll down to the end of the page for "Ready to Create Image" final instructions.

Then once you have done that, go to the instructions about what to do with that MBR image.