OSDev.org

Hi there, I've been trying to replace GRUB (because of its limitations and for proprietary reasons) and I was wondering how I could theoretically include say a c header from GAS assembly.
As a preface, I am building my bootloader/kernel in Linux Mint 20 in VirtualBox and I am using the Bare Bones corss compiler (i686-elf) to compile most of my sources. Every once in a while switching to NASM as it seems to have more resources on the internet. My problem is this:
I want to be able to do this


The problem is, I don't know how to compile the .S file with the C header. I found no resources on it. It seems to compile the .o when I do i686-elf-gcc -c -m32 boot.S -o boot.o, but how do I turn it into a binary that QEMU can run? Is there special magic? I tried following (to a point) from this https://www.codeproject.com/Articles/664165/Writing-a-boot-loader-in-Assembly-and-C-Part. It basically uses aAS and LD, but when I use LD it says: ld: i386 architecture of input file `boot.o' is incompatible with i386:x86-64 output.

I understand this means the .o file might not have been compiled with the right format, but what can I do to fix this?

You should compile the boot.S file with GNU AS (GAS) and not with GCC, you don't need to compile nothing more if is a only a header, if is a header with C code, you should compile the C code with GCC and link the boot.o and your C code binary with GCC or LD (I recommend using GCC). Also if you don't really know nothing you should use Grub instead.

The way you're doing it above. But Assembly can't understand C, so you're limited to the pre-compiler features. For example "#define" works as expected (because it's pre-compiler interpreted), but "enum" does not (because that's a C construct).
The GNU C pre-compiler provides an __ASSEMBLER__ macro, so that you can write your header file with #ifdefs in a way that it can be included from both Assembly and C.

Here is an example header that I include with both C and GAS (note: I declare _AS myself and I do not rely on __ASSEMBLER__ macro because I support multiple compilers, not just gcc). Both Assembly and C uses the defines from the file, but "typedef struct" which is C only is surrounded in an #ifdef guard.

This has nothing to do with includes.

Your compiler by default outputs executable code in ELF format. In order to boot something in qemu, you need to
a) convert and create a disk image with your code in it (so that you can load it with "qemu -hda boot.img")
b) or implement Linux boot protocol (so that you can use "qemu -kernel boot.o")
c) you use a boot loader which can understand ELF, so that you don't have to convert your compiled code (but chances are good you still have to create a disk image with that loader)

Yes. Check out our wiki:
Cross-compiler to see how to compile for i386 on x86_64
Bootable disk image creation to see how to convert your compiled file into a bootable disk image
Tutorials, this page has links to various pages which has all the information you need.

Cheers,
bzt

Ah, thank you for your reply. I tried to be as clear as I could, but obviously failed.

I already know about that and the C macro, that is not my concern, although it might be useful to somebody who does not know about it. I've used this method in protected mode, but recently I hasve been trying to get it to work in real mode. The problem is (perhaps my explanation is off), but I effectively want to be able to use C headers for both. That is, I can call, the parts that the assembly requires in assembly, but use the same header (in an else condition to the macro you supplied) to do things in C. If you look at GRUB for example, in the file "grub/grub-core/boot/i386/qemu/boot.S" it has a few includes at the top. config.h, grub/symbol., grub/machine/memory.h... All I want to be able to be able to do is compile the object file as always and link it similar to that link I posted without a proper linker and generate a .bin to be run by QEMU. I can handle the rest . What are the command line operators I need to do this. I've already been on the pages you linked and they have helped me not. Thanks.

Hi,

If the assembler and compiler support the same syntax for preprocessing, it is easy to combine them in one file. The assembler file isn't including a C header file, it is including another file to process as an assembler file.

Well, yes and no. Yes, you can share the same header file with both, and they both can share the same definitions. But no, you can't put Assembly and C things in that header. You have to limit yourself to a syntax that's common to both, which is not much, mostly pre-compiler stuff (parts which aren't common must be #ifdef guarded).

For example:In this case the pre-compiler takes care of it that by the time the text source is passed to the C compiler and Assembler, what they actually receive would be:
andNeither the C compiler, nor the Assembler parses "#include", and they never see "SOMETHING" either. Those are removed and replaced by the pre-compiler.

Those are nothing more than configuration defines. There's nothing Assembly nor C in them, purely pre-compiler things.

Which you can do. Include does not influence linking in any way. Think about include as you would copy'n'paste a text file into another; on the other hand linker operates on binary object files (non-text).

Which as I've said, not simple. You must wrap the generated .bin in a disk image in order to run it in qemu. No compiler will do that for you (maybe an Assembler if you code all file system information into your source with 'db' instructions). You'll need other tools than a compiler and a linker. One way is to use dd, losetup, mount etc.
(FYI: the tutorial you linked uses floppy.img as disk image, but the linker outputs test.bin in that tutorial. In section "How to copy the executable code to a bootable device and then test it?" there's (an incorrect) dd command that should update floppy.img with test.bin. The dd command is incorrect as it's missing the notrunc option. Although that tutorial uses bochs, if you did everything right, you should be able the boot floppy.img in qemu too as "qemu -fd floppy.img".)

Another way, which I prefer, is to use a specific tool that generates the disk image for you, that's what mkbootimg is about. It takes a directory and its contents (with your ELF kernel in it) as input, adds the loader to it and wraps all of it, and finally outputs a single binary file which then you can feed to qemu.

Those pages demonstrate exactly those commands. Don't get me wrong, but if those pages haven't helped you at all, then are you sure you have the required knowledge for OSDev?

Cheers,
bzt

Just now saw this and I must say that the last part of your comment wasn't very helpful. So I don't understand it. Does that mean I shouldn't try to learn? If I wanted to stick with GRUB, I'd stick with GRUB. But I do thank you for your reply.

In truth I just scan these documents. Tell me exactly where I should look for these commands. I already have everything setuop in my OS except multithreading, tasks, user mode and a buffer for my command line in the keyboard driver. I'm not starting over from "required knowledge." I started working with computers and components from the age of 12. I'm now 27. Computers, programming, digital electronics software and hardware are all I know. My problem is understanding what I read. I find it easier to look at examples, disassemble them piece by piece and create my own algorithms that do practically the same thing in a more efficient way if possible. All I asked for was the commands or at least a pointer to where I can find verbatim what needs to be used. Thanks.

Your asking for a step-by-step, and those documents provide that information. Your telling us that you've got a lot in your OS and yet you don't know how the preprocessor works and whether your binary is linked properly, and how to run it on QEMU?
You say you know computer programming. If you know computer programming as well as you claim, you should already know this information, no?
I don't mean to be presumptuous or rude, but you've put yourself in a corner. You claim that you don't know how to include a C header in an assembly file and then to link it all together. Then, you claim that you don't know how to run the generated binary image in QEMU. And yet you've got a keyboard driver and who knows what else in your OS already. So I'm kinda confused on how you even managed to implement all of that if you can't link it all together and run it to see if it even works?
Perhaps I'm misunderstanding the problem, but it feels like your giving us two different stories. I don't know how long you've worked on programming, but if you've been working with computers and components for 15 years, I posit that the majority of that time, or a lot of it, at least, was set aside for programming. Therefore the only logical conclusion that any of us can draw from your claim is that you should already know this information, especially if you've already passed the "required knowledge" test.

No I shouldn't. I built the bootloader from a tutorial somewhere else then frankenstein-ed the rest from common sense, research and as I mentioned before, disassembling other peopole's sources. Most of the work I've done in the past has been web development, application development for windows and android, game development, and things like that. I'm applying what I know from those fields and my understanding of digital electronics. Knowing how a transistor works doesn't tell me how to build what I need for Operating System development. Especially the specifics of GCC and such. Different applications and packages work differently. And as far as being backed into a corner: Nobody backs me into a corner. Also not to be rude, you might think so, but you don't know me. You can't make a claim to my knowledge without spending five minutes picking my brain. Operating system development is very specific in comparison to what I've worked on in the past and performance and getting things right is extremely important. I simply asked for the line I would need to pass to do it. I will take a look at it again, but I don't see what's so hard about doing something helpful like this:

"Here is what you need to do to create X:
i686-elf-gcc -c -m32 kernel.c -o kernel.o
This should give you what you need."

or

"It's at [insert URL here], scroll down the page and you should see X."

Here is what I actually have in my OS:
- GRUB based bootloader (to be replaces with proprietary bootloader).
- Basic Teletype (terminal)
- GDT
- IDT
- Kernel Heap
- Paging
- CPUID (For detecting Intel or AMD; For later use)
- Keyboard Driver
- Graphic Mode enabled (In NASM Assembly 800x600 32bpp)
- PIT
- IRQ's
- And my own implementation of stdarg, stdbool, stdint, stdio, stddef, and string.

I just want to replace GRUB.
Thanks.

Four of these (stdarg, stdbool, stdint, stddef) are freestanding headers, and you should not write them yourself, nor should you need to. stdarg especially can only be written in terms of compiler magic, and is therefore compiler dependent, although GCC and Clang share the same magic words here.

Then write a multiboot compliant boot loader. You do not need to link boot loader and kernel together in any way, just load the kernel as a blob, look for the multiboot header, look for the ELF header or the a.out kludge, and do what needs to be done. Keeping things separate makes it easier to debug and test each part on its own. The hard part here is going to be to load the files from the file system. The simplest way is probably going to be to go the route of LILO, and just ask the running OS for a list of blocks where the files are stored, and write that list to the boot loader. That way, you do not have to parse the file system in 16-bit real mode.

Why shouldn't you have to go through the "Required knowledge" test? As you stated, OS development is a highly specific field. The "requirements knowledge" page explicitly says:

Therefore, is it not reasonable of us to assume, based on what you've stated here, that you should already know how the preprocessor works? I would think that if I were a new member of a firmware or OS development community just starting out it would not be unreasonable for the community to expect me to know C/C++ in very good detail, including how the preprocessor works. The community might help you some, but it is expected that you will know "basics" like this (because, really, in this kind of work, how the preprocessor works is practically fundamental knowledge). Maybe I have a higher set of expectations than others do, and if so, then that's just me, I suppose.
I apologize if I'm being too hard on you, and I don't mean to do that. But knowing how your compiler works -- not in intimate detail but to a reasonable degree including the preprocessor if it has one -- just seems like common sense if your going to write an operating system of all things.

To nullplan:
I am aware of that, it's just in the past I had problems with my include path's which lead me to write very simple implementations of them. They are temporary as I don't see myself reinventing the wheel by rewriting say gcc for example.
I took that into consideration, but I just want to start with the basics and work my way up from there. Basically, I just want to do things in an oddly specific way. To be specific, I have boot.asm (at some poiunt I'd like to switch to GAS assembly, but graphic mode malfunctions I still have to look into that, but I won't be using GRUB for much longer so maybe it's a moot point?).

To Ethin:
Because I already read it and it's how I got to the building a cross compiler which is how I'm using i686-elf as I stated before.
I do understand how the preprocessor works (besides, common sense it's in the word). I started messing around with C/C++ back in 2011 when I grew tired of building websites on Wix and Bravenet and fiddling around with (at that point) Game Maker 8.0. I can respect that you have "high expectations" in this field because I feel that the gaming industry is an oversaturated market of poor quality games. I can appreciate and accept your apology, it is true, that my knowledge of cross compilers and linkers is a bit weak, but we all start from nothing. As I mentioned before, I started out when I was 12 years old. My mom was with a guy who was into computers and when I saw the inside of one for the first time, I knew that this is what I wanted to do with my life. Suffice to say, since that time I've been learning from components and voltage/current up to this. Operating System development has been my dream since I learned about FPGA's and GAL's and I realized that I too can do this seemingly impossible task of programming hardware at home. For a while I messed around with Arduino and Raspberry PI, but was still unsatisfied. Arduino uses C++ and by that point (2013) I had a pretty solid understanding of the structure of both C and C++ from a book that "coincidentally" fell on my lap. It was a hard cover book about learning C++. A text book. And I read it cover to cover. As a teenager while everybody else was going out and partying and having fun. I think it's safe to say I'm ready for the task.

Hi,

The point is that boot.asm is a separate program entirely so what is the end goal here? GrUB is a C boot loader, hence why it includes C files. If your boot loader isn't in C then there is no reason for it to be including C header files: except, perhaps, something that defines a boot protocol.

As long as boot.h only contains preprocessor directives (#include, #define, #ifdef, etc) or assembly code (and is not included by any C code) then the current configuration (using i686-elf-gcc rather than i686-elf-as) will work fine.
Remember that #include basically copy-pastes the contents of the file into that spot. If C code is pasted into assembly, or vice versa, the compiler won't like it. If you truly need to, use two separate header files.

If you're looking to call C functions from assembly, you should already be able to. LD should automatically fill in the blanks.
If you're looking to call assembly functions from C, then make sure you export the symbols using ".global function_name_here" in the assembly file.


Since i686 should already be 32-bit code, is there a reason -m32 is used?
This should work fine: "i686-elf-gcc -c boot.S -o boot.o"

Secondly, remember you are NOT linking your kernel to your bootloader. They are two separate entities.
If you are using LD the way the website does, you should be doing "i686-elf-ld -Ttext 0x7c00 --oformat=binary boot.o –o boot.bin"
MAKE SURE you are using i686-elf-ld, rather than standard ld. They are built for different architectures.

At this point you can run the .bin file as so: "qemu-system-i386 -hda boot.bin"

Good luck.