OSDev.org

The Place to Start for Operating System Developers
It is currently Thu Mar 28, 2024 3:23 pm

All times are UTC - 6 hours




Post new topic Reply to topic  [ 31 posts ]  Go to page 1, 2, 3  Next
Author Message
 Post subject: Relocatable ELF Object Help
PostPosted: Fri Jul 16, 2021 12:55 am 
Offline
Member
Member
User avatar

Joined: Tue Sep 15, 2020 8:07 am
Posts: 264
Location: London, UK
Hi all,

Now my OS has FAT32 support, it is time to allow it to load programs from disk.

Since I don't use a virtual address space (yet...), the ELF Executable format is rather unsuited to my needs, and after reading the ELF spec it seem the basic relocatable object (as output by a simple compilation) will be fine for what I need right now.

All I need is essentially a single compiled function with access to a single external symbol (to be resolved at load time).

My test object is very simple:

Code:
typedef struct{
    void (*Call)(void);
} exec_t;

extern exec_t* sysbase;

void main(){
   
    sysbase->Call();
   
}


My ELF Loader can see the sections, and I can get the section names from the e_shstrndx section... but I'm not sure how the the relocation information is stored...

I want to load the main function into memory fixup the sysbase symbol and then call the main.

Edit: The sections are thus:
Code:
.text
.data
.bss
.text.startup
.rel.text.startup
.comment
.symtab
.strtab
.shstrtab


I'm guessing my main function will be found in section .text.startup with the reloc information at rel.text.startup

_________________
CuriOS: A single address space GUI based operating system built upon a fairly pure Microkernel/Nanokernel. Download latest bootable x86 Disk Image: https://github.com/h5n1xp/CuriOS/blob/main/disk.img.zip
Discord:https://discord.gg/zn2vV2Su


Last edited by bloodline on Fri Jul 16, 2021 7:25 am, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Relocatable ELF Object Help
PostPosted: Fri Jul 16, 2021 1:54 am 
Offline
Member
Member
User avatar

Joined: Mon Jan 15, 2018 2:27 pm
Posts: 201
It's just my opinion, but when I started with OS programming, I found that PE format (one from Windows) was easier to relocate, use dynamic linking on and understand in general. MS provides nice documentation for it as well. So you might try that instead. Look for something like 'portable executable file format' or something like that. You'll find plenty of info.

ELF is nice too but is kind of weird in a way that it is divided into segments and sections at the same time, which in turn may cause some confusion. Some features work on sections while some work on segments. Also in-memory data alignment vs. in-file alignment isn't exactly clear to me either. ELF provides more flexibility but, because of that, it can be also a little bit trickier, since there is more things to consider.

And nothing stops you from using both formats :)


Last edited by pvc on Fri Jul 16, 2021 1:58 am, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Relocatable ELF Object Help
PostPosted: Fri Jul 16, 2021 1:58 am 
Online
Member
Member
User avatar

Joined: Sat Mar 31, 2012 3:07 am
Posts: 4591
Location: Chichester, UK
https://www.intezer.com/blog/malware-an ... locations/


Top
 Profile  
 
 Post subject: Re: Relocatable ELF Object Help
PostPosted: Fri Jul 16, 2021 3:01 am 
Offline
Member
Member
User avatar

Joined: Tue Sep 15, 2020 8:07 am
Posts: 264
Location: London, UK
pvc wrote:
It's just my opinion, but when I started with OS programming, I found that PE format (one from Windows) was easier to relocate, use dynamic linking on and understand in general. MS provides nice documentation for it as well. So you might try that instead. Look for something like 'portable executable file format' or something like that. You'll find plenty of info.

ELF is nice too but is kind of weird in a way that it is divided into segments and sections at the same time, which in turn may cause some confusion. Some features work on sections while some work on segments. Also in-memory data alignment vs. in-file alignment isn't exactly clear to me either. ELF provides more flexibility but, because of that, it can be also a little bit trickier, since there is more things to consider.

And nothing stops you from using both formats :)


Yeah, I have had a look at the PE format, and it is probably better suited to my current task... But I will need ELF at some point, so now is as good a time as any to learn it, after all it is the point of my Hobby OS to learn things I wouldn't normally. :D

_________________
CuriOS: A single address space GUI based operating system built upon a fairly pure Microkernel/Nanokernel. Download latest bootable x86 Disk Image: https://github.com/h5n1xp/CuriOS/blob/main/disk.img.zip
Discord:https://discord.gg/zn2vV2Su


Top
 Profile  
 
 Post subject: Re: Relocatable ELF Object Help
PostPosted: Fri Jul 16, 2021 3:03 am 
Offline
Member
Member
User avatar

Joined: Tue Sep 15, 2020 8:07 am
Posts: 264
Location: London, UK
iansjack wrote:
https://www.intezer.com/blog/malware-analysis/executable-and-linkable-format-101-part-3-relocations/


OK, this looks pretty good! Cheers for the link, I'll have a proper read tonight.

_________________
CuriOS: A single address space GUI based operating system built upon a fairly pure Microkernel/Nanokernel. Download latest bootable x86 Disk Image: https://github.com/h5n1xp/CuriOS/blob/main/disk.img.zip
Discord:https://discord.gg/zn2vV2Su


Top
 Profile  
 
 Post subject: Re: Relocatable ELF Object Help
PostPosted: Sat Jul 17, 2021 6:18 am 
Offline
Member
Member

Joined: Wed Mar 30, 2011 12:31 am
Posts: 676
I just rewrote my relocatable object loader that I use for kernel modules today, if you want a small reference. Notably that's Elf64, but the old one was Elf32.

_________________
toaruos on github | toaruos.org | gitlab | twitter | bim - a text editor


Top
 Profile  
 
 Post subject: Re: Relocatable ELF Object Help
PostPosted: Sat Jul 17, 2021 4:18 pm 
Offline
Member
Member
User avatar

Joined: Tue Sep 15, 2020 8:07 am
Posts: 264
Location: London, UK
klange wrote:
I just rewrote my relocatable object loader that I use for kernel modules today, if you want a small reference. Notably that's Elf64, but the old one was Elf32.


Many thanks for sharing this with me, I’ll have a read over it tomorrow. I’m finding the ELF format quite a struggle… I think that actually what I want from the format are “dynamic” executables :?

_________________
CuriOS: A single address space GUI based operating system built upon a fairly pure Microkernel/Nanokernel. Download latest bootable x86 Disk Image: https://github.com/h5n1xp/CuriOS/blob/main/disk.img.zip
Discord:https://discord.gg/zn2vV2Su


Top
 Profile  
 
 Post subject: Re: Relocatable ELF Object Help
PostPosted: Sat Jul 17, 2021 4:33 pm 
Offline
Member
Member
User avatar

Joined: Fri Feb 17, 2017 4:01 pm
Posts: 640
Location: Ukraine, Bachmut
klange, so you use plain object files as loadable entities (system modules)? could you tell about your reasons of not using shared objects for this purpose? and also, I don't quite get - say a driver consists of N source code files (C files). that would generate N obj files. how then you make them into a single object file, is there some linker option for this?

since I don't want to ditch the idea of supporting MIPS, I decided to overcome the lack of PE capable compiler for it by using ELF for this architecture. And others with the same situation (others mean RISCV and PPC, haha, I know, it's ridiculously many of targets). (but if I could choose, I'd definitely go with PE, - it's to the OP).

_________________
ANT - NT-like OS for x64 and arm64.
efify - UEFI for a couple of boards (mips and arm). suspended due to lost of all the target park boards (russians destroyed our town).


Top
 Profile  
 
 Post subject: Re: Relocatable ELF Object Help
PostPosted: Sat Jul 17, 2021 4:49 pm 
Offline
Member
Member

Joined: Tue Feb 18, 2020 3:29 pm
Posts: 1071
zaval wrote:
(but if I could choose, I'd definitely go with PE, - it's to the OP).

PE is much better the ELF in some ways (mainly dynamic linking). Loading an ELF, however, is very simple, so hence they both have there advantages and disadvantages.
@bloodline - For one thing, if you build an ELF app as position independent in GCC, then you can relocate it just like a simple object file. Loading only object files will prevent growth of your OS, hence I recommend looking into PIC (position independent code). Also, if you aren't making a Unix clone, you could trying looking into PE or Mach-O. PE is probably the simplest out of the bunch. I know nothing about Mach-O, so there is no help from me there :) TL;DR Don't just look at ELF because it is the de facto standard, there are other good formats out there you should check out

_________________
"How did you do this?"
"It's very simple — you read the protocol and write the code." - Bill Joy
Projects: NexNix | libnex | nnpkg


Top
 Profile  
 
 Post subject: Re: Relocatable ELF Object Help
PostPosted: Sat Jul 17, 2021 6:17 pm 
Offline
Member
Member

Joined: Mon Jul 05, 2021 6:57 pm
Posts: 118
zaval wrote:
... say a driver consists of N source code files (C files). that would generate N obj files. how then you make them into a single object file, is there some linker option for this?

Yes, for GNU ld there is
Code:
-r


However:

bloodline wrote:
and after reading the ELF spec it seem the basic relocatable object (as output by a simple compilation) will be fine for what I need right now.

You almost definitely don't want that. Use a dynamic executable or dynamic shared library instead. The output of simple compilation is relocatable, yes, but the relocations may include many inter-object fixups which aren't necessary if you just produce an executable/shared library.

You can use `--allow-shlib-undefined` at link time if you need to refer to a symbol that will be provided by the system at load time. Alternatively, `--warn-unresolved-symbols`. I haven't experimented with this myself so YMMV, but documentation suggests this should be possible.

Compile with `-fpic` or `-fPIC` to produce a shared library. For an executable, you can use `-fpie` or `-fPIE` to produce an executable that won't need relocations, or use `-q`/`--emit-relocs` if you want to be able to perform relocations at load time.


Top
 Profile  
 
 Post subject: Re: Relocatable ELF Object Help
PostPosted: Sun Jul 18, 2021 12:00 am 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 1593
davmac314 wrote:
You almost definitely don't want that. Use a dynamic executable or dynamic shared library instead. The output of simple compilation is relocatable, yes, but the relocations may include many inter-object fixups which aren't necessary if you just produce an executable/shared library.
He did say he was making a kernel module, right? Kernel modules tend to call functions of the main kernel. So yes, inter-object fixups may be necessary.

Also, fixing up relocations isn't all that hard. Linux uses normal relocatable files for its kernel modules. You do it in two steps: First you walk through the symbol table and determine the value of each symbol (recording it in st_value of the symbol table), then you walk the relocation table and apply the relocations. (If you want to look it up, the function to do the first step is called simplify_symbols(), and the other one is called apply_relocations(), both in kernel/module.c. But apply_relocations() will call the arch-specific functions to actually do the calculations).

_________________
Carpe diem!


Top
 Profile  
 
 Post subject: Re: Relocatable ELF Object Help
PostPosted: Sun Jul 18, 2021 4:21 am 
Offline
Member
Member

Joined: Wed Mar 30, 2011 12:31 am
Posts: 676
davmac314 wrote:
You almost definitely don't want that. Use a dynamic executable or dynamic shared library instead. The output of simple compilation is relocatable, yes, but the relocations may include many inter-object fixups which aren't necessary if you just produce an executable/shared library.

Shared libraries have relocations, too. The only difference tends to be where they are and how many there are, but "how many there are" does not affect the difficulty of the task of performing them.

_________________
toaruos on github | toaruos.org | gitlab | twitter | bim - a text editor


Top
 Profile  
 
 Post subject: Re: Relocatable ELF Object Help
PostPosted: Sun Jul 18, 2021 5:27 am 
Offline
Member
Member

Joined: Mon Jul 05, 2021 6:57 pm
Posts: 118
nullplan wrote:
He did say he was making a kernel module, right?


No...

nullplan wrote:
Also, fixing up relocations isn't all that hard. Linux uses normal relocatable files for its kernel modules. You do it in two steps: First you walk through the symbol table and determine the value of each symbol (recording it in st_value of the symbol table), then you walk the relocation table and apply the relocations


Sure. It'll be a lot more relocations than if you had linked to a s.o. or executable, of course. I guess it doesn't matter that much, though.


Top
 Profile  
 
 Post subject: Re: Relocatable ELF Object Help
PostPosted: Sun Jul 18, 2021 6:53 pm 
Offline
Member
Member

Joined: Tue Mar 04, 2014 5:27 am
Posts: 1108
klange wrote:
I just rewrote my relocatable object loader that I use for kernel modules today, if you want a small reference. Notably that's Elf64, but the old one was Elf32.


So, it looks like you're loading object files, not executable ones?
This is similar to how I statically link ELF32 objects in Smaller C's linker.

Do I understand it correctly that a proper ELF executable that can be loaded anywhere (because of ASLR or lack of page translation) needs to have a whole bunch of extra sections for the dynamic loader, even where a simple object file would suffice?


Top
 Profile  
 
 Post subject: Re: Relocatable ELF Object Help
PostPosted: Sun Jul 18, 2021 7:19 pm 
Offline
Member
Member

Joined: Wed Mar 30, 2011 12:31 am
Posts: 676
alexfru wrote:
So, it looks like you're loading object files, not executable ones?
Yes, you'll find that my relocatable object loader loads relocatable objects. :wink: There is an executable loader in the function below that, though. There's also a dynamic linker but that is in userspace...
alexfru wrote:
Do I understand it correctly that a proper ELF executable that can be loaded anywhere (because of ASLR or lack of page translation) needs to have a whole bunch of extra sections for the dynamic loader, even where a simple object file would suffice?
Don't confuse position independence with dynamic executables: you can have one, the other, both, or neither. Time for some rambling:

Object files can represent any sort of code: executable, library; static, dynamic; position-independent or not... But regardless of what's in them, you absolutely need a linker to turn them into a final product - they are only an intermediary format. Object files consist only of sections, and lack the program headers found in executables and shared objects.

On most architectures, position-independent executables and libraries can actually be built statically and without any extra sections through the use of "thunks" and native IP-relative addressing schemes. On x86-32 these were notably slower at runtime than patched up absolute code, which was a very good reason to pick relocatable objects for device drivers. x86-64 and various other architectures don't suffer from the same speed impact, with x86-64 in particular introducing RIP-relative instructions.

The main benefit behind position-independent code, even when it does involve relocations - which it will in the case of a dynamic library that references other libraries as relocations are what fills in the addresses of runtime-resolved symbols - is that the relocations are moved away from the code so that the number of pages that need to be modified per-process is minimized (usually through a global offset table). This means you can "load once, run anywhere" in the sense that you can have a shared library in physical memory once and then map it into userspace memory layouts for every process that references that library, and they'll only need their own private copies of the small set of pages with relocations.

_________________
toaruos on github | toaruos.org | gitlab | twitter | bim - a text editor


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 31 posts ]  Go to page 1, 2, 3  Next

All times are UTC - 6 hours


Who is online

Users browsing this forum: 8infy, Bing [Bot], DotBot [Bot] and 68 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group