OSDev.org

The Place to Start for Operating System Developers
It is currently Thu Mar 28, 2024 2:30 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 41 posts ]  Go to page 1, 2, 3  Next
Author Message
 Post subject: Debugging binutils under my OS?
PostPosted: Sat Oct 10, 2015 5:00 pm 
Offline
Member
Member

Joined: Sat Oct 16, 2010 3:38 pm
Posts: 587
I recently ported binutils to my OS. I've already been using cross-compiling binutils running under Linux and targetting Glidix (x86_64-glidix), and they were working fine. So recently I used my cross-compiler to build a native binutils. Configuration line:

Code:
../glidix-binutils/configure --prefix=/usr --exec-prefix=/usr --host=x86_64-glidix --target=x86_64-glidix --disable-werror --disable-nls


I installed it under a destination directory (DESTDIR=/glidix-mipdir make install) then used that directory as the root of an installable package that my OS recognises. There were some symbolic links in the directory, and the package creator copied the files instead of creating links (I'm not sure how to handle symlinks in an archive creator). The symlinks were just /usr/x86_64-glidix/bin stuff, I don't think that is needed for the simple test I attempted.

Anyway, I installed the binaries under my OS, and I attempted to assemble this simple program which basically does:

Code:
write(1, "hello world\n", 12);
exit(0);


In assembly:

Code:
   .text
   .globl _start
_start:
   mov $1,         %rdi
   mov $hello_world,   %rsi
   mov $12,      %rdx
   ud2
   .hword 1
   
   mov $0,         %rdi
   ud2
   .hword 0

hello_world:
   .ascii "hello world\n"


I tried assembling under Glidix with:

Code:
as -c hello.s -o hello.o


At first it saw an instruction "text" instead of ".text" for some reason (this could be due to the fact there was no tab at the beginning of the line? I know it shouldn't happen). I then changed the configuration to the one specified above. Now it assembles without errors. But, when I look at the "hello.o" file, it is far too small (about 200 bytes smaller than the one produced by the cross-assembler under Linux) and horribly corrupt.

OK, so I attempt to skip this step to see what else is wrong. I assemble under Linux (using x86_64-glidix-as) and copy the hello.o into Glidix. Then I try linking it:

Code:
ld hello.o -o hello


Again, "hello" is too small and horribly corrupt! By horribly corrupt I mean:

Code:
Elf file type is EXEC (Executable file)
Entry point 0x400078
There are 1 program headers, starting at offset 64

Program Headers:
  Type           Offset             VirtAddr           PhysAddr
                 FileSiz            MemSiz              Flags  Align
  <unknown>: 9cc 0x010b0f0000000cc2 0x00000000c7c74800 0x6c6c656800000b0f
                 0x0a646c726f77206f 0x0000000000000000         0

Section to Segment mapping:
  Segment Sections...
   00     


But if I produce "hello" under Linux, it runs perfectly under Glidix and prints "hello world".

The problem appears to be similar in both cases: smaller and corrupt files. So I decided to trace all fopen(), fwrite(), fseek(), fclose() calls, and here's what I got (from running ld):

Code:
{fopen 'hello', mode 'w+'}
{fopen '/media/cdrom/memes/hello.o', mode 'r'}
{fseek 168:0 in 6}
{fseek 616:0 in 6}
{fseek 232:0 in 6}
{fseek 112:0 in 6}
{fseek 800:0 in 6}
{fseek 848:0 in 6}
{fseek 704:0 in 6}
{fseek 64:0 in 5}
{fwrite 0+56 -> 5}
{fseek 680:0 in 6}
{fseek 64:0 in 6}
{fwrite 56+48 -> 5}
{fseek 528:0 in 5}
{fwrite 104+216 -> 5}
{fwrite 320+1 -> 5}
{fwrite 321+27 -> 5}
{fwrite 348+12 -> 5}
{fwrite 360+7 -> 5}
{fwrite 367+12 -> 5}
{fwrite 379+7 -> 5}
{fwrite 386+5 -> 5}
{fseek 168:0 in 5}
{fwrite 168+1 -> 5}
{fwrite 169+8 -> 5}
{fwrite 177+8 -> 5}
{fwrite 185+10 -> 5}
{fwrite 195+6 -> 5}
{fseek 0:0 in 5}
{fwrite 0+64 -> 5}
{fseek 208:0 in 5}
{fwrite 208+320 -> 5}
{fclose}
{fclose}


So indeed with this series of calls, we expect the size to be 528 bytes, which is what it is - but it is this series of calls that produces the corrupt binary! When linking under Linux, the binary is 796 bytes and not corrupt.

I'm not sure exactly what could be the problem and how to debug it. Any clues?


Top
 Profile  
 
 Post subject: Re: Debugging binutils under my OS?
PostPosted: Sat Oct 10, 2015 6:04 pm 
Offline
Member
Member

Joined: Thu Mar 25, 2010 11:26 pm
Posts: 1801
Location: Melbourne, Australia
mariuszp, this is a very difficult problem. I had many similar problems with binutils and gcc. Both were producing output without showing any error but the object files were rubbish. sortie guessed that my problems were due to libc bugs. Small errors in my stdio implementation mainly. I do remember that ungetc was one problem. Which libc are you using ?

_________________
If a trainstation is where trains stop, what is a workstation ?


Top
 Profile  
 
 Post subject: Re: Debugging binutils under my OS?
PostPosted: Sat Oct 10, 2015 6:24 pm 
Offline
Member
Member

Joined: Sat Oct 16, 2010 3:38 pm
Posts: 587
gerryg400 wrote:
mariuszp, this is a very difficult problem. I had many similar problems with binutils and gcc. Both were producing output without showing any error but the object files were rubbish. sortie guessed that my problems were due to libc bugs. Small errors in my stdio implementation mainly. I do remember that ungetc was one problem. Which libc are you using ?

Custom libc.

The stdio implementation is quite simple:
https://github.com/madd-games/glidix/tr ... /src/stdio

Perhaps a second pair of eyes will notice something. Sorry for the mess.
(Now that I think of it... should fread() return the character placed by ungetc() ???)


Top
 Profile  
 
 Post subject: Re: Debugging binutils under my OS?
PostPosted: Sat Oct 10, 2015 6:30 pm 
Offline
Member
Member

Joined: Thu Mar 25, 2010 11:26 pm
Posts: 1801
Location: Melbourne, Australia
I'm not at my desk and can't remember exactly what happens with fread. After a lecture from sortie I went away and wrote unit tests for my stdio that covered all the weird corner cases in the spec. Eventually it all started to work. Getting a conforming stdio is not a small task. Mine is not there yet. I also had some subtle bugs in my scanf driver that broke things. I guess I'd look at ungetc and scanf first.

[edit] I just checked and indeed all the read functions (including fread) must use ungetc chars.

_________________
If a trainstation is where trains stop, what is a workstation ?


Last edited by gerryg400 on Sat Oct 10, 2015 6:40 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Debugging binutils under my OS?
PostPosted: Sat Oct 10, 2015 6:38 pm 
Offline
Member
Member

Joined: Sat Oct 16, 2010 3:38 pm
Posts: 587
gerryg400 wrote:
I'm not at my desk and can't remember exactly what happens with fread. After a lecture from sortie I went away and wrote unit tests for my stdio that covered all the weird corner cases in the spec. Eventually it all started to work. Getting a conforming stdio is not a small task. Mine is not there yet. I also had some subtle bugs in my scanf driver that broke things. I guess I'd look at ungetc and scanf first.


Hmm.. actually, my formatting functions are terrible. I remember scanf() failing to parse an IPv4 address ("%d.%d.%d.%d"), and printf() does not support qualifiers (it only understands single characers after '%', e.g. "%s", "%d", etc). I kept postponing proper implementation because it worked for the stuff I was working on.

I guess it is now the time to implement it properly and rigirously.


Top
 Profile  
 
 Post subject: Re: Debugging binutils under my OS?
PostPosted: Sat Oct 10, 2015 6:42 pm 
Offline
Member
Member

Joined: Thu Mar 25, 2010 11:26 pm
Posts: 1801
Location: Melbourne, Australia
Yep, it's time. It's actually a task I enjoyed. It did take a considerable time though. Perhaps a 100 hours or more to tidy up my 'working' stdio. Make certain to write unit tests or you will keep breaking stuff since lot's of the stuff is intertwined and messy.

_________________
If a trainstation is where trains stop, what is a workstation ?


Top
 Profile  
 
 Post subject: Re: Debugging binutils under my OS?
PostPosted: Sat Oct 10, 2015 6:46 pm 
Offline
Member
Member

Joined: Sat Oct 16, 2010 3:38 pm
Posts: 587
gerryg400 wrote:
Yep, it's time. It's actually a task I enjoyed. It did take a considerable time though. Perhaps a 100 hours or more to tidy up my 'working' stdio. Make certain to write unit tests or you will keep breaking stuff since lot's of the stuff is intertwined and messy.


How do you handle buffering?

I mean currently my fwrite() writes data to a buffer in RAM which is then forwarded to a write() call when calling fflush() or fclose() (but since that doesn't work when for example someone forgets to call fclose(), I had to add an fflush() call at the end of fwrite() which is just stupid).


Top
 Profile  
 
 Post subject: Re: Debugging binutils under my OS?
PostPosted: Sat Oct 10, 2015 6:50 pm 
Offline
Member
Member

Joined: Thu Mar 25, 2010 11:26 pm
Posts: 1801
Location: Melbourne, Australia
It's the programs responsibility to cal fflush or fclose. If it's not called bad luck.

_________________
If a trainstation is where trains stop, what is a workstation ?


Top
 Profile  
 
 Post subject: Re: Debugging binutils under my OS?
PostPosted: Sat Oct 10, 2015 6:51 pm 
Offline
Member
Member

Joined: Sat Oct 16, 2010 3:38 pm
Posts: 587
gerryg400 wrote:
It's the programs responsibility to cal fflush or fclose. If it's not called bad luck.

So the C standard allows me not to care about lost data if fflush() or fclose() are not called?


Top
 Profile  
 
 Post subject: Re: Debugging binutils under my OS?
PostPosted: Sat Oct 10, 2015 6:55 pm 
Offline
Member
Member

Joined: Thu Mar 25, 2010 11:26 pm
Posts: 1801
Location: Melbourne, Australia
Actually there are other ways to flush the data including reading, seeking etc. but generally yes.

_________________
If a trainstation is where trains stop, what is a workstation ?


Top
 Profile  
 
 Post subject: Re: Debugging binutils under my OS?
PostPosted: Sun Oct 11, 2015 12:57 am 
Offline
Member
Member
User avatar

Joined: Thu Mar 27, 2014 3:57 am
Posts: 568
Location: Moscow, Russia
AFAIK, on a "normal" exit (the exit function) all streams must be flushed implicitly by the implementation, they may not be flushed on an "abnormal" exit (abort, _Exit, quick_exit).

_________________
"If you don't fail at least 90 percent of the time, you're not aiming high enough."
- Alan Kay


Top
 Profile  
 
 Post subject: Re: Debugging binutils under my OS?
PostPosted: Sun Oct 11, 2015 1:13 am 
Offline
Member
Member

Joined: Thu Mar 25, 2010 11:26 pm
Posts: 1801
Location: Melbourne, Australia
Roman, I think you're right.

_________________
If a trainstation is where trains stop, what is a workstation ?


Top
 Profile  
 
 Post subject: Re: Debugging binutils under my OS?
PostPosted: Sun Oct 11, 2015 5:19 am 
Offline
Member
Member

Joined: Sat Oct 16, 2010 3:38 pm
Posts: 587
I guess I could just map fwrite() etc directly to write() on a file descriptor (in this case close() is implicitly called on all file descriptors when the process exits). Is there any reason this should not happen?


Top
 Profile  
 
 Post subject: Re: Debugging binutils under my OS?
PostPosted: Sun Oct 11, 2015 5:57 am 
Offline
Member
Member

Joined: Sat Oct 16, 2010 3:38 pm
Posts: 587
Can I just ask, does anyone have rigorous unit tests they've used for their library, that I could also use?

I mean if I write unit tests myself, I'll end up just using the same assumptions I had in the first place that caused me to write buggy code. Googling "stdio unit tests" or "c library unit tests" yields nothing useful.


Top
 Profile  
 
 Post subject: Re: Debugging binutils under my OS?
PostPosted: Mon Oct 12, 2015 2:28 am 
Offline
Member
Member

Joined: Thu Mar 25, 2010 11:26 pm
Posts: 1801
Location: Melbourne, Australia
Hi, please see the attached file. I wouldn't describe these as rigorous. Most of these failed when I first tried but they all pass now :)


Attachments:
stdio_test.c [6.85 KiB]
Downloaded 119 times

_________________
If a trainstation is where trains stop, what is a workstation ?
Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 41 posts ]  Go to page 1, 2, 3  Next

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 22 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group