OSDev.org

The Place to Start for Operating System Developers
It is currently Thu Mar 28, 2024 5:00 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 5 posts ] 
Author Message
 Post subject: Porting newlib, messy first steps
PostPosted: Thu May 25, 2017 8:20 pm 
Offline
Member
Member

Joined: Wed Jul 10, 2013 9:11 am
Posts: 51
So I just completed http://wiki.osdev.org/Porting_Newlib. I realize that page does not have the best advice and glosses over libgloss (sorry) but it seems to work just fine for my purposes. I'm interested to see if what I did seems fine for basic purposes. Note that I only cared about getting newlib working and I didn't do anything in the way of OS specific toolchain.

- Compile binutils and GCC with --with-sysroot pointing to some directory
- Follow the above link
- Have a implementation-defined list of indices for system calls and have each stub call an implementation-defined software interrupt with arguments, i.e:
Code:
  #define sys_write 4
  int write(int file, char *ptr, int len) {
      int res;
      asm volatile("int $0x80" : "=a" (res) : "0" (sys_write), "b" ((int)file), "c" ((long)ptr), "d" ((int)len)); 
      return res;
  }

Real code would wrap this in a macro
- Compile and install newlib files to the sysroot
- Treat 0x80 like any other interrupt and add an assembly stub/to the IDT
- In the ISR handler, if it's interrupt 0x80, pass the regs to the system call handler
- Check regs->eax for the system call type; call the corresponding function with the rest of the GPRs as arguments
- Link against -lc -lm (I have the feeling this is not necessary, so I screwed up somewhere)

Code:
write(1, "Test", sizeof("Test"));
seems to work just fine.

What I'm looking to do next is to figure out how to replace newlib's malloc with liballoc (I already have liballoc working kernel-wise) as it seems sbrk is not very popular nowadays. This would ostensibly make testing parts of the library easier before I get userspace working.

Sidenote:

From my search it seems other people are having problems with sbrk and need to zero out the BSS. I'm guessing this only becomes a problem when you enter userspace. For now my stub looks like this:

Code:
caddr_t sbrk(int incr) {
      errno = ENOMEM;
      return (caddr_t) -1;
  }


which I mirror'd after the linux man page.


Top
 Profile  
 
 Post subject: Re: Porting newlib, messy first steps
PostPosted: Fri May 26, 2017 2:50 am 
Offline
Member
Member

Joined: Tue May 13, 2014 3:02 am
Posts: 280
Location: Private, UK
goku420 wrote:
What I'm looking to do next is to figure out how to replace newlib's malloc with liballoc (I already have liballoc working kernel-wise) as it seems sbrk is not very popular nowadays. This would ostensibly make testing parts of the library easier before I get userspace working.


Not sure exactly how it's done in Newlib's build system (since I replaced it with a simple makefile after struggling to convince it to build me a shared library for over a month), but there's a macro called "MALLOC_PROVIDED" macro that will disable Newlib's internal malloc/free implementation. Looking at the wiki, you likely define it in the same way as "SIGNAL_PROVIDED" in "configure.host".

Once that's done, you should be able to just add the liballoc source file to your "EXTRA_lib_a_SOURCES".

_________________
Image


Top
 Profile  
 
 Post subject: Re: Porting newlib, messy first steps
PostPosted: Fri May 26, 2017 3:36 am 
Offline
Member
Member

Joined: Wed Jul 10, 2013 9:11 am
Posts: 51
mallard wrote:
Not sure exactly how it's done in Newlib's build system (since I replaced it with a simple makefile after struggling to convince it to build me a shared library for over a month), but there's a macro called "MALLOC_PROVIDED" macro that will disable Newlib's internal malloc/free implementation. Looking at the wiki, you likely define it in the same way as "SIGNAL_PROVIDED" in "configure.host".

Once that's done, you should be able to just add the liballoc source file to your "EXTRA_lib_a_SOURCES".


Thanks. I guess it would be the perfect time to actually write mmap. Since I don't have a filesystem or userspace, my implementation should be relatively simple:

- Flags: MAP_ANONYMOUS, PROT_WRITE, PROT_READ, MAP_PRIVATE
- As a result, the fd and offset arguments are irrelevant
- The start address will be a nearby page boundary (round down?)
- It needs to be mapped in multiples of page sizes


Top
 Profile  
 
 Post subject: Re: Porting newlib, messy first steps
PostPosted: Sat May 27, 2017 11:35 am 
Offline
Member
Member

Joined: Mon Jul 05, 2010 4:15 pm
Posts: 595
mallard wrote:
goku420 wrote:
What I'm looking to do next is to figure out how to replace newlib's malloc with liballoc (I already have liballoc working kernel-wise) as it seems sbrk is not very popular nowadays. This would ostensibly make testing parts of the library easier before I get userspace working.


Not sure exactly how it's done in Newlib's build system (since I replaced it with a simple makefile after struggling to convince it to build me a shared library for over a month), but there's a macro called "MALLOC_PROVIDED" macro that will disable Newlib's internal malloc/free implementation. Looking at the wiki, you likely define it in the same way as "SIGNAL_PROVIDED" in "configure.host".

Once that's done, you should be able to just add the liballoc source file to your "EXTRA_lib_a_SOURCES".


I struggled too convincing the Newlib create a shared library for a bare metal system. The bare metal configuration for ARM just makes statically linked archived that is supposed to be used in statically linked embedded system like Cortex-M3 systems with simple operating systems. Shared library configurations only exists for existing OSes like Linux together with ARM. My solution was the following:

First force -fPIC into the build that creates an archive. Something like this for the configuration script (here I use -fpie but -fPIC can be used just as well).

Code:
./configure --prefix=/home/johndoe/newlib-pie --target=arm-none-eabi --disable-newlib-supplied-syscalls --enable-newlib-io-long-long --enable-newlib-io-long-double \
--enable-newlib-io-c99-formats --disable-multilib --with-cpu=armv7-a --with-mode=thumb --with-float=soft --disable-nls CFLAGS_FOR_TARGET="-g -O2 -fpie -fdata-sections -ffunction-sections"


Then you have an .a file but built with -fPIC. Now you want to link this archive into an actual share library and this can be done with the following command line for the GCC linker.

Code:
-shared -T SharedLibrary.lds -z common-page-size=4096 -z max-page-size=4096 \
-Bsymbolic -no-allow-shlib-undefined -nostdlib -whole-archive ${CLIB_LIBRARY_PATH}/libc.a -no-whole-archive


SharedLibrary.lds is a linker script customized for your OS for shared libraries. Also you need extra source files that implements some of the required POSIX calls. If you know you will not be using a POSIX call you can just put something like "_wait = 0;" in the linker script in order to "stub" _wait (it will crash if ever used).

Obviously the Newlib configuration script is for existing systems like Linux and so on. Also it is using libtool to make shared libraries which is another dependency which is OS dependent. Another way is to create "customized toolchain" but I haven't bothered doing that. Instead you can "fool" Newlib like this instead.

The same strategy will also work for libstdc++ together with newlib (--with-newlib). I hope they keep the possibility to compile for newlib.


Top
 Profile  
 
 Post subject: Re: Porting newlib, messy first steps
PostPosted: Sat May 27, 2017 3:33 pm 
Offline
Member
Member

Joined: Wed Jul 10, 2013 9:11 am
Posts: 51
So that was actually relatively painless. I only made 3 changes:

- Made the change to configure.host like mallard suggested
- Add _malloc_r and friends
- Replace all instances of #include <liballoc.h> with #include <stdlib.h>

newlib requires _malloc_r and friends to be defined even if you don't use reentrancy. You can just discard the _reent parameter and return a call to malloc. Since I'm linking in liballoc anyway, it will be found by newlib.

In the future I suspect that I will have to fix this and actually add the liballoc hooks as a source as suggested by mallard; the problem is my kernel is in C++ so I'm not sure at the moment how I'll do that.

EDIT:

Success. The solution was to not to distribute liballoc_hooks with the newlib glue, only liballoc itself. To confirm that it's working I called strdup (something I knew would guarantee newlib to call malloc) and set a breakpoint in gdb:

Code:
(gdb) break _malloc_r
Breakpoint 1 at 0xc0104d80: file ../../../../../../newlib-2.2.0-1/newlib/libc/sys/myos/malloc_wrappers.c, line 6.
(gdb) continue
Continuing.

Breakpoint 1, _malloc_r (ptr=0xc0120380 <impure_data>, size=14) at ../../../../../../newlib-2.2.0-1/newlib/libc/sys/myos/malloc_wrappers.c:6
6      return PREFIX(malloc) (size);
(gdb) s
malloc (req_size=14) at ../../../../../../newlib-2.2.0-1/newlib/libc/sys/myos/liballoc.c:242
242   {


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 5 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: Bing [Bot] and 66 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group