Building 64-bit GCC target for RDOS

[rdos]

I've now started to build the GCC related tools required in order to produce 64-bit applications for RDOS.

First is binutils, which require the following changes when moving from 32-bit output to 64-bit output.

bfd/config.bfd:

Code: Select all

*** config.bfd.old	2012-09-04 16:15:00.000000000 +0200
--- config.bfd	2012-11-17 20:53:06.000000000 +0100
***************
*** 699,706 ****
      targ_cflags=-DSTRICT_PE_FORMAT
      ;;
    i[3-7]86-*-rdos*)
!     targ_defvec=bfd_elf32_i386_vec
!     targ_selvecs=i386coff_vec
      ;;
    i[3-7]86-*-mingw32* | i[3-7]86-*-cygwin* | i[3-7]86-*-winnt | i[3-7]86-*-pe)
      targ_defvec=i386pe_vec
--- 699,706 ----
      targ_cflags=-DSTRICT_PE_FORMAT
      ;;
    i[3-7]86-*-rdos*)
!     targ_defvec=bfd_elf64_x86_64_vec
!     targ_selvecs=bfd_elf32_i386_vec
      ;;
    i[3-7]86-*-mingw32* | i[3-7]86-*-cygwin* | i[3-7]86-*-winnt | i[3-7]86-*-pe)
      targ_defvec=i386pe_vec

ld/configure.txt:

Code: Select all

*** configure.tgt.old	2012-09-04 14:53:48.000000000 +0200
--- configure.tgt	2012-11-17 20:41:50.000000000 +0100
***************
*** 197,203 ****
  i[3-7]86-*-coff)	targ_emul=i386coff ;;
  i[3-7]86-*-rtems*)	targ_emul=elf_i386 ;;
  i[3-7]86-*-aros*)	targ_emul=elf_i386 ;;
! i[3-7]86-*-rdos*)	targ_emul=elf_i386 ;;
  i[3-7]86-*-bsd)		targ_emul=i386bsd ;;
  i[3-7]86-*-bsd386)	targ_emul=i386bsd ;;
  i[3-7]86-*-bsdi*)	targ_emul=i386bsd ;;
--- 197,203 ----
  i[3-7]86-*-coff)	targ_emul=i386coff ;;
  i[3-7]86-*-rtems*)	targ_emul=elf_i386 ;;
  i[3-7]86-*-aros*)	targ_emul=elf_i386 ;;
! i[3-7]86-*-rdos*)	targ_emul=elf_x86_64 ;;
  i[3-7]86-*-bsd)		targ_emul=i386bsd ;;
  i[3-7]86-*-bsd386)	targ_emul=i386bsd ;;
  i[3-7]86-*-bsdi*)	targ_emul=i386bsd ;;

Note that contrary to the tutorial, it is not necessary to create a new emulation file, rather the standard elf_x86_64 will do.

I'll post these patches to the binutils project, hoping for acceptance.

[bluemoon]

i[3-7]86-*-rdos*)
! targ_defvec=bfd_elf64_x86_64_vec

Why not start a new target, something like x86_64-elf-rdos, instead of reuse the "386-coff-rdos" but surprisingly emit x86-64 instructions?

[rdos]

i[3-7]86-*-rdos*)
! targ_defvec=bfd_elf64_x86_64_vec

Why not start a new target, something like x86_64-elf-rdos, instead of reuse the "386-coff-rdos" but surprisingly emit x86-64 instructions?

The current target was never used for anything. I moved to OpenWatcom for C/C++ support instead, partly because I never got the binutils patches propagated to newlib and GCC, and partly because OpenWatcom also could support kernel-mode drivers. Now the situation is somewhat different because OpenWatcom doesn't support 64-bit and has poor implementation of advanced C++ issues. There are plans to introduce 64-bit to OpenWatcom, but I think that may take a long time, if it ever is finished.

Besides, the target I use is only "rdos". I don't need to specify the full target. There are some defaults in the configuration (if somebody is interested I could dig them up).

Edit: If I don't misunderstand it, the default target is changed to 64-bit, while the original target (which seems to be called bfd_elf32_i386_vec now) is retained. I originally also used i386coff_vec, but it seems to have disappeared because it emits warnings.

[rdos]

The next issue is what C library to use. The primary alternatives seems to be newlib or libc. While newlib is much easier to port, I really wonder if it is enough for porting more complex *nix applications like web-browsers, Java virtual machines and alike? Does somebody know what to expect from newlib and what to expect from libc in terms of portability of standard applications?

[Owen]

You say "libc" as if it's a library, not a generic term for a C library (on Unix systems, derived from the lib* file naming convention

Newlib will probably get you command line Unix apps, provided you can implement the Unix syscalls correctly (e.g. fork/open/close and their semantics). The code cleanliness leaves lot to be desired, but it seems at least somewhat battle hardened. Newlib is aimed at POSIX/UNIX compatibility.

Other C libraries have different aims. For example, PDCLib, the project I inherited from Solar, is aimed solely at compliance with the C/C++ standards. I implement a few POSIX functions, but they're generally very widely used (e.g. strndup) or impossible to implement externally (e.g. the _unlocked family). I'm explicitly not aiming at implementing POSIX/Single Unix Specification (though I'm not intending to conflict with it either)

[rdos]

You say "libc" as if it's a library, not a generic term for a C library (on Unix systems, derived from the lib* file naming convention

When I write libc, I mean GNU Libc. Last I looked at libc, it seemed a lot more complex, but also covers a lot more of the POSIX standard than newlib.

Newlib will probably get you command line Unix apps, provided you can implement the Unix syscalls correctly (e.g. fork/open/close and their semantics). The code cleanliness leaves lot to be desired, but it seems at least somewhat battle hardened. Newlib is aimed at POSIX/UNIX compatibility.

Yes, I know that the primary obstacle is to provide fork and exec, and make them work like they are expected to work. I have some ideas about how to provide fork for 64-bits, without breaking the DOS/Win32-like interface in 32-bit protected mode.

Other C libraries have different aims. For example, PDCLib, the project I inherited from Solar, is aimed solely at compliance with the C/C++ standards. I implement a few POSIX functions, but they're generally very widely used (e.g. strndup) or impossible to implement externally (e.g. the _unlocked family). I'm explicitly not aiming at implementing POSIX/Single Unix Specification (though I'm not intending to conflict with it either)

I don't think that PDCLib is adequate for my ambitions. I won't use this for a kernel, and I expect better results than with OpenWatcom, which isn't that bad, but still isn't compatible enough for some things I have in mind.

[rdos]

i[3-7]86-*-rdos*)
! targ_defvec=bfd_elf64_x86_64_vec

Why not start a new target, something like x86_64-elf-rdos, instead of reuse the "386-coff-rdos" but surprisingly emit x86-64 instructions?

Seems like this is the position of the binutils maintainer as well. It would be nice to be able to build both versions, but is it really necessary to clutter binutils with one additional target, and especially when the old one is expected to be obsolete?

Wouldn't it be possible to use multiple targets instead?

[rdos]

OK, some more research into binutils, and here are some new patches:

config.sub: (this is the one that defines simplified targets, and which is shared with GCC)

Code: Select all

*** config.sub.old	2012-11-20 19:51:30.000000000 +0100
--- config.sub	2012-11-20 20:29:36.000000000 +0100
***************
*** 1020,1028 ****
--- 1020,1036 ----
  		os=-pw32
  		;;
  	rdos)
+ 		basic_machine=x86_64-pc
+ 		os=-rdos
+ 		;;
+ 	rdos32)
  		basic_machine=i386-pc
  		os=-rdos
  		;;
+ 	rdos64)
+ 		basic_machine=x86_64-pc
+ 		os=-rdos
+ 		;;
  	rom68k)
  		basic_machine=m68k-rom68k
  		os=-coff

So now the standard target "rdos" refers to a 64-bit build, and there are also specific targets for 32-bits (rdos32) and 64-bits (rdos64).

bfd/config.bfd: (adds a new target for 64-bits)

Code: Select all

*** config.bfd.old	2012-09-04 16:15:00.000000000 +0200
--- config.bfd	2012-11-20 21:21:10.000000000 +0100
***************
*** 660,665 ****
--- 660,670 ----
      want64=true
      targ_underscore=no
      ;;
+   x86_64-*-rdos*)
+     targ_defvec=bfd_elf64_x86_64_vec
+     targ_selvecs="bfd_elf32_i386_vec bfd_elf64_l1om_vec bfd_elf64_k1om_vec"
+     want64=true
+     ;;
  #endif
    i[3-7]86-*-lynxos*)
      targ_defvec=bfd_elf32_i386_vec

ld/configure.tgt:

Code: Select all

*** configure.tgt.old	2012-09-04 14:53:48.000000000 +0200
--- configure.tgt	2012-11-20 21:20:08.000000000 +0100
***************
*** 198,203 ****
--- 198,204 ----
  i[3-7]86-*-rtems*)	targ_emul=elf_i386 ;;
  i[3-7]86-*-aros*)	targ_emul=elf_i386 ;;
  i[3-7]86-*-rdos*)	targ_emul=elf_i386 ;;
+ x86_64-*-rdos*)     targ_emul=elf_x86_64 ;;
  i[3-7]86-*-bsd)		targ_emul=i386bsd ;;
  i[3-7]86-*-bsd386)	targ_emul=i386bsd ;;
  i[3-7]86-*-bsdi*)	targ_emul=i386bsd ;;

I'll send these new patches and hope for acceptance.

[rdos]

Still no luck in getting the patches accepted into binutils.

The patches required to GCC are still minimal. The only file affected is gcc/config.gcc, which need some new lines:

Code: Select all

--- config.gcc.old	2012-11-20 22:46:54.000000000 +0100
+++ config.gcc	2012-11-20 22:47:02.000000000 +0100
@@ -705,6 +705,10 @@
   use_gcc_stdint=wrap
   tm_defines="$tm_defines DEFAULT_LIBC=LIBC_UCLIBC SINGLE_LIBC"
   ;;
+*-*-rdos*)
+      case ${enable_threads} in
+         "" | yes | rdos) thread_file='rdos' ;;
+   esac
+  use_gcc_stdint=wrap
+  ;;
 *-*-solaris2*)
   # i?86-*-solaris2* needs to insert headers between cpu default and
   # Solaris 2 specific ones.
@@ -1208,6 +1212,9 @@
 x86_64-*-elf*)
 	tm_file="${tm_file} i386/unix.h i386/att.h dbxelf.h elfos.h newlib-stdint.h i386/i386elf.h i386/x86-64.h"
 	;;
+i[34567]86-*-rdos*)
+      tm_file="${tm_file} i386/unix.h i386/att.h dbxelf.h elfos.h newlib-stdint.h i386/i386elf.h i386/rdos.h"
+      tmake_file="i386/t-i386elf t-svr4"
+	;;
+x86_64-*-rdos*)
+      tm_file="${tm_file} i386/unix.h i386/att.h dbxelf.h elfos.h newlib-stdint.h i386/i386elf.h  i386/x86-64.h i386/rdos.h"
+      tmake_file="i386/t-i386elf t-svr4"
+	;;
 i[34567]86-*-freebsd*)
 	tm_file="${tm_file} i386/unix.h i386/att.h dbxelf.h elfos.h ${fbsd_tm_file} i386/freebsd.h"
 	;;

I don't think I require an additional .h file, but instead use the elfos.h file, which is supposed to be a generic elf template which should work. At least for now it should.

Edit: Added 32-bit build.

[rdos]

OK, so I downloaded newlib, and to my surprise found that all the work I did back in 2006 is in fact part of newlib. IOW, I didn't need to make any substantial patches to newlib, and all the relevant changes are already provided. There also is a sys/rdos directory with (32-bit) RDOS adaptations that are somewhat broken.

Changes to newlib/configure.host:

Code: Select all

--- configure.host.old	2011-11-29 07:33:48.000000000 +0100
+++ configure.host	2012-12-04 21:58:26.000000000 +0100
@@ -408,7 +408,11 @@
 	;;
   i[34567]86-*-rdos*)
 	sys_dir=rdos
-	newlib_cflags="${newlib_cflags} -DMISSING_SYSCALL_NAMES"
+	newlib_cflags="${newlib_cflags} -DSIGNAL_PROVIDED -DREENTRANT_SYSCALLS_PROVIDED"
+	;;
+  x86_64-*-rdos*)
+	sys_dir=rdos
+	newlib_cflags="${newlib_cflags} -DSIGNAL_PROVIDED -DREENTRANT_SYSCALLS_PROVIDED"
 	;;
   i[34567]86-*-sco*)
 	sys_dir=sysvi386

[rdos]

I think the advice on how to build libgcc in the Wiki is wrong. At least when building thread-safe versions, as I don't seem to be able to build that for RDOS. The build process seems to want pthread.h, even when --without-headers is specified as a configuration option. However, I guess it should be possible to build libgcc once the newlib build is ok (with headers).

[Owen]

The thread safe libgcc is likely looking for thread support routines in the usual POSIX place (i.e. pthreads). Newlib doesn't provide them; you'll need to provide them (based upon your own threading interfaces), or teach libgcc your native threading interface (as I assume is done for e.g. Windows)

[rdos]

The thread safe libgcc is likely looking for thread support routines in the usual POSIX place (i.e. pthreads). Newlib doesn't provide them; you'll need to provide them (based upon your own threading interfaces), or teach libgcc your native threading interface (as I assume is done for e.g. Windows)

I think you are be correct. I do plan to provide the standard POSIX-thread interface, rather than my own, since I want compability. I think that would be no big deal as they are similar enough. The only problem I have with (all?) threading interfaces, is that there is no thread name parameter, which means you get some strange PID that says nothing when dumping threads.

Wouldn't it be possible to put this code in newlib and export pthreads.h as part of the RDOS include-files for newlib?

I looked at the reentrancy interface for newlib, and I don't exactly like it. I think the way they should have implemented errno is by having it in a "TLS" area pointed to by FS or GS which would be per thread. That way syscalls doesn't need extra parameters, and still can be made reentrant. That's also how I did it in OW.

[rdos]

The patches for binutils (except for the "global" config.sub file) has now been committed.

Next, I'll look into getting the patches for config.sub accepted.

[Owen]

I looked at the reentrancy interface for newlib, and I don't exactly like it. I think the way they should have implemented errno is by having it in a "TLS" area pointed to by FS or GS which would be per thread. That way syscalls doesn't need extra parameters, and still can be made reentrant. That's also how I did it in OW.

Newlib's reentrancy interface is one of the most horrid things ever conceived, a terrible way to do it, and violates POSIX and the more recent C11 standard.

Some bits in Newlib's "struct reent" are OK; for example, errno, and getting it via the _get_reent() function/macro is also pretty standard. Normally this would be implemented on top of pthread_key_get/tss_get, which, yes, on x86 tends to be implemented by something along the lines of "mov X, [gs:0 + sizeof(void*) * key]" (or more directly, e.g. by hardcoding a slot for more efficiency)

The fact that stdin/stdout/stderr are there, however, is absolute madness and makes no sense. Both C11 and POSIX declare the file streams to be thread safe and mutex protected; in particular, there are behavioral requirements which can't be met any other way.

This is one of the prime reasons I gave up on Newlib in disgust and picked up the PDCLib project. If you're planning on porting existing POSIXy apps, however, you'll need more than it gives you (or ever will)