Building 64-bit GCC target for RDOS
Re: Building 64-bit GCC target for RDOS
The cpuid-related bug is on the way to be resolved. I provided a patch for saving rbx and using another register. However, another guy now has provided a patch for the real problem with using RBX for medium memory model. So I think this wll be incorporated into the 4.8 branch pretty soon.
Re: Building 64-bit GCC target for RDOS
I'm putting together some new (basic) patches for binutils and GCC. In binutils, I added my own script file + some new settings for the linker script (for example, the ability put large-data at any virtual address). In GCC, I think I'll submit the general things first, and wait with the libgcc related (which still is not finished). The GCC changes also contains some enhancements to GCC. Like an ability to change the default threshold for large data (which I plan to set to 16 bytes so only simple data-types will end up in the small data section). The best is that this new section does not add 2MB to the file-size.
So, this will be my layout for 64-bit userland:
0000000000000000-00000180DFFFFFFF: invalid
00000180E0000000-000001815FFFFFFF: code and small data (application)
0000018160000000-000001FFFFFFFFFF: code and small data (shared libraries)
0000020010000000-000002FF30000000: thread stacks, TLS
0000080020000000-00000FFFFFFFFFFF: large data and bss (application + shared libraries)
0000100000000000-00007EFFFFFFFFFF: heap
00007F0000000000-00007FFFFFFFFFFF: invalid
0000800000000000-FFFF7FFFFFFFFFFF: non-canonical
FFFF800000000000-FFFFFEFFFFFFFFFF: device drivers (CPL to be decided)
FFFFFF0000000000-FFFFFF7FFFFFFFFF: fork page table mapping (kernel only)
FFFFFF8000000000-FFFFFFFFFFFFFFFF: page table mapping (kernel only)
So, this will be my layout for 64-bit userland:
0000000000000000-00000180DFFFFFFF: invalid
00000180E0000000-000001815FFFFFFF: code and small data (application)
0000018160000000-000001FFFFFFFFFF: code and small data (shared libraries)
0000020010000000-000002FF30000000: thread stacks, TLS
0000080020000000-00000FFFFFFFFFFF: large data and bss (application + shared libraries)
0000100000000000-00007EFFFFFFFFFF: heap
00007F0000000000-00007FFFFFFFFFFF: invalid
0000800000000000-FFFF7FFFFFFFFFFF: non-canonical
FFFF800000000000-FFFFFEFFFFFFFFFF: device drivers (CPL to be decided)
FFFFFF0000000000-FFFFFF7FFFFFFFFF: fork page table mapping (kernel only)
FFFFFF8000000000-FFFFFFFFFFFFFFFF: page table mapping (kernel only)
Last edited by rdos on Sat Feb 02, 2013 4:18 pm, edited 1 time in total.
Re: Building 64-bit GCC target for RDOS
The patch for GCC base (without libgcc, which is not yet done) has been accepted. In this patch, I also change PIC-register to R15 in order to not have to save RBX on every syscall.
Re: Building 64-bit GCC target for RDOS
I think I've found out a possible reason why __register_frame_info and __deregister_frame_info is not found in my setup. I've now verified that these functions are part of my libgcc.a file, but they are marked as ".hidden". Perhaps there is some bug in the linker related to hidden symbols with medium memory model?
The error log looks like this:
Anybody has an idea how to solve this issue?
Edit: It doesn't help to remove .hidden. The errors remain even when the entries are not hidden. It is also not the wrong libraries being linked against since using rdos-ld and specifying the libraries and objects still give the same error.
Relevant object dump (-t flag) of libgcc.a:
If the object file is removed from the library, and instead linked as a object, ld seems to find the required functions, but then emit a lot of other problems (which probably are correct since these are unimplemented).
The error log looks like this:
Code: Select all
/usr/local/lib/gcc/rdos/4.8.0/../../../../rdos/bin/ld: error in /usr/local/lib/gcc/rdos/4.8.0/crtend.o(.eh_frame); no .eh_frame_hdr table will be created.
/usr/local/lib/gcc/rdos/4.8.0/crtbegin.o: In function `__do_global_dtors_aux':
crtstuff.c:(.text+0xda): relocation truncated to fit: R_X86_64_PLT32 against undefined symbol `__deregister_frame_info'
/usr/local/lib/gcc/rdos/4.8.0/crtbegin.o: In function `frame_dummy':
crtstuff.c:(.text+0x119): relocation truncated to fit: R_X86_64_PLT32 against undefined symbol `__register_frame_info'
collect2: error: ld returned 1 exit status
Edit: It doesn't help to remove .hidden. The errors remain even when the entries are not hidden. It is also not the wrong libraries being linked against since using rdos-ld and specifying the libraries and objects still give the same error.
Relevant object dump (-t flag) of libgcc.a:
Code: Select all
unwind-dw2-fde-dip.o: file format elf64-x86-64
SYMBOL TABLE:
0000000000000000 l df *ABS* 0000000000000000 unwind-dw2-fde-dip.c
0000000000000000 l d .text 0000000000000000 .text
0000000000000000 l d .data 0000000000000000 .data
0000000000000000 l d .bss 0000000000000000 .bss
0000000000000000 l F .text 0000000000000047 read_sleb128
0000000000000050 l F .text 000000000000001b fde_unencoded_compare
0000000000000070 l F .text 00000000000000d5 frame_downheap
0000000000000150 l F .text 00000000000000ac frame_heapsort
0000000000000200 l F .text 0000000000000014 init_object_mutex
0000000000000010 l O .bss 0000000000000004 object_mutex
0000000000000220 l F .text 0000000000000067 size_of_encoded_value
0000000000000290 l F .text 000000000000012c read_encoded_value_with_base
0000000000000000 l d .rodata 0000000000000000 .rodata
00000000000003c0 l F .text 000000000000010a get_cie_encoding
00000000000004d0 l F .text 0000000000000068 base_from_object.isra.6
0000000000000540 l F .text 000000000000008c fde_single_encoding_compare
00000000000005d0 l F .text 0000000000000180 classify_object_over_fdes
0000000000000750 l F .text 0000000000000165 add_fdes
00000000000008c0 l F .text 00000000000001a7 linear_search_fdes
0000000000000a70 l F .text 00000000000000ad fde_mixed_encoding_compare
0000000000000b20 l F .text 0000000000000765 search_object
0000000000000008 l O .bss 0000000000000008 terminator.4746
0000000000000000 l O .bss 0000000000000008 marker.4640
0000000000000020 l O .bss 0000000000000008 unseen_objects
0000000000000018 l O .bss 0000000000000008 seen_objects
0000000000000000 l d .debug_info 0000000000000000 .debug_info
0000000000000000 l d .debug_abbrev 0000000000000000 .debug_abbrev
0000000000000000 l d .debug_loc 0000000000000000 .debug_loc
0000000000000000 l d .debug_aranges 0000000000000000 .debug_aranges
0000000000000000 l d .debug_ranges 0000000000000000 .debug_ranges
0000000000000000 l d .debug_line 0000000000000000 .debug_line
0000000000000000 l d .debug_str 0000000000000000 .debug_str
0000000000000000 l d .eh_frame 0000000000000000 .eh_frame
0000000000000000 l d .comment 0000000000000000 .comment
0000000000000000 *UND* 0000000000000000 _GLOBAL_OFFSET_TABLE_
0000000000000000 *UND* 0000000000000000 __rdos_thread_mutex_init
0000000000000000 *UND* 0000000000000000 abort
0000000000000000 *UND* 0000000000000000 strlen
0000000000000000 *UND* 0000000000000000 malloc
0000000000000000 *UND* 0000000000000000 free
0000000000001290 g F .text 0000000000000071 __register_frame_info_bases
0000000000000000 *UND* 0000000000000000 __rdos_thread_once
0000000000000000 *UND* 0000000000000000 __rdos_thread_mutex_lock
0000000000000000 *UND* 0000000000000000 __rdos_thread_mutex_unlock
0000000000001310 g F .text 0000000000000009 __register_frame_info
0000000000001320 g F .text 0000000000000026 __register_frame
0000000000001350 g F .text 0000000000000060 __register_frame_info_table_bases
00000000000013b0 g F .text 0000000000000009 __register_frame_info_table
00000000000013c0 g F .text 000000000000001a __register_frame_table
00000000000013e0 g F .text 000000000000010e __deregister_frame_info_bases
00000000000014f0 g F .text 0000000000000005 __deregister_frame_info
0000000000001500 g F .text 0000000000000025 __deregister_frame
0000000000001530 g F .text 0000000000000190 _Unwind_Find_FDE
Re: Building 64-bit GCC target for RDOS
Just rebuilt binutils with no patches, and it works. The GCC (bootstrap) part has also been committed, but there is no snapshot yet. I hope one will appear tomorrow (they seem to appear on Sundays).
Re: Building 64-bit GCC target for RDOS
Just downloaded new releases of binutils (2.23.2) and GCC (4.8.0), and they now build binutils for rdos, and the bootstrap of GCC "out-of-the-box". 

Re: Building 64-bit GCC target for RDOS
I have recently studied building custom-target binutils, gcc, and newlib. It has been somewhat helpful to check what you have done. At least I roughly now what steps are needed. I really appreciate that you got your patches accepted to vanilla versions of binutils and gcc.
When thinking about this and the fact you have quite many real installations of your OS, one can really say you have made an OS. Although you receive some critics from OSDev community, you still have done something remarkable. Technical issues are a topic of its own but I think it is quite rare to have stable and production-use releases. Many of us are stuck in "alpha-test-versions" forever.
When thinking about this and the fact you have quite many real installations of your OS, one can really say you have made an OS. Although you receive some critics from OSDev community, you still have done something remarkable. Technical issues are a topic of its own but I think it is quite rare to have stable and production-use releases. Many of us are stuck in "alpha-test-versions" forever.
Re: Building 64-bit GCC target for RDOS
The bad thing is that I have not had time to do anything more on the 64-bit port since February. That's partly due to our company expanding with a new product for the polish market. I also anticipate being involved in other projects in southern Europe. All of these will most likely use RDOS as an operating system.
I would need to continue research into newlib and libgcc to finalize the GCC port. Possibly I will get time for this during summer.
I would need to continue research into newlib and libgcc to finalize the GCC port. Possibly I will get time for this during summer.
Re: Building 64-bit GCC target for RDOS
I stopped developing my actual kernel until I fully understand this. It is not so easy as I thought. Of course I can already make some sort of toolchain but it is not the same as fully understanding the issue. What is confusing me is how the Linux From Scratch book handles the creation of toolchain (Chapter 5, constructing temporary system). Although it is Linux-specific, the toolchain notes are not highly dependent on that. For example, there are two passes when building binutils and gcc with a lot of configure options. It uses glibc instead of newlib and it may be one of the reason why it is done that way.rdos wrote:research into newlib and libgcc to finalize the GCC port
However, I am quite sure that LFS is quite well tested. That is why I am a little bit skeptical whether our instructions for building OS specific toolchains etc. are fully complete. They probably are and I am just starting to "check" them. Like I said, I do not fully understand the issue yet. I am not happy to just "pass these configure options" because they "seem to work."
Re: Building 64-bit GCC target for RDOS
My current research on the issue indicates that the only environments ported to GCC which support complex user-level applications are Windows and *nix-clones. The Windows port has a large number of changes throughout the tool-chain, and ported utilities sometimes malfunctions. Therefore, there are no examples of how to create a port that can be used to compile complex user-level applications to an OS that is neither Windows nor *nix-like. Setting up an environment for an OS-kernel is another issue at a very different level of complexity.
I also have experience from porting the libc of OpenWatcom to RDOS. OpenWatcom already supported a more varied range of OSes (from MS-DOS to Linux), but adding a completely different environment there was not easy, but possible. I got my own branch to work on, which was eventually merged back into trunk. This required 100s of changes to the repository. Then I only added support for RDOS as a target environment, and not as a build-environment, which would have required a lot more.
The big issues with the GCC environment seems to be with the c-library and libgcc. Building the compiler and tool-set is just a minor issue.
I also have experience from porting the libc of OpenWatcom to RDOS. OpenWatcom already supported a more varied range of OSes (from MS-DOS to Linux), but adding a completely different environment there was not easy, but possible. I got my own branch to work on, which was eventually merged back into trunk. This required 100s of changes to the repository. Then I only added support for RDOS as a target environment, and not as a build-environment, which would have required a lot more.
The big issues with the GCC environment seems to be with the c-library and libgcc. Building the compiler and tool-set is just a minor issue.
Re: Building 64-bit GCC target for RDOS
My current OS project has lost its fun. It is ridiculous that I am struggling with configure scripts, reading binutils/gcc/newlib/bash/etc source code, making patches (!), studying things like termios and considering porting user-space programs. I should be making my own system. I want to use a word 'system'. If I used the word 'kernel', it would not describe what I mean. I am now seriously starting to consider making new design even if it ended up being bad. I am not going to port anything. The biggest problem is the toolchain because I would like the system eventually be self-hosted. C programming language causes a big problem here...
Re: Building 64-bit GCC target for RDOS
I think self-hosted is a little bit over-reclamated and an overkill. Sure, it is a real challenge, and nice not to need any code that anybody else wrote (rather compiled), but it seems pretty pointless. By using GCC or any other 3:d part code-base, you would not have done all the code yourself anyway, only had compiled everything yourself, which you could do with a Linux kernel as well (even if you didn't write a single line of code in Linux).
Making RDOS self-hosted is very low on the priority-list, and I find it unlikely it will ever happen. I find it more convenient to use ready-to-use IDEs, compilers, editors and version control and debug using remote debugging on a second computer. That means the host system doesn't need to reboot, and the target can be rebooted for each debug-session with no problem since this is fast.
Making RDOS self-hosted is very low on the priority-list, and I find it unlikely it will ever happen. I find it more convenient to use ready-to-use IDEs, compilers, editors and version control and debug using remote debugging on a second computer. That means the host system doesn't need to reboot, and the target can be rebooted for each debug-session with no problem since this is fast.
Re: Building 64-bit GCC target for RDOS
Yes, making a system being self-hosted without third-party software is quite a tough task. Even with third-party software it is not easy. If I were to write everything in assembly, it could be possible (at least in theory) to do this. A C compiler seems to be far too hard to implement.
If taking "gcc path", it is not just the compiler that causes troubles. I would need to have almost fully working "bash-environment" with various other tools also ported (e.g. make) to be able to compile the toolchain. This toolchain is also quite "bloated" so it would probably take days to compile it on my system because it takes a while to compile the full toolchain even on Linux.
It would be very easy to just stop silliness and give up trying to reach the self-host environment. However, I would then set a fundamental restriction to the project. Maybe I should create a new programming language that is more compiler-friendly. Then I could create a compiler too and I would not need third-party code at all. Something like a "portable assembly" language. Of course I should think of how to "bootstrap" this whole system. Maybe I could have a translator that converts my simple language to C and then use a gcc-like compiler to compile it first time.
I start reading the Dragon Book.
If taking "gcc path", it is not just the compiler that causes troubles. I would need to have almost fully working "bash-environment" with various other tools also ported (e.g. make) to be able to compile the toolchain. This toolchain is also quite "bloated" so it would probably take days to compile it on my system because it takes a while to compile the full toolchain even on Linux.
It would be very easy to just stop silliness and give up trying to reach the self-host environment. However, I would then set a fundamental restriction to the project. Maybe I should create a new programming language that is more compiler-friendly. Then I could create a compiler too and I would not need third-party code at all. Something like a "portable assembly" language. Of course I should think of how to "bootstrap" this whole system. Maybe I could have a translator that converts my simple language to C and then use a gcc-like compiler to compile it first time.
I start reading the Dragon Book.
Re: Building 64-bit GCC target for RDOS
A useful guide to TLS implementation in GCC: http://www.akkadia.org/drepper/tls.pdf
One problem is that they change to FS as a base in x86-64, something that I must change for RDOS as it is not possible to use FS for TLS due to using FS in ISRs.
Another thing is that I should define TARGET_TLS_DIRECT_SEG_REFS, so that the compiler will generate code that uses gs-relative addressing to get to thread local storage.
One problem is that they change to FS as a base in x86-64, something that I must change for RDOS as it is not possible to use FS for TLS due to using FS in ISRs.
Another thing is that I should define TARGET_TLS_DIRECT_SEG_REFS, so that the compiler will generate code that uses gs-relative addressing to get to thread local storage.
Re: Building 64-bit GCC target for RDOS
The TLS support is finalized, and uses GS as a base for x86-64. I'll post the patches to GCC in order to change the segment register. I can also create a new thread, open and read/write files, the futex-based critical sections work, so soon I'll be able to provide newlib and libgcc patches for something that should at least support the most basic functionality.