Orthox-64 — an x86-64 OS that compiles its own kernel (musl, Python/NumPy, lwIP, DOOM)

[aitoh]

Orthox-64 is a from-scratch, monolithic x86-64 (long mode) OS. The headline feature, and the thing I'm proudest of: it self-hosts — the running OS can rebuild its own kernel from source using a natively-ported GCC 4.7.4, and the resulting kernel boots and runs.

What works today
- Boot: Limine (UEFI/BIOS)
- Memory: PMM + paging VMM
- Scheduling: preemptive multitasking, SMP (4 CPUs in QEMU, LAPIC timer, reschedule IPI, per-CPU run queue)
- FS: VFS + a read-write xv6fs (ported from xv6-riscv, extended with triple-indirect blocks, ~16 GB/file)
- Userland: musl libc + BusyBox; full dynamic linking (.so, dlopen/dlsym, TLS, C++ runtime); Python 3.12 importing/running NumPy 1.26.4
- Networking: virtio-net + lwIP — DHCP, DNS, ICMP, UDP, TCP, sockets, BusyBox httpd, and a userspace HTTPS client (BearSSL)
- Self-hosting: GCC 4.7.4 + binutils 2.26 ported to run on the OS; rebuilds /kernel.elf in-place; boots it
- And yes, it runs DOOM (doomgeneric)

A few design decisions that might interest people here
- I started the "sensible" way — extending xv6 with newlib, assuming something 10+ years old would have room to grow. I actually got surprisingly far: Binutils (as/ld/ar/objcopy), a POSIX shell, and a triple-indirect ~1 GB filesystem all running on xv6. But it was all held together by xv6-specific hacks — a hand-rolled printf to dodge Newlib's FPU use (xv6 has no FPU context switching), a 16 MB gap so malloc wouldn't smash the stack, a 1 MB stack for Binutils' deep call chains. The moment I tried to do it properly and move to a standard ABI, the whole thing collapsed — the shell jumped into kernel space in an infinite reboot loop. That was the lesson: bridging xv6's deliberately minimal design and what the GNU toolchain actually demands is a losing battle. So I started clean with an original kernel, and later moved newlib -> musl — which is what finally made a real userland possible.

- Why GCC 4.7.4? It's the last GCC that bootstraps with a C compiler alone (4.8+ needs a working C++ toolchain to build itself). That gave me a C-only self-hosting loop first, and an older/lighter compiler is far more forgiving on a young kernel whose MM and FS are still stabilizing.
- Actually closing the self-hosting loop took ~100 smoke-test iterations and ripping out the original filesystem entirely when it couldn't survive a real build load — that's where the ~16 GB triple-indirect xv6fs came from.

Demo — a recording of it compiling its own kernel and then booting that kernel:
https://github.com/itoh5588/Orthox-64/b ... ADME.en.md
(raw asciicast is in the repo under assets/)

Repo + build/run (host toolchain is clang -target x86_64-elf + lld; boots in QEMU):
https://github.com/itoh5588/Orthox-64 (see INSTALL.md)

Honest notes
- Integration over reinvention: third-party ports are credited — musl, lwIP, BearSSL, Limine, CPython, zlib, BusyBox, GNU make/binutils/gcc; the FS is from xv6-riscv (MIT).
- It runs in QEMU; I haven't tested it on real hardware yet.
- A short personal note: I'm in my 70s, ~46 years in the industry (DEC, Apollo Computer, and OSF during the Unix Wars — on the localization side, not the kernel). Orthox-64 is the first OS kernel I've written end to end.
- I did use an AI agent as a tool. It made generating code cheap; it did not make the judgment to close the self-hosting loop cheap — the decisions above and the ~100 iterations were the actual work. Happy to go into specifics on any of it.

Feedback and pointers very welcome — especially on what you'd consider worth tackling next (I'm eyeing procfs expansion and more userland).

[sandras]

Kudos to you for being straight-up about using third-party software and AI.

Did you use AI to write this post too?

[klange]

from-scratch

xv6fs (ported from xv6-riscv[...])

musl libc + BusyBox

lwIP

I did use an AI agent

So it's not from-scratch at all.

[aitoh]

Did you use AI to write this post too?

Thanks, and fair to ask. I'll be straight: yes, I used the AI to help with the English, since it's not my
first language. But I wrote what I wanted to say first — the AI cleaned up the phrasing, it didn't decide the
content. Same role it played with the code: it made the output cheaper, not the thinking.

To be concrete about that: closing the self-hosting loop meant doing dozens of full in-OS kernel rebuilds —
each one a real GCC 4.7.4 compile of the whole kernel running on the OS itself, then booting the result — and
watching where it fell over. The original filesystem couldn't survive that load at all; I had to tear it out
and rebuild it with triple-indirect blocks before a full build would even complete. None of that — the
judgment about what to fix, and the patience to grind it out — comes out of an AI. That part was mine, and
it's the part I'd want to be judged on.

[aitoh]

So it's not from-scratch at all.

That's a fair hit, and "from-scratch" in the title was a bad word choice on my part — it overclaims.

To be clear about what I actually mean: the kernel is original (boot glue, PMM/VMM, the SMP scheduler, the
syscall layer, the VFS). The userland and the network stack are ports, and I tried to credit every one of them
in the post — musl, lwIP, BearSSL, CPython, BusyBox, binutils/gcc. The xv6fs is the one borrowed kernel
component, and even that I had to rip out and rebuild with triple-indirect blocks before it could survive a
real build load.

So "from-scratch userland" would be a lie, and I'm not claiming it. What I'm actually claiming is narrower: a
from-scratch kernel that can host a ported GNU toolchain well enough to rebuild itself and boot the result. If
the title read as "I wrote all of this from nothing," that's on me — point taken.

[dseller]

So it's not from-scratch at all.

That's a fair hit, and "from-scratch" in the title was a bad word choice on my part — it overclaims.

To be clear about what I actually mean: the kernel is original (boot glue, PMM/VMM, the SMP scheduler, the
syscall layer, the VFS). The userland and the network stack are ports, and I tried to credit every one of them
in the post — musl, lwIP, BearSSL, CPython, BusyBox, binutils/gcc. The xv6fs is the one borrowed kernel
component, and even that I had to rip out and rebuild with triple-indirect blocks before it could survive a
real build load.

So "from-scratch userland" would be a lie, and I'm not claiming it. What I'm actually claiming is narrower: a
from-scratch kernel that can host a ported GNU toolchain well enough to rebuild itself and boot the result. If
the title read as "I wrote all of this from nothing," that's on me — point taken.

Lmao.