Toolchains and portability

[ihateneedles]

So I was reading a thread on here yesterday, and someone got me thinking.

Maybe this is just a difference of definitions, but I don't think there is such a thing as portability. Every software stack, every OS, every library, every program you write ultimately depends on some platform specific implementation detail. The operating system, shared libraries, hardware interfaces, memory mappings, or just the instruction set itself. A program built for one platform won't run unmodified on any other platform --- you'll have to modify and rebuild --- so all software is inherently not portable.

But what programmers seem to mean when they say "portability," though, is that of a general principle (or maybe commitment) to compartmentalizing as much of the platform-specific time wasting as possible. Quarantining x86-specific kernel sources to their own directory, enabled at build-time through conditional flags, for example.

Being as ridiculously lazy as I am (and also an extremely nervous person), this kind of tedium is worse than sub-optimal, it's actively dread-inducing. And makefile script just sucks. Why should I have to learn an entirely new language just to save on some typing and some CPU cycles? Not everything needs to have an entire Turing-complete programming language tacked onto it, let alone something as ugly and difficult to understand as that. (Aaah!! Bleach my eyes!! Aaah!!) Like, I already know C, at that point it would just be easier to write my own thing that does what I want.

And come to think of it, the GNU tools really aren't much fit for purpose, are they? They're bloated and unwieldy. Sure gcc will compile code, as will assemble it and ld will link it, but that's the bare minimum anyone expects from a tool, that it does what it says on the tin. And the GNU tools are hard to fork, code base is really hard to understand (and apparently intentionally so in some cases), which makes them philosophically unpalatable to me. Right, like, are they trying to go against their own values? Being hard to work with defeats the whole point of free software. (At least partially: just because someone could do it in principle, doesn't mean most of us mid-wits could do it.)

On the bright side, I think we (and by "we" I mean me) can improve on this situation. Obviously, one needs something make-like, perhaps it implements a dramatically simplified build script?

You know how programming languages are an abstraction over machine code? And you know how assembly language is a programming language? What if we designed an easily-parsed, generic assembly language that could (in theory) be made to emit instructions for any instruction set architecture? Then we could cram all of the architecture-specific kernel stuff into one source file! And save on storage because our tools are sleek and slim.


Also first post! Hi guys

[JackScott]

A few quick thoughts:

• Have you looked up LLVM?
• A generic assembly language would still need some platform specific parts though. Does the target architecture have a stack? Does it only have a stack? Does it support floating point instructions? At this point it's not an assembler since it doesn't map to machine code, it's a virtual ISA (like the JVM or .NET bytecode) that we can trans-compile.
• My issue with make is more that it's not quite complex enough for what I need to do. That's why there are so many tools that sit on top of GNU Make to extend its functionality (Automake/Autoconf being one example, but also CMake etc). If make was just slightly more intelligent, a lot of those extra tools would disappear.

[ihateneedles]

Have you looked up LLVM?

Many months ago, but I haven't considered it since. If the goal of working on a language toolchain now is to reduce the amount of work that goes into future porting, I'd later have to port LLVM and a whole bunch of other things when it comes time to get the OS to build itself, right?

A first implementation of this generic assembler idea could be done for Linux in C, and then immediately rewritten in itself. Put some fix-ups and conditional build stuff at the start, maybe implement call-wrappers for the DOS API, and then it seems like we'd have a good starting point.

A generic assembly language would still need some platform specific parts though. Does the target architecture have a stack? Does it only have a stack? Does it support floating point instructions? At this point it's not an assembler since it doesn't map to machine code, it's a virtual ISA (like the JVM or .NET bytecode) that we can trans-compile.

Yeah, you can't actually get around having to do platform-specific business, you can only change how, when and where it happens. I was exaggerating a little, it's mainly x86 that causes me dread

I guess in a broad sense the idea is to put all of the architecture specific parts in one place and then abstract it away so that it's a mostly one-and-done deal, something you only have to handle when really dealing with nitty-gritty platform differences. Things that can't be regularized and have to be done manually. One doesn't need to deal with the syntax or grammar of multiple different tools. Just rerun the build with a slightly different command, and bam! you have an ARM port.

I wouldn't think the virtual ISA classification necessarily applies, as the mnemonics don't have any defined intermediate codings ie it's not a bytecode onto itself. Just a face for other instruction sets. But I think I get what you're saying.

Part of the motivation (for me) is that I'd like to use my own executable format and calling conventions in my eventual OS project, and have come to the conclusion that I need custom tools to make that happen. And 8086/286 support will be harder without it. May as well handle three issues at once and save loads of work later.

(Yes, most of the last few years has been spent on theoretical considerations rather than actually writing any code. I wanna have a detailed plan solidified so that I'm not dealing with the accumulation of feature creep and code debt later.)