OSDev.org

Would it be possible to have a userspace program (on a unix os), to do the reading of the executable into memory yourself, and then tell the OS to skip that step and run it? For this (admittedly strangely specific) use case, the program would generate the executable itself directly into memory.

I figure an option could be to make some sort of pseudo-fs and trick the os into thinking it's reading from a file, but it's a very clunky way of doing things.

Thanks

Yes.

You mmap the portion of the file that has the executable code as executable (PROT_EXEC in mmap), do whatever relocations are required to make the code runnable at the address it is located, then jump to it.

This is basically what a dynamic linker is doing. When a dynamically linked ELF binary is loaded, the binary specifies an "interpreter", which the kernel loads and jumps to instead. The interpreter (typically something like /lib64/ld-linux-x86-64.so.2 on Linux) then loads in the actual binary (if required) and any libraries pulled in as dependencies, using mmap above, fixes up what is required to dynamic link functions and data, and jumps to the binary entry point.

But it sounds like what you're doing is some sort of Just In Time compilation (JIT), which you can do with correctly protected memory (mprotect with PROT_EXEC again). Language VMs do this all the time.

I couldn't recommend a labguage VM to look at in inspiration, but you can browse the source on any of the following open source language VMs:

• https://en.wikipedia.org/wiki/LuaJIT
• https://en.wikipedia.org/wiki/V8_(JavaScript_engine)
• https://en.wikipedia.org/wiki/SpiderMonkey

Andreas Kling (SerenityOS) also has an offshoot Javascript library with JIT, along with videos of hacking on the JIT library that you may find useful:

https://youtube.com/playlist?list=PLMOp ... kfUlLgq8dO