omarrx024 wrote:
The second option is mostly a project of its own, and doesn't belong with OSDev, really.
If you're not averse to using existing, well-tested, third-party code in your OS (I'm not going to get into the debate about whether you should do this, but I have no issue with it; I see myself as the "architect" of my OS, not the "bricklayer"), the emulation option becomes by
far the simplest and most compatible option. The
"libx86emu" emulator is pretty trivial to port and has been in widespread use for some time (forming part of SciTech's graphics driver products and the driver layers of XFree86/Xorg; used extensively on non-x86 platforms to run video BIOS code). It's not exactly the most performant emulator ever, but there's not a lot of real-mode x86 code that actually needs to be run at full speed on a modern CPU.
Also, while it's more complex, it
is possible to use V86 mode in a "mostly 64-bit" OS by switching to 32-bit pmode as an intermediate step. Unofficial patches exist to implement this on Linux. However, some recent CPUs have shipped with bugs in their V86 mode implementations, so I'd still recommend emulation as a more future-proof solution.