OSDev.org

As a programmer, I write a lot of software. Often, when I decide to write a new tool, I have conflicting self-interests. I want to develop it for my operating system (just to build up the collection of usable software on my OS - because that is cool), but I also want to use the software I write in practical ways - like in every day situations at work. However, my OS is far from being usable as an everyday OS, and there are limitations to running it in a Virtual Machine (for example, it's lot of effort involved in writing drivers to allow a guest OS to access files on the host.)

We often talk about the portability of our operating systems between different architectures. Some of these architectures require creative solutions (such as working out process protection/memory management on MMU-less architectures.) But, if we're able to write a system that is so portable, then it seems logical to me that is would be possible to port our system as a user-mode program that runs underneath another OS.

There would be technical challenges to overcome, but also many simplifications (no need to write device drivers - free graphics acceleration, etc.)

Why do I want to do this?

I envision the ability to develop an awesome presentation tool for my operating system, and be able to launch it at work on demand. Or, offer a download link to a Windows version of the tool on my website, which is effectively the tool bundled with a user mode version of my OS.

My first thought was that I could develop my tools in a portable way, which is great, however it really appeals to me be able to directly run binaries for my OS on a more mainstream OS. Also, I use a lot of custom libraries and APIs, and the effort to port the runtime/system libraries appears greater than the effort to port my kernel to user mode - the latter would only require plugging in platform wrappers at the lowest common-denominator level.

Actually, this is something I've been working on for a while.

I have written a wrapper around chunks of my kernel code (terminals, VFS) that exposes syscalls via (currently) TCP IPC. The applications are either loaded by a custom loader (that shares code with my dynamic linker), or can be compiled as linux/windows programs and linked against a library that provides my syscalls as IPC.

The biggest upside here (apart from a faster turnaround for developing userland code) is that "natively" linked versions (using the library instead of loader) can be run using valgrind/gdb and debugged very simply.

Of course, the biggest downside is emulating a pre-emptive multitasking kernel in a single userland process.

Source Link

My first reaction was to write:

Check out the first link and the video here: http://forum.osdev.org/viewtopic.php?f=15&t=27293. The system described has been implemented both as an OS, and as an application on pretty much every desktop OS since the mid 1980s. The VM implemented on bare metal is your OS, the same VM in user space is your distributable OS.

but then I noticed this

Which made me unsure. I'm not sure I understood what you wrote. What do you mean by "underneath"? Do you mean user-mode sofware hooked into this "another OS"?

ps Since you're interesting in presentation tools, the presention tool used here is based on the VM described above, so he's using the system to present the system.

Kernel has a different development objective than usual application, however some modules may well be reused.
In fact I am using the same module (with minor adjustments) of VFS, window manager, and others, in some of my applications.

If you are talking about running your OS on top of other OS, it's way more easier to use VM.

While you may have limitations regarding your choice of scheduler, I don't see why you could not use threads provided by the host OS?

My thoughts exactly. The portable/reusable parts of my code are much larger than the non-portable/hardware-dependent parts.

If I want my applications to run under another OS - unmodified, it makes much more sense to me to port my kernel, treating it as a VM, than to re-implement everything twice as a library. There are advantages to this - a single code-base that is easy to maintain, allowing any of your OS's applications to run seamlessly on another OS.

I'm fantasizing about one day when I can run:

./kernel someawesomeprogram.prog

or assigning .prog programs to open with my kernel, and it runs like any other native program.

I am not sure if I understand the point exactly. Kernels need kernel mode by definition.

I understand your goal though. Are you thinking of a Cygwin-like approach (where an emulation layer is put on top of the host OS) or a coLinux-like approach (which is technically not a user mode program but does serve your goal)?

I don't think you can run many parts of your kernel in user mode in another OS because of the restrictions of user mode. Running your kernel in an emulation is the way to get all the privileged functionality again.

Do you want to run your user programs for your OS in another OS? Then you should write an emulation layer.

If course you can try to do something in between, partial kernel emulation in user space but you are likely to be forced to rewrite many parts.

No, users want to do this:

or even better, just launch with shortcut icon.

Check how different platform provide emulation layers for other OS:
- FeeeBSD
- Cygwin on windows
- wine

Or in general, user also accepted the virtual machine approach (Parallel or VMWare on mac), which seamlessly integrate two OS.

I'm thinking like coLinux.

I'm thinking about something like a binary emulation layer or virtual machine when compiled as user mode program, and a kernel when compiled as bootable binary. Yet they share the same source base - you would abstract away the platform specific code:

Memory management, scheduling, etc is shared with the hardware platforms.
Binary emulation, hardware passthrough, is shared with the user mode platforms.

The majority of your OS stays the same (programs, services, libraries and APIs, window manager, etc.)

Seems like a great way to get decoupling of hardware specific code. There's no reason not to write your OS in a way that makes it totally irrelevant what hardware, VM or master OS it's executed on, basically the scheduling would have some set of criteria what it needs to be able to function, and if the OS is compiled to run on x86 that would include x86 hardware handling for the timer and whatnot, otherwise something else.

The NetBSD rump kernel http://rumpkernel.org can run in userspace in many other operating systems, which is great for debugging. It uses the host OS memory management, and optionally threading, but everything else is the OS. I would recommend any developer to start with something that runs in userspace - there is a lot of code you can delay writing until later. The rump kernel is unmodified NetBSD code, so it is definitely possible to do this with an OS.

It's exactly the Java way. Command line would be like this:

And it is obviously possible to create shortcut for such command (indeed the shortcuts are already in play as Java application launchers).

And now one point about implementation. There is a space between emulation of complete native system and creation of a small set of stubs for highly "componentized" OS. In this space VM based approach is closer to the last solution. It requires just simple bytecode emulator with relatively small native stubs. In case of Java I managed to run some OS's code during system image build and it was relatively easy because of already existing bytecode execution environment (Java) and small and separated set of native methods. So, if you have a VM - just implement some simple bytecode emulator and create a bit of stubs to cover native calls.

Thanks embryo.

My OS does use bytecode. But, there is more to my OS than just the virtual machine. The OS will come with standard libraries, file system, networking, window management, standard directory layouts, etc. I would rather have my OS that can be compiled to run on either baremetal or as a usermode program, than having two separate programs (a baremetal kernel, and a user mode VM).

The complexity will be at least comparable with the VM variant. But if you like it - why not?