Cache Kernel / Exokernel thoughts...

[BoomSukka]

Hi all,

I'm not sure to what extent anyone here has any familiarity with Exokernels and the notion of a Cache kernel, but I've been doing some reading into each lately and thought I'd make them known to those who may not have heard of them:

Cache Kernel
http://www-dsg.stanford.edu/papers/cach ... /main.html

Exokernels
http://www.pdos.lcs.mit.edu/exo.html
http://www.pdos.lcs.mit.edu/PDOS-papers.html#Exokernels

Others...
http://www.cs.washington.edu/research/p ... lwork.html


I've spent a good 10-12 hours reading through many of the research papers devoted to extensible kernels, and I have to tell you that the experience has dramatically changed my approach to designing an operating system -- most notably the approach that the Cache Kernel takes.

What exokernels and the cache kernel have in common is a belief that removing abstractions at the operating system level allows applications finer control of the hardware. I believe that the best example for the need of such a kernel is the fact that other kernels (micro or monolithic) were implemented making certain trade-offs for the most general case of an application's needs, and this can be an impedement to ideal application design. In other words, you and your application has to live with the abstractions chosen by the operating system -- always removed from the total control and low-level decision-making at the hardware level.

Heck, I'll never do these approaches any justice in this post, but I do want to encourage people to read a couple of the papers on exokernels, and then finally the paper on the cache kernel. This last paper seems to hit home with the problems today's kernels and operating systems propagate, and certainly makes you start thinking of new and exciting (IMHO) ways to design kernels and OSs that can have a REAL place today and in the future.

Oh, BTW, the reason for this post is threefold:
(1) What else can I investigate along these lines that I'm not already familiar with?
(2) I'd love to read people's opinions on this paper: http://citeseer.nj.nec.com/engler95exterminate.html
(3) Am I the only one that has any fascination with kernel/os design along the lines of the Cache Kernel (which in my opinion is a better take on the exokernel idea)?

Tim.

PS: While I'm giving out links, here's one for various os-design papers and summaries: http://www-db.stanford.edu/~manku/quals/summaries.html

[BoomSukka]

Me again... I thought I should make a point or two about these approaches in general:

Instead of a kernel abstracting the system's hardware and resources, it should multiplex them so that multiple process can properly share them at the lowest possible level.

For instance, in the cache kernel each process (they call them application kernels) recieves interrupts, handles its own physical memory etc., and the kernel merely coordinates these application kernels securely. Essentially this means that each process can do its own thread scheduling, memory management, resource management, IPC, etc COMPLETELY INDEPENDANT of other processes which may also be doing the same thing.

You could even go so far as to say every process can be an entire operating system. Wow... think about it.

The bottom line is that your cache kernel or exokernel can run multiple applications on one application OS and specialized applications can run either on another OS or run DIRECTLY against the hardware. Imagine web servers with access to physical memory and disk blocks, or multiprocess applications with power over their own thread scheduling, and don't forget that this total control is in the application space -- not kernel space...

Finally, it should be mentioned that these designs allow the os to be loaded dynamically -- they can be modified and enhanced on-the-fly.

For me, I'm thinking, now THIS gives me some creative room for designing a kernel and OS (the prospect of a dynamic OS is awesome)...

Tim.

[Curufir]

My problem with the cache/exo-kernel approach is the level of power handed over to application software. It's all well and good handing over that much control to software running on a server, because they generally run only small amounts of trusted code, but it makes no sense on a desktop. Consider how many times people have had Windows crash because of buggy applications, and this is with the kernel offering a decent amount of protection. Allow those same applications direct access to hardware and things will just get worse. In effect it would be like stepping back to DOS in terms of software protection.

It doesn't help from a compatibility aspect either. Are developers expected to write a new kernel lib for every platform they wish to support? Having the kernel offer controlled access to a virtualised machine makes a lot more sense IMO.

If someone wants the absolute last drop of efficiency from their webserver then the first thing to go is the kernel. They might as well just drop the idea of multi-tasking and include as much OS-like functionality as they need in the webserver code. If they run everything in ring 0 they'll get all the speed boost they're gonna get without better webserver code.

An interesting project might be to create a kind of wrapper for standalone applications. The kind of thing where if you compile your code with the wrapper you can dump the result to a CD, then on bootup the wrapper code just sets up the system and starts the application code. Kinda like Knoppix, but with only one application that's started automatically. It has limited uses (Routers, webservers and standalone game disks spring to mind), but is still a neat idea IMO.

[neowert]

Isnt the entire purpose of an OS to abstract hardware, so you dont need to write drivers in your app or deal with redudant lowlevel code?

[BoomSukka]

Aah, but just imagine:

You've created a kernel that does nothing but allow processes to share in the computer's resources. Then, you start writing abstractions in the form of an application OS. For each level of abstraction you could modularize; having more than one possible object to choose from at that level. Eventually you have a heirarchy of modules stemming from the kernel up to a complete abstraction environment (like a VM or a typical OS).

Now, when someone wants to design an application, they can essentially choose from which level of abstraction their application is implemented. Naturally you would have a standard set of defaults for those that aren't interested in any low-level specifics, and they would merely be implicetely 'linked' to the application. For other applications that demand, let's say, direct access to physical memory, they can either
(1) choose a different module for memory management, tweaked for their needs, or
(2) embed all memory functionality directly into the application (perhaps by writing a compatible memory management object for their specific needs)

It makes logical sense that ANY application will need varying levels of support for each resource it makes use of -- it simply isn't fair of an operating system to categorically decide beforehand. You have to understand that the application developer doesn't have to write a complete OS for their application -- merely to choose which components it will interface with.

Here's an analogy I'm fond of:

You're in a 4X strategy game, and you want to build a ship. On the ship design screen, you are presented with various hull sizes and default configurations. From here, you could take a 'stock' ship, program it and launch it, or you could alter the characteristics of the ship on a component-by-component basis. Maybe there's a laser burst weapon available, but you'd like to control it's rate of fire vs. the amount of energy in each burst. Perhaps there are three types of targeting subsystems, and you'd like to choose which is installed. Or, maybe you'd like to alter the position and number of thrusters on your ship, actually redesigning how it turns, strafes, etc. Whether you choose a default configuration or customize the heck outta it, you at least have the CHOICE.

So, in closing, here's how I'd like to develop an application:
(1) Look at each resource that I'll need, and choose the level of abstraction I'd like (default = full abstraction).
(2) For each level of abstraction with each resource, choose the module that is most closely meets my application's needs, or write my own

And if your application crashes, or even the OS under it, don't worry -- it's all simply a process in the kernel. All other operating systems / applications are running in their own kernel.

Neowert: The problem with an OS is that it chooses all abstractions for the hardware. How about dynamically choosing these abstractions for sophisticated applications?

Tim.

[Solar]

We did some research on the subject for Pro-POS, and in the end, dismissed the concept as being not suited for an end-user desktop OS. Curufir already brought up the correct points:

For server systems, where performance is paramout and the task at hand is well-known and well-defined, an exokernel gives you all the flexibility you need to do a real fine job.

But on the desktop, there would be no common look & feel, a dependency hell that makes Linux look a piece of cake, and you could absolutely forget about ever porting your OS to some other hardware.

Exokernels allow you to do libOS'es, which in turn would allow you to run Linux, MacOS, and Windows software simultaneously on the same system. "Hey great!" you say.

Now wait. Just picture it for a moment. The user saves his data under /mnt/hda5/, and loads it in Excel from D:\... and that's only one minute detail of the problems you could encounter with exokernels.

Add to this the fact that, AFAIK, the technology is still considered "beta" at best by the very ones who invented it...

[BoomSukka]

I was in the process of typing up a reply when I came accross a paper that you should check out:

http://www.osdever.net/docs/persistent_ ... ?the_id=50

Specifically, check out SECTION 4. It discusses different OS approaches.

Tim.

[mr. xsism]

Isnt the entire purpose of an OS to abstract hardware, so you dont need to write drivers in your app or deal with redudant lowlevel code?

purpose depends on target Like curufir said, servers can do this because they really never run nontrusted code. But for a user system it is different. How do you know that mirc.exe is the real thing and can be trusted?

a cache kernel would be very appropriate for industrial static task computers. They have one thing to do and no real chance of being bugged from the outside. They often have OSes that do specific things. Most of these types of computers don't need MM or tasking because their OS owns everything on the machine.

User OSes need an amount of abstraction as a form of protection. If there weren't mad hackers and crazy virii out there it wouldn't be needed because everything could be trusted. Except for maybe buggy code, but even that wouldn't be a big problem.

[Pype.Clicker]

the real problem, imho is that there are things that cannot be multiplexed without being abstracted. The best example i can see is the network card. Okay, you could just multiplex and let any program access the NIC to send packets, but when packets are received, how would you know which program should receive it without the help of abstractions like sockets ?

[Schol-R-LEA]

The multiplexing issue is less serious than I think some of you imagine, as demonstrated by the most successful 'exo-kernels' to date: VM/390 and VMware. The fact that both are primarily used to run multiple copies of other OSes will tell you something about such things usually end up in practice.

A more serious contention, IMO, is that the claims of improved efficiency are predicated on circumstances that occurs in certain types of servers but almost never does in desktop machines - a system dominated by several I/O bound processes with little or no IPC or shared resources. I wrote an earlier essay on it, the bulk of which I quote here:

I think that the exokernel as its normally presented is aimed at a false economy - most applications don't exist in a vaccuum, especially in PCs and workstations. A disk manager for a database (so the theory goes) should be different from one for a hypertext or a mail server - except that you will inevitably need to access the database records from the hypertext manager, or import a hypertext chunk into a mail message, or what have you. In an exokernel design, this can only be done by either

• providing either IPC or library sharing in the exokernel, which directly contradicts the goal of eliminating all non-essential kernel-level access,
• requiring all libraries to include the needed IPC and translation code, a ridiculous waste of disk and memory space, or
• requiring the libraries to contain a version of the code to access the shared resources directly, an even greater waste of space.

Since the latter is the design concept for exokernels in general, we see that, while the code for accessing, say, a database, may run at an order of magnitude faster and take up an order of magnitude less space, the size of the code grows exponentially with each application run on the system as a whole (since each application needs not only the code for the
application, but also the optimized - and therefore unshared - code for the system, and the code for accessing and communicating with all other applications which the app shares data with or may share data with in the future).

Furthermore, the 'economy' it offers is of little value in a single-user system. Virtually every process in a modern workstation is 'user-event bound'; that is to say, the computer does very, very little except for those periods when the user requests something directly. Much of the inefficiency of commercial OSes today, ironically enough, comes from poor usage of the this 'free time'. Many of the housekeeping and optimization tasks that currently require operator intervention could easily be automated and run as background tasks, in an OS designed to allow such. However, an exokernel design would not only make such services impossible, it would also make many of the tasks currently handled as background tasks - such as mail and print spooling - much less efficient. Distributed processing could also fill such slack time, but again, the exo design would make such operations unmanageble.

Servers face different problems - they are usually network- or IPC-bound, and thus have to have very rapid communication service - but here, too, virtually everything depends on trap/interrupt services, which on most platforms are always kernel level, and thus are the slowest operations on an exokernel (which must make [i/]two[/i] level changes, one to handle the interrupt and one to pass the result back to the 'unsecure driver', even before it is handled - admittedly, this is a platform-dependent issue, and microkernel designs suffer the same flaw). This is less of problem than in other areas, so it may be that dedicated servers would be a suitable application for the exokernel design, but it is still a serious flaw.

Mind you, I do think that the idea is any intriguing one, and one which has some merit; but the designs I know of them have serious flaws.

BTW, on a related subject, I was wondering if any of you were familiar with the Synthesis Kernel. While the idea of on-the-fly optimized code generation has a definite appeal to a LISPer like me, I can't help but wonder how the ratio of synthesis overhead to speed improvement changes with different hardware configurations, especially the modern Intel platform (which is notoriously difficult to generate efficient code for).