Operating systems research advice

Discussions on more advanced topics such as monolithic vs micro-kernels, transactional memory models, and paging vs segmentation should go here. Use this forum to expand and improve the wiki!
Post Reply
BobJoe
Posts: 2
Joined: Mon Aug 17, 2026 5:03 pm

Operating systems research advice

Post by BobJoe »

Due to one of my classes in school, and mainly personal interest, I need to do research on a topic. I have an interest in operating systems, so I want to do research on operating system design. I have been trying to find and read existing research on this topic (experimental design for both for entire kernels or for some subsystem in the kernel). From this, I am hoping to be able to find an area that has not been researched in detail yet, and I want to do research on this (e.g. performance compared to standard designs). Also, I have a bit of experience with basic operating system development and have been working on my own hobby operating system for around a year now, and from this I have what I feel is a basic understanding of the main concepts in operating systems.

The reason I am making this post is to inquire whether there are any resources I can use to find these gaps in research easier / quicker, or places where I can go to get more advice on this topic, in case this type of post isn't for this site.
User avatar
Demindiro
Member
Member
Posts: 165
Joined: Fri Jun 11, 2021 6:02 am
Libera.chat IRC: demindiro
Location: Belgium
Contact:

Re: Operating systems research advice

Post by Demindiro »

I believe some underexplored areas are:
  • Multikernels; and
  • Kernels designed around IOMMUs
  • Architectures with hardware capabilities.
None of these areas are truly new and have existed in some form for a long time, but they do seem to be forgotten about.

Multikernels
The basic idea is: instead of one kernel for an entire system, have one kernel for each core of a system and use message passing between each kernel. It is aimed at addressing scalability problems as more and more cores and memory is crammed into a single system. I believe it would also be a very good fit for heterogenous processors (e.g. BIG.little) as with some processors each core may have different feature sets and/or accelerators.

Barrelfish OS seems to have coined the term, so that is a good starting point.

IOMMUs
IOMMUs have been around for a while, yet I feel no kernel takes full advantage of it. Micro- and exokernels in particular could benefit from it as it allows software to access hardware directly while still enforcing isolation between processes.

Hardware capabilities
Basically: tagging pointers with permissions and a length. CHERI is a notable example that is mostly compatible with software written for current architectures, but you can to more extreme lengths and e.g. enforce a distinction between addresses and "normal" values. It is closer to HW (or VM) than OS design though.

----------

If I had to pick one of these topics I'd go for multikernels. I think it'd be the most rewarding in terms of effort, though I wouldn't expect too much from any of the topics I listed.
GeneSYS exokernel (Codeberg)
Lemmings! micro-/multikernel (Github, Codeberg)
Waddle container tool (Codeberg)
BobJoe
Posts: 2
Joined: Mon Aug 17, 2026 5:03 pm

Re: Operating systems research advice

Post by BobJoe »

@Demindiro Thank you so much for the guidance man! Truly appreciate it :D
Post Reply