OSDev.org

The Place to Start for Operating System Developers
It is currently Thu Feb 25, 2021 7:13 pm

All times are UTC - 6 hours




Post new topic Reply to topic  [ 64 posts ]  Go to page Previous  1, 2, 3, 4, 5  Next
Author Message
 Post subject: Re: Designs of microkernels
PostPosted: Thu Jan 07, 2021 5:43 pm 
Offline
Member
Member

Joined: Wed Apr 01, 2020 4:59 pm
Posts: 62
AndrewAPrice wrote:
There are a lot of implementation details we'd need to figure out if we want to do function calls between processes.

True. Here are some ideas:

AndrewAPrice wrote:
Can you share memory between processes? (Let's say I allocate a class in Process A, can Process B access it and call methods on it?)

I don't see why not. If process B has a pointer to the object, it must be because process A explicitly gave it one. Shared memory is an essential form of IPC.

AndrewAPrice wrote:
How do you determine what process owns a chunk of memory?
If ownership of objects can be moved, could a malicious program keep calling a critical service with a large payload?


I don't think it's very interesting to let processes own memory. Modern allocator designs based around thread-local caches are well-suited to such a design as this one. (Incidentall, memory management as a whole becomes simpler and faster in an SASOS. You still want to have a multi-level allocator—first pages, or superpages; then individual objects—to avoid false sharing problems, but the second level isn't slow. That means the allocator has less incentive to keep around extra pages, which means less overall memory usage.) Mimalloc's design works especially well here; it allows you to efficiently free objects originating in other threads. But even classic tcmalloc/jemalloc will work well.

AndrewAPrice wrote:
How do you handle terminating processes? (E.g. Imagine Process A calls Process B which calls Process C, but B terminates while C is still doing something. What will happen when C returns?)

Options:

  1. Make all messages async. Have a synchronous call to 'wait for any message to arrive along any channel', or one to 'wait for any message to arrive along this specific channel'. If a process dies, all its channels are destroyed. If you wait for a destroyed channel, you get a signal (or some other form of error).
  2. Same as (1) but don't destroy a process's channels when it dies. Instead, keep around the channels and let the process transparently resume and continue communicating along them. Prior art: erlang.
  3. Messages are synchronous (more like a classic function call). If a process encounters an error, that error is returned or signaled as it would be if the error happened in the same process

AndrewAPrice wrote:
What happens if Process A implements an interface, sends the object to Process B, then A terminates, and B tries to call a virtual method on the interface?

Like I said previously, all memory can be shared. Since the code implementing the interface still has references to it, it isn't freed when A terminates it, and B is free to continue calling it. (Or the code is copied (or COWed), depending on the semantics you want. Same story either way.)

AndrewAPrice wrote:
What happens if Process A is updated and the exported function's signature changes, but Process B is a commercial closed source application and the developers haven't rebuilt it for the newer API, and it tries to call Process A with the wrong function signature?

Express APIs abstractly in on-disc binaries. The runtime is responsible for doing zero-copy (de)serialization, which gives you the same performance but lets you verify API calls at startup time.


Top
 Profile  
 
 Post subject: Re: Designs of microkernels
PostPosted: Fri Jan 08, 2021 1:01 pm 
Offline
Member
Member
User avatar

Joined: Mon Jun 05, 2006 11:00 pm
Posts: 2138
Location: USA (and Australia)
I was thinking that you could garbage collect code.

The theory being, that let's say you were using a GUI app and close the window, the program will terminate worker threads that it owns, and code that is no longer reachable can be garbage collected.

It's possible that your program doesn't completely clean up (this would be relatively easy to detect - tag allocated code with the program it belonged to, and if the program should have terminated, you could see if the any allocated code still remains that belonged to that program.) I don't know how you could safely force kill this code if something is still pointing to this code (a callback, or an interface implentation) and tries to call it.

At some point, you might not care. If this is a personal computer, you can just notify the user that some app is staying resident, and they should save their work but soon reboot. Maybe have a feedback mechanism that tags this app as being naughty, and a warning shows up in your app store when other users try to launch it.

---

If you're going with your own language that everything must be built with, you could get creative with language-level security. All code exists in a global repository (it could be in bytecode form or source form.) Every namespace/package/whatever is prefixed with the owner's username. You could have visibility rules that specify which namespaces can touch what functions/methods/members/etc.

_________________
My OS is Perception.


Top
 Profile  
 
 Post subject: Re: Designs of microkernels
PostPosted: Fri Jan 08, 2021 4:37 pm 
Offline
Member
Member

Joined: Wed Apr 01, 2020 4:59 pm
Posts: 62
AndrewAPrice wrote:
If you're going with your own language that everything must be built with, you could get creative with language-level security. All code exists in a global repository (it could be in bytecode form or source form.) Every namespace/package/whatever is prefixed with the owner's username. You could have visibility rules that specify which namespaces can touch what functions/methods/members/etc.


Relevant (erlang again... :)

My feeling is that security should be done with a capability-based system, with thread granularity rather than namespace granularity; but that that can be mostly separate from modularization.


Top
 Profile  
 
 Post subject: Re: Designs of microkernels
PostPosted: Sat Jan 09, 2021 7:38 am 
Offline
Member
Member
User avatar

Joined: Tue Sep 15, 2020 8:07 am
Posts: 218
Location: London, UK
I’ve been desperate to join in this thread as I love Microkernel designs. But my specific choice of design is very specific and I don’t think it will scale well for general purpose use cases, so I’m keen not to dilute the discussion unhelpfully.

I think I will add that while I will always choose a pure micro kernel over other designs, I actually think the Hybrid kernel has demonstrated more practicality in real world usage, over both monolithic and micro kernel architectures. I personally believe that all the major kernels in large scale use now are hybrid kernels.

-edit- So the point I totally forgot to make, was don’t get too hung up on what other micro kernels have done, just try and solve your problems as best you can, and implement the features which work best for your goals! :)

_________________
Building a single address space Microkernel, as used in embedded applications, for the desktop... Download latest build bootable Disk Image: https://github.com/h5n1xp/CuriOS/blob/main/disk.img.zip


Top
 Profile  
 
 Post subject: Re: Designs of microkernels
PostPosted: Sat Jan 09, 2021 7:52 am 
Offline
Member
Member
User avatar

Joined: Tue Feb 18, 2020 3:29 pm
Posts: 639
bloodline wrote:
I’ve been desperate to join in this thread as I love Microkernel designs. But my specific choice of design is very specific and I don’t think it will scale well for general purpose use cases, so I’m keen to to dilute the discussion unhelpfully.

I think I will add that while I will always choose a pure micro kernel over other designs, I actually think the Hybrid kernel has demonstrated more practicality in real world usage, over both monolithic and micro kernel architectures. I personally believe that all the major kernels in large scale use now are hybrid kernels.

-edit- So the point I totally forgot to make, was don’t get too hung up on what other micro kernels have done, just try and solve your problems as best you can, and implement the features which work best for your goals! :)

So for me, I want a high performance, high security distributed system. So I will choose a hybrid kernel with a powerful distributed IPC. I think this will let me get closer to my goal.

_________________
Currently working on the Nexware project, an attempt to make a less bloated version of GNU. All repos for it can be found at https://github.com/Nexware-Project.


Top
 Profile  
 
 Post subject: Re: Designs of microkernels
PostPosted: Sat Jan 09, 2021 12:43 pm 
Offline
Member
Member
User avatar

Joined: Mon May 22, 2017 5:56 am
Posts: 546
AndrewAPrice wrote:
I was thinking that you could garbage collect code.

I've noticed Linux without any configured swap space finding memory from seemingly nowhere, then acting like it's swapping when I tried to resume suspended processes. It must have reallocated the physical memory used by inactive parts of text and maybe rodata sections of those binaries, then reloaded them from the executable files when they were resumed. It makes sense to me as Linux and other OSs (incl. Plan 9) demand-load executables; they only load pages which are used.

Or do you mean gabage collecting the generated code of the Synthesis kernel? We have another thread for that if you want to look at it.

_________________
Kaph link pending. code pending. design pending. plans in a state of flux! everything pending! choice of language still up in the air! why is nothing coming together?!?


Top
 Profile  
 
 Post subject: Re: Designs of microkernels
PostPosted: Sat Jan 09, 2021 12:51 pm 
Offline
Member
Member
User avatar

Joined: Fri Nov 22, 2019 5:46 am
Posts: 567
eekee wrote:
I've noticed Linux without any configured swap space finding memory from seemingly nowhere, then acting like it's swapping when I tried to resume suspended processes.

As far as I know Linux uses always swap space. If it doesn't find a swap partition, it uses a file for that.

Greetings
Peter

_________________
https://wiki.osdev.org/User:PeterX


Top
 Profile  
 
 Post subject: Re: Designs of microkernels
PostPosted: Sat Jan 09, 2021 1:37 pm 
Offline
Member
Member
User avatar

Joined: Mon May 22, 2017 5:56 am
Posts: 546
PeterX wrote:
eekee wrote:
I've noticed Linux without any configured swap space finding memory from seemingly nowhere, then acting like it's swapping when I tried to resume suspended processes.

As far as I know Linux uses always swap space. If it doesn't find a swap partition, it uses a file for that.

Greetings
Peter

Thanks, but it didn't when I was working with it. It could be configured to use files, but it wasn't automatic. Uh... maybe the kernel started making anonymous files (inode but no dir entry) for swap in the years since then. It certainly wouldn't make named files, that would go against Torvalds' policy of "no policy in the kernel," (he even rejected devfs,) but I can very much imagine distro init systems automatically creating files for swap even if it's unnecessary. (They can be annoying that way.)

_________________
Kaph link pending. code pending. design pending. plans in a state of flux! everything pending! choice of language still up in the air! why is nothing coming together?!?


Top
 Profile  
 
 Post subject: Re: Designs of microkernels
PostPosted: Sat Jan 09, 2021 2:02 pm 
Offline
Member
Member
User avatar

Joined: Thu Oct 13, 2016 4:55 pm
Posts: 1266
PeterX wrote:
As far as I know Linux uses always swap space. If it doesn't find a swap partition, it uses a file for that.
Actually you have to manually specify the file, use "mkswap" on it and add it to /etc/fstab or run "swapon" manually. If no swap is configured (/proc/swaps is empty), then the Linux kernel does not use any swap at all. It is true though that most Linux distribution installers configure a swap file or partition by default, however you can turn those off with "swapoff" (or by removing its entry from fstab and reboot).

Cheers,
bzt


Top
 Profile  
 
 Post subject: Re: Designs of microkernels
PostPosted: Mon Jan 11, 2021 4:42 am 
Offline
Member
Member

Joined: Mon Jul 05, 2010 4:15 pm
Posts: 587
PeterX wrote:
As far as I know Linux uses always swap space. If it doesn't find a swap partition, it uses a file for that.


The swap file can be turned off and there are many embedded Linux systems where the swap file is turned off. For example if the flash is filled almost to the capacity or if the system has a required lifespan so you want to reduce the writes. Also as soon the system starts to swap to the file, the system usually becomes very slow which degrades the user experience so many turns it off because of this. With embedded systems you can more easily predict the amount of memory required.

When there is no swap file Linux will still try to find pages it can reuse and that is usually read-only pages like code it can take when memory is getting low.


Top
 Profile  
 
 Post subject: Re: Designs of microkernels
PostPosted: Mon Jan 11, 2021 8:32 am 
Offline
Member
Member

Joined: Tue Apr 03, 2018 2:44 am
Posts: 152
OSwhatever wrote:
PeterX wrote:
As far as I know Linux uses always swap space. If it doesn't find a swap partition, it uses a file for that.


When there is no swap file Linux will still try to find pages it can reuse and that is usually read-only pages like code it can take when memory is getting low.


That said, not being able to swap anonymous data pages really causes problems, as we'd end up thrashing read-only executable file pages.


Top
 Profile  
 
 Post subject: Re: Designs of microkernels
PostPosted: Wed Jan 13, 2021 9:12 am 
Offline
Member
Member
User avatar

Joined: Mon Jun 05, 2006 11:00 pm
Posts: 2138
Location: USA (and Australia)
@eekee: Sorry, I was meaning just as we garbage collect objects by tracing reachable objects, we could do something similar with code.

So let's say your whole OS was one giant VM in a custom language where everything belongs in a global repository. Let's say you double click on "Calculator" on the desktop and it loads in the calculator code and executes andrewaprice.utils.Calculator.main() in the VM.

This main() function creates a window with buttons. By adding the window to the window manager, there exists a reference to the Calculator's window object and the code for the various handlers for keyboard input and UI buttons, and so this code will stay in memory.

If you close the calculator window (removing it from the window manager), the Calculator's code is no longer reachable and thus can be unloaded.

This system would have automatic function-granularity dead-code elimination. Functions that can't be reached can be unloaded from memory.

---

If you had a system like this, drivers could simply implement an interface.

Security could be language level, e.g. some way of requesting
"is andrewaprice.utils.Calculator.* allowed to call system.fileSystem.*?"

We'd want to figure out a way of dealing with forcefully unloading code that doesn't want to clean up. (Maybe this is not as big of an issue as I'm thinking.)

_________________
My OS is Perception.


Top
 Profile  
 
 Post subject: Re: Designs of microkernels
PostPosted: Wed Jan 13, 2021 10:29 am 
Offline
Member
Member
User avatar

Joined: Thu Oct 13, 2016 4:55 pm
Posts: 1266
AndrewAPrice wrote:
If you close the calculator window (removing it from the window manager), the Calculator's code is no longer reachable and thus can be unloaded.
...
We'd want to figure out a way of dealing with forcefully unloading code that doesn't want to clean up. (Maybe this is not as big of an issue as I'm thinking.)
Probably. All you need is a reference counter. When the counter is decreased to zero, you can remove the code from memory (you could store a timestamp and call its clean up, then only forcefully remove those which haven't finished for a long time. This is typical in GCs).

Cheers,
bzt


Top
 Profile  
 
 Post subject: Re: Designs of microkernels
PostPosted: Wed Jan 13, 2021 10:43 am 
Offline
Member
Member
User avatar

Joined: Mon Jun 05, 2006 11:00 pm
Posts: 2138
Location: USA (and Australia)
I was thinking about garbage collecting code - what about static and global variables? (Do they reset upon being unreachable?) Maybe we forbid them (in our custom language), and have a root 'System' object which devices, the vfs, window manager, etc, live under that we pass into a program's main?

_________________
My OS is Perception.


Top
 Profile  
 
 Post subject: Re: Designs of microkernels
PostPosted: Wed Jan 13, 2021 10:59 am 
Offline
Member
Member

Joined: Tue Apr 03, 2018 2:44 am
Posts: 152
AndrewAPrice wrote:
I was thinking about garbage collecting code - what about static and global variables? (Do they reset upon being unreachable?) Maybe we forbid them (in our custom language), and have a root 'System' object which devices, the vfs, window manager, etc, live under that we pass into a program's main?


Or port a JVM. Perhaps we could call it JavaOS :)

Joking aside, JavaOS on modern HW, with modern RAM levels and JIT would probably work pretty well.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 64 posts ]  Go to page Previous  1, 2, 3, 4, 5  Next

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group