OSDev.org

The Place to Start for Operating System Developers
It is currently Fri Mar 24, 2017 9:58 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 25 posts ]  Go to page Previous  1, 2
Author Message
 Post subject: Re: Memory Segmentation in the x86 platform and elsewhere
PostPosted: Sun Feb 05, 2017 11:13 am 
Offline
Member
Member

Joined: Wed Oct 01, 2008 1:55 pm
Posts: 2173
Octocontrabass wrote:
I'm actually aiming for more of a hybrid kernel, where components that can do a lot of damage regardless of separation (like the memory manager) are part of the kernel since the benefits of separation do not outweigh the costs. This means system calls for things like memory allocation don't require an address space switch at all.

For things like passing messages between tasks, the kernel has no need to validate the contents of the message; it only checks to make sure that the two tasks are allowed to communicate. It's up to the receiver of the message to validate its contents. What's to stop a driver from making fraudulent requests in your system?


I use registered entry points both for functions that are callable from userland and drivers. These are either patched to call-gates (userland), or with the actual 48-bit address when they are called. Sure, it is always possible to code direct far-jumps, but then malicious manipulation is not in the scope of the design. Normal requests to other drivers (like memory allocation) always are done with a syscall macro which is then translated on first use to a far call. Drivers are also separately linked, so no huge kernel binary.

Octocontrabass wrote:
Don't ignore the costs of using segments with non-zero base addresses. Remember, on modern x86, that adds a lot of overhead too. I can work towards a system that minimizes address space switches; there is nothing you can do to remove segmentation overhead.


Certainly. The x86-64 solution would have better performance, but poorer locality (more TLB misses).

Octocontrabass wrote:
The use of separate address spaces means my drivers can't corrupt each another, too. So, where is my design lacking in adequate protection? Where does your design prevent malicious code from guessing the right segment to use to access someone else's data?


As I already wrote, preventing malicious manipulation is not the goal. Rather, I solve this issue by not letting people install drivers freely (in production versions).

Octocontrabass wrote:
rdos wrote:
Nothing, but just like segmentation, it narrows the scope of bugs which means they are found quicker and are less likely to remain in release versions as fatal errors.

So you're saying that RDOS is as easy to compromise as MS-DOS?


Not really. RDOS uses paging for process separation and runs userland at ring 3. MS-DOS actually doesn't use segmentation in a smart way. It is only used to extended the address range, and cannot be used for limit checking.


Top
 Profile  
 
 Post subject: Re: Memory Segmentation in the x86 platform and elsewhere
PostPosted: Sun Feb 05, 2017 12:34 pm 
Offline
Member
Member
User avatar

Joined: Fri Oct 27, 2006 9:42 am
Posts: 663
Location: Athens, GA
@rdos: OK, I think I am slowly getting a clearer picture here.

First, I think we have mostly misunderstood how you were using segmentation, and why. The fact that it applies specifically to kernel drivers, rather than applications, is a crucial factor I (and I suspect most others here) misunderstood.

Also, the fact that you were discussing it in terms of software development and debugging rather than protection from malicious code - and that malicious code was not a significant consideration in the first place - puts most of us in a very unfamiliar position, as it means something most of us see as a primary concern isn't even in the picture. A stand-alone system (or closed set of nodes) with a tightly controlled software environment and no real chance of introducing unmonitored code (unless the system is physically compromised in ways that would make OS security meaningless) is a situation that most OS devs haven't seen since the early 1980s at the latest.

What I am saying is that we have been debating at cross purposes: most of the others, including myself, have been talking about general design strategies, and focused on prevention of malicious code, whereas you are discussing the specific case of an RTOS which a) does not run general applications from unvetted sources (and presumably only communicates with other nodes of the same type, if it is doing any networking at all), b) is designed for a specific platform with no intention of portability, and c) is meant to address a set of constraints where RT performance is the key factor (meaning that it has to take actions in very specific timing ranges - not necessarily as fast as possible, but within the window between t and t' without fail).

One of the things this means is that the use of virtual memory, while not impossible, must be limited to processes that lack RT constraints, and page-ins and page-outs cannot be initiated if there is a chance that the processing time of a page request will encroach on a RT window, among other things. As a rule, RT and virtual memory are pretty much incompatible, so using paging for any core operations except in a supporting role would be an unattractive proposition.

The fact that portability isn't on the horizon at all also changes a lot of things, but that isn't the real key factor here.

_________________
Rev. First Speaker Schol-R-LEA;2 LCF ELF JAM POEE KoR KCO PPWMTF TGIF
Verbum, a boot sector code example
Lisp programmers tend to seem very odd to outsiders, just like anyone else who has had a religious experience they can't quite explain to others.


Top
 Profile  
 
 Post subject: Re: Memory Segmentation in the x86 platform and elsewhere
PostPosted: Sun Feb 05, 2017 12:59 pm 
Offline
Member
Member
User avatar

Joined: Fri Oct 27, 2006 9:42 am
Posts: 663
Location: Athens, GA
rdos wrote:
I use registered entry points both for functions that are callable from userland and drivers. These are either patched to call-gates (userland), or with the actual 48-bit address when they are called.


EDIT: never mind, on re-reading your statement it isn't what I was thinking. I'll keep the posted text in case anyone read it already, but I can see I was filtering it through my own expectations.

The former method sounds a bit like a C-list to me (a common way of implementing capabilities), except it also sounds like you aren't generating them using a hash - not that you would need to, given the security situation already discussed.

This is also pretty close to how Synthesis passes a handle to a synthesized code unit to a quaject:

Masslin, Pu, and Ioannidis wrote:
Synthesized code is protected through memory management. Each address space has its own page table, and synthesized code is placed in protected pages, inaccessible to the user program. To prevent the user program from tricking the kernel into executing code outside the protected pages, the synthesized routines are accessed via a jump table in the protected area of the address space. Since the user program can only specify an index into this table, the synthesized routines are entered at the proper entry points. This protection mechanism is similar to C-lists to prevent the forgery of capabilities.


I have been considering doing the same, but I am more inclined to use a more general closure-based capability model and have access to synthesized code as a special case of that - I am increasingly thinking that what we usually call capabilities can have a more general role than security alone.

_________________
Rev. First Speaker Schol-R-LEA;2 LCF ELF JAM POEE KoR KCO PPWMTF TGIF
Verbum, a boot sector code example
Lisp programmers tend to seem very odd to outsiders, just like anyone else who has had a religious experience they can't quite explain to others.


Top
 Profile  
 
 Post subject: Re: Memory Segmentation in the x86 platform and elsewhere
PostPosted: Sun Feb 05, 2017 2:27 pm 
Online
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 1010
rdos wrote:
Sure, it is always possible to code direct far-jumps, but then malicious manipulation is not in the scope of the design.

So, there is nothing preventing a driver from doing anything malicious.

rdos wrote:
As I already wrote, preventing malicious manipulation is not the goal. Rather, I solve this issue by not letting people install drivers freely (in production versions).

What prevents drivers from being installed?

rdos wrote:
Not really. RDOS uses paging for process separation and runs userland at ring 3. MS-DOS actually doesn't use segmentation in a smart way. It is only used to extended the address range, and cannot be used for limit checking.

I'm using MS-DOS as an example of an operating system that can easily be compromised. If you'd prefer, I could use classic Mac OS as my example instead.

Schol-R-LEA wrote:
RTOS

I don't see anything on this page that suggests RDOS is designed for real-time operation, but it does mention "segment protection and paging to provide a stable and secure system". You're not confusing Leif Ekblad's RDOS with Data General's RDOS, are you?


Top
 Profile  
 
 Post subject: Re: Memory Segmentation in the x86 platform and elsewhere
PostPosted: Sun Feb 05, 2017 3:22 pm 
Offline
Member
Member
User avatar

Joined: Fri Oct 27, 2006 9:42 am
Posts: 663
Location: Athens, GA
Octocontrabass wrote:
Schol-R-LEA wrote:
RTOS

I don't see anything on this page that suggests RDOS is designed for real-time operation, but it does mention "segment protection and paging to provide a stable and secure system". You're not confusing Leif Ekblad's RDOS with Data General's RDOS, are you?


I'm going by what Leif has said here, namely that the primary use is in embedded control systems. Leif states flat out that:
Rewrite from Scratch
rdos wrote:
My initial design goal was to write an OS that was compatible with MS-DOS, with multithreading and multiprocess support (the "r" in RDOS stands for "realtime DOS").


The primary use case is stated here:

Best processor for 32-bit [rd]OS - a.k.a RDOS-OS is best (I wish I had seen this thread earlier, as it covers quite a lot of what we've discussed here)
rdos wrote:
Solar, I have 15+ years of experience professional embedded system development for petrol stations, and I know what works and what doesn't, and there is no design limitations in RDOS in this regard. The API mostly was designed during the last 15 years, and adapted to what I regard best practices for such applications. So, the API is the consequence of my experience in the area, not a baggage to overcome. Therefore, I don't need to defend anything.


The limited scope for the system is stated here:

OS Graphics
rdos wrote:
I have no general end-users that can do things like that anyway, so it is out-of-scope for my design.


However, he has stated elsewhere that RDOS is not hard real-time, something I wasn't certain about until I read this:


OdinOS: I'd love some design feedback
rdos wrote:
At one time I planned to add hard real-time extensions. My idea was to reserve a single core for real-time tasks that would not use preemption, and that would receive no hardware IRQs. In my OS, I use IRQ balancing to even out load, so I regularly change IRQ assignments between cores to achieve even load. This works because many IRQs will schedule a thread on the core the IRQ executes on. Threads are mostly sticky, so if they start executing on one core, they would stay on that core, unless the load balancer moves them to the global thread queue where any core can pick them from. The load balancer works on the time scale of 100s of milliseconds, so moving threads have little effect on performance.


Mind you, had I seen the thread "Your exprience on Segmentation vs. Paging", and remembered the discussion in "How each process can have same kernel address space?", I would have known that most if not all of this had been hashed out earlier, rather a critical research failure on my part.

_________________
Rev. First Speaker Schol-R-LEA;2 LCF ELF JAM POEE KoR KCO PPWMTF TGIF
Verbum, a boot sector code example
Lisp programmers tend to seem very odd to outsiders, just like anyone else who has had a religious experience they can't quite explain to others.


Last edited by Schol-R-LEA on Sun Feb 05, 2017 3:42 pm, edited 2 times in total.

Top
 Profile  
 
 Post subject: Re: Memory Segmentation in the x86 platform and elsewhere
PostPosted: Sun Feb 05, 2017 3:30 pm 
Offline
Member
Member

Joined: Wed Oct 01, 2008 1:55 pm
Posts: 2173
Octocontrabass wrote:
rdos wrote:
As I already wrote, preventing malicious manipulation is not the goal. Rather, I solve this issue by not letting people install drivers freely (in production versions).

What prevents drivers from being installed?


Mostly that drivers cannot be installed. RDOS boots from a single binary image that contains the kernel, drivers and applications. It has CRC-sums both for the whole image and per component. That means it's not possible to simply add or change the image (won't boot). It must be built with a special tool. Also, in the production installations, there is a loader that checks exactly which drivers are installed against what is defined on a central site, and rebuilds when there are mismatches.

Octocontrabass wrote:
Schol-R-LEA wrote:
RTOS

I don't see anything on this page that suggests RDOS is designed for real-time operation, but it does mention "segment protection and paging to provide a stable and secure system". You're not confusing Leif Ekblad's RDOS with Data General's RDOS, are you?


No, it is not an RTOS. I had some plans to add an RTOS component by dedicating a single core to it on multicore systems, but that is not finalized. In fact, RDOS would work as a general-purpose OS. It would need a special installation utility, but it certainly would be possible.


Top
 Profile  
 
 Post subject: Re: Memory Segmentation in the x86 platform and elsewhere
PostPosted: Sun Feb 05, 2017 3:33 pm 
Offline
Member
Member

Joined: Wed Oct 01, 2008 1:55 pm
Posts: 2173
The description on my site was recently updated, and should cover a lot of these aspects: www.rdos.net/rdos


Top
 Profile  
 
 Post subject: Re: Memory Segmentation in the x86 platform and elsewhere
PostPosted: Sun Feb 05, 2017 3:46 pm 
Offline
Member
Member
User avatar

Joined: Fri Oct 27, 2006 9:42 am
Posts: 663
Location: Athens, GA
OK, that clarifies things. I didn't realize that (despite what you said about the origin of the name) it had never actually worked as a real-time OS, hard or soft. I knew it was used for embedded systems, and that you had said the name originally meant "Realtime DOS", but I must have drawn the wrong conclusions.

_________________
Rev. First Speaker Schol-R-LEA;2 LCF ELF JAM POEE KoR KCO PPWMTF TGIF
Verbum, a boot sector code example
Lisp programmers tend to seem very odd to outsiders, just like anyone else who has had a religious experience they can't quite explain to others.


Top
 Profile  
 
 Post subject: Re: Memory Segmentation in the x86 platform and elsewhere
PostPosted: Sun Feb 05, 2017 4:06 pm 
Offline
Member
Member

Joined: Wed Oct 01, 2008 1:55 pm
Posts: 2173
Schol-R-LEA wrote:
The fact that portability isn't on the horizon at all also changes a lot of things, but that isn't the real key factor here.


Well, the project started in 1988, when the 386 processor was fairly new, and before there were any good compilers (at least that I had access to) for it. It actually turned out that focusing on the 386 processor was a pretty good design choice, given that the code still runs 28 years after the choice was made. :-)

This origin also explains the use of assembler code. Actually, I initially used assemblers for DOS that produced DOS executables as drivers. The code patching design was a way to link the modules without having any real linker. New driver models were eventually introduced, but code patching was kept as it was a nice way to ensure entry-points. DOS emulation was initially a major goal, but was later replaced with Win32 emulations, before eventually producing the native mode with the OpenWatcom compiler it uses today.

In fact, there are little similarities between the code from the early 90s and today, but it never was redesigned from scratch.


Top
 Profile  
 
 Post subject: Re: Memory Segmentation in the x86 platform and elsewhere
PostPosted: Mon Feb 06, 2017 4:10 am 
Offline
Member
Member
User avatar

Joined: Thu Jul 12, 2012 7:29 am
Posts: 566
Location: Tallinn, Estonia
I wonder how often this segmentation argument with rdos pops up so that you all fall for it each time? Once a year?

_________________
Learn to read.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 25 posts ]  Go to page Previous  1, 2

All times are UTC - 6 hours


Who is online

Users browsing this forum: Yahoo [Bot] and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group