OSDev.org

The Place to Start for Operating System Developers
It is currently Thu Mar 28, 2024 9:20 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 13 posts ] 
Author Message
 Post subject: System call cost
PostPosted: Tue Oct 17, 2017 5:24 am 
Offline

Joined: Sun Apr 30, 2017 10:14 am
Posts: 18
Hello, I don't know if this is the correct place for this question, so feel free to move it elsewhere.

My question is related to the cost of system calls. I've always heard this is an expensive operation, but I don't really understand why. Shouldn't the cost be similar or cheaper than a common exception?

Thank you in advance


Top
 Profile  
 
 Post subject: Re: System call cost
PostPosted: Tue Oct 17, 2017 10:25 am 
Offline
Member
Member

Joined: Thu May 17, 2007 1:27 pm
Posts: 999
The exact execution times or cycles that system calls need obviously depends on the processor in question (and on the mechanism used: syscall vs. sysenter vs. int vs. call gate). However, generally system calls are quite slow. System calls take 100s or 1000s of cycles and not 10s. This cost comes from the CPU's need to reload a bunch of registers, invalidate pipelines and speculative execution state and perform privilege checks.

What is a "common exception"? If you're talking about CPU exceptions, then yes, system calls are cheaper than exceptions. If you're talking about exceptions in a high level language like C++, then that depends on how exceptions are implemented there; exceptions are usually very slow themselves.

_________________
managarm: Microkernel-based OS capable of running a Wayland desktop (Discord: https://discord.gg/7WB6Ur3). My OS-dev projects: [mlibc: Portable C library for managarm, qword, Linux, Sigma, ...] [LAI: AML interpreter] [xbstrap: Build system for OS distributions].


Top
 Profile  
 
 Post subject: Re: System call cost
PostPosted: Tue Oct 17, 2017 10:47 am 
Offline
Member
Member
User avatar

Joined: Sat Mar 31, 2012 3:07 am
Posts: 4591
Location: Chichester, UK
The difference being that exceptions are (relatively) exceptional, so the overhead isn't a huge problem, whereas system calls are (relatively) common.

Use sysenter (or syscall).


Top
 Profile  
 
 Post subject: Re: System call cost
PostPosted: Tue Oct 17, 2017 8:45 pm 
Offline
Member
Member
User avatar

Joined: Sat Jan 15, 2005 12:00 am
Posts: 8561
Location: At his keyboard!
Hi,

josecm wrote:
Hello, I don't know if this is the correct place for this question, so feel free to move it elsewhere.

My question is related to the cost of system calls. I've always heard this is an expensive operation, but I don't really understand why. Shouldn't the cost be similar or cheaper than a common exception?


A new car is cheap (compared to the cost of a new house) and a new car is expensive (compared to the cost of a sandwich). System calls are typically compared to the cost of a normal function call, and are a lot more expensive than a normal function call. The cost of an interrupt (including CPU's exceptions, IRQs, IPIs, etc) is typically a little higher than the cost of a system call.

The additional cost (for system calls and interrupts/exceptions/IRQs) can be split into 4 categories:
  • The effect on the CPU's pipeline when it happens (e.g. CPU has to either discard "already in progress" instructions or wait until they retire)
  • The cost of changing privilege levels (e.g. from CPL=3 to CPL=0 and back to CPL=0, including changing stacks, changing segment registers, etc)
  • Bloat because your compiler isn't great (e.g. expects a specific ABI and therefore can't avoid saving/restoring various registers, etc).
  • Bloat because you were lazy (e.g. an additional jump table and mispredicted branch that could've been avoided, but exists because you just did a "common exception handler" for your own convenience).

Note that avoidable bloat is typically at least as large as the actual (minimal) cost, and something like SYSCALL (that can be done in "several tens of cycles") can easily become "several hundred cycles" because of this. ;)


Cheers,

Brendan

_________________
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.


Top
 Profile  
 
 Post subject: Re: System call cost
PostPosted: Sat Oct 21, 2017 3:13 pm 
Offline
Member
Member
User avatar

Joined: Fri Jan 27, 2017 12:15 pm
Posts: 149
Location: Belgium
I think this is a good topic to make a joke on...

_________________
AQUA OS: https://obiwac.wordpress.com/aqua-os/


Top
 Profile  
 
 Post subject: Re: System call cost
PostPosted: Fri Nov 17, 2017 10:50 am 
Offline

Joined: Sun Apr 30, 2017 10:14 am
Posts: 18
Thank you for all your answers, especially Brendan who detailed exactly where this costs come from.

To obiwac,
obiwac wrote:
I think this is a good topic to make a joke on...

Why? Just trying to learn...


Top
 Profile  
 
 Post subject: Re: System call cost
PostPosted: Sat Nov 18, 2017 8:00 am 
Offline
Member
Member

Joined: Mon Jul 05, 2010 4:15 pm
Posts: 595
josecm wrote:
Hello, I don't know if this is the correct place for this question, so feel free to move it elsewhere.

My question is related to the cost of system calls. I've always heard this is an expensive operation, but I don't really understand why. Shouldn't the cost be similar or cheaper than a common exception?

Thank you in advance


Actually the answer is, system calls are always too slow. Always assume the system calls are slow. There is a lot of bulk job that has to be done before jumping to the actual function that performs the operation, like checking parameters, finding the system call function, check errors, check if reschedule is necessary and so on. This takes time compared to a function call in user space.

I've seen programs that reads the timer counter each 100us resulting a system call for obtaining the timer counter. Despite the system call is one of the faster, it is not fast enough obviously and the the program was just stealing cycles. I had to explain to the engineer that checking the time that often slows down the system and eats power. No matter how fast system calls are, assume they are slow. Avoid them if you can.


Top
 Profile  
 
 Post subject: Re: System call cost
PostPosted: Sat Nov 18, 2017 7:03 pm 
Offline

Joined: Sun Apr 30, 2017 10:14 am
Posts: 18
OSwhatever wrote:
josecm wrote:
Hello, I don't know if this is the correct place for this question, so feel free to move it elsewhere.

My question is related to the cost of system calls. I've always heard this is an expensive operation, but I don't really understand why. Shouldn't the cost be similar or cheaper than a common exception?

Thank you in advance


Actually the answer is, system calls are always too slow. Always assume the system calls are slow. There is a lot of bulk job that has to be done before jumping to the actual function that performs the operation, like checking parameters, finding the system call function, check errors, check if reschedule is necessary and so on. This takes time compared to a function call in user space.

I've seen programs that reads the timer counter each 100us resulting a system call for obtaining the timer counter. Despite the system call is one of the faster, it is not fast enough obviously and the the program was just stealing cycles. I had to explain to the engineer that checking the time that often slows down the system and eats power. No matter how fast system calls are, assume they are slow. Avoid them if you can.


Thank you for your answer. I found your explanation very informative. However, I was actually asking the question from a hardware perspective - system call, as in the instruction itself that generates the synchronous exception.


Top
 Profile  
 
 Post subject: Re: System call cost
PostPosted: Sun Nov 19, 2017 4:08 am 
Offline
Member
Member

Joined: Thu May 17, 2007 1:27 pm
Posts: 999
OSwhatever wrote:
I've seen programs that reads the timer counter each 100us resulting a system call for obtaining the timer counter. Despite the system call is one of the faster, it is not fast enough obviously and the the program was just stealing cycles. I had to explain to the engineer that checking the time that often slows down the system and eats power. No matter how fast system calls are, assume they are slow. Avoid them if you can.

Your general remarks about syscalls are fine; however, for the timer, that is heavily OS-dependent. Some OS (e.g. Linux, click me!) implement high-precision timers as a combination of rdtsc and a mapping that is shared with the kernel. You can call that every few us without a noticeable performance difference.

_________________
managarm: Microkernel-based OS capable of running a Wayland desktop (Discord: https://discord.gg/7WB6Ur3). My OS-dev projects: [mlibc: Portable C library for managarm, qword, Linux, Sigma, ...] [LAI: AML interpreter] [xbstrap: Build system for OS distributions].


Top
 Profile  
 
 Post subject: Re: System call cost
PostPosted: Sun Nov 19, 2017 4:38 am 
Offline
Member
Member

Joined: Mon Jul 05, 2010 4:15 pm
Posts: 595
Korona wrote:
Your general remarks about syscalls are fine; however, for the timer, that is heavily OS-dependent. Some OS (e.g. Linux, click me!) implement high-precision timers as a combination of rdtsc and a mapping that is shared with the kernel. You can call that every few us without a noticeable performance difference.


I have for a long time wanted that the HW designers of timer blocks realized this. They could put a free running counter in a separate 4kB block so that you can map it as read only into user space and just read the counter when you want. Also the user space counter could also have a dedicated divider in order to control the accuracy for the user space processes.


Top
 Profile  
 
 Post subject: Re: System call cost
PostPosted: Sun Nov 19, 2017 8:48 am 
Offline
Member
Member
User avatar

Joined: Fri Feb 17, 2017 4:01 pm
Posts: 640
Location: Ukraine, Bachmut
But reading the counter is not a problem by itself, polling or interrupt storm caused by such high frequency requirements are. How does your program know when "the timer event is fired"? It either gets notified through the interrupt or even worse - is polling the counter in a busy loop. Do you think that busy loop polling counter is better? Interrupts are better for sure, but still it is unavoidable to have a huge overhead with high frequency timers.

_________________
ANT - NT-like OS for x64 and arm64.
efify - UEFI for a couple of boards (mips and arm). suspended due to lost of all the target park boards (russians destroyed our town).


Top
 Profile  
 
 Post subject: Re: System call cost
PostPosted: Sun Nov 19, 2017 7:34 pm 
Offline
Member
Member
User avatar

Joined: Sat Jan 15, 2005 12:00 am
Posts: 8561
Location: At his keyboard!
Hi,

Korona wrote:
OSwhatever wrote:
I've seen programs that reads the timer counter each 100us resulting a system call for obtaining the timer counter. Despite the system call is one of the faster, it is not fast enough obviously and the the program was just stealing cycles. I had to explain to the engineer that checking the time that often slows down the system and eats power. No matter how fast system calls are, assume they are slow. Avoid them if you can.

Your general remarks about syscalls are fine; however, for the timer, that is heavily OS-dependent. Some OS (e.g. Linux, click me!) implement high-precision timers as a combination of rdtsc and a mapping that is shared with the kernel. You can call that every few us without a noticeable performance difference.


Ironically; this creates the opposite problem - the precision you get from TSC and how quickly you can get the time is the basis for lots of different security attacks involving timing side-channels; including using cache timing to make brute force attacks on crypto libraries easier and to extract information (e.g. key-presses) from a completely "isolated" virtual machine.

Fortunately Intel knew of the security implications from the start, and added a "time stamp disable at CPL=3" flag in CR4 and have always mentioned it in their manual (something like "a secure OS should disable TSC at CPL=3 using..."). Unfortunately most OSs have cared more about performance than security and didn't follow Intel's advice. ;)


Cheers,

Brendan

_________________
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.


Top
 Profile  
 
 Post subject: Re: System call cost
PostPosted: Thu Jan 04, 2018 7:37 pm 
Offline
Member
Member
User avatar

Joined: Fri Jan 27, 2017 12:15 pm
Posts: 149
Location: Belgium
josecm wrote:
To obiwac,
obiwac wrote:
I think this is a good topic to make a joke on...

Why? Just trying to learn...


Nonononononono sorry I wasn't referring to your question but on the title :?
Which was a terrible joke in itself but anyways sorry.

_________________
AQUA OS: https://obiwac.wordpress.com/aqua-os/


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 13 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 23 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group