OSDev.org

The Place to Start for Operating System Developers
It is currently Mon Mar 25, 2019 12:00 pm

All times are UTC - 6 hours




Post new topic Reply to topic  [ 7 posts ] 
Author Message
 Post subject: Kernel preemption during software interrupts
PostPosted: Fri Mar 01, 2019 4:32 am 
Offline
Member
Member
User avatar

Joined: Mon Mar 05, 2012 11:23 am
Posts: 580
Location: Germany
Hey,

I can't quite understand how a preemptible kernel is supposed to work, meaning that one task can be interrupted while doing a syscall. My kernel supports user and kernel privilege threads. Each user level thread has a kernel stack and a userspace stack. Each kernel level thread only has one stack where the registers are just pushed on-top when an interrupt occurs. This means a system call from userspace currently looks like this:

Code:
Userspace task running
-> System call is fired
-> CPU pushes SS, ESP, EFLAGS, CS, EIP (to kernel stack of user task)
-> Enters interrupt handler
--> CLI so we don't get interrupted
--> Push general registers (to kernel stack of user task)
--> Store register state in task structure
---> System call is processed
--> Restore register state from task structure
--> Interrupt handler pops general registers (off the kernel stack of user task)
-> IRET pops EIP, CS, EFLAGS, ESP SS
-> Returns into the userspace task


What I can't wrap my head around, how does it work if this is interrupted again, say by a timer interrupt? It would be a Ring 0 -> Ring 0 switch so the stack would not be switched.. where would you store the registers now? On top of the stack again? Like...

Code:
Userspace task running
-> System call is fired
-> CPU pushes SS, ESP, EFLAGS, CS, EIP (to kernel stack of user task)
-> Enters interrupt handler
--> CLI so we don't get interrupted
--> Push general registers (to kernel stack of user task)
--> Store register state in task structure
--> Now STI again?
---> System call is processed
---> We now get interrupted by timer interrupt
---> ???
Any clarifications greatly appreciated!

Best greets

_________________
Ghost OS - GitHub


Top
 Profile  
 
 Post subject: Re: Kernel preemption during software interrupts
PostPosted: Fri Mar 01, 2019 5:18 am 
Offline
Member
Member

Joined: Tue Mar 04, 2014 5:27 am
Posts: 941
max wrote:
... how does it work if this is interrupted again, say by a timer interrupt? It would be a Ring 0 -> Ring 0 switch so the stack would not be switched.. where would you store the registers now? On top of the stack again?


Yes, until you run out of stack space or interrupt priorities.


Top
 Profile  
 
 Post subject: Re: Kernel preemption during software interrupts
PostPosted: Fri Mar 01, 2019 7:08 am 
Offline
Member
Member

Joined: Fri Jun 28, 2013 1:48 am
Posts: 28
You should have a dedicated stack for interrupts. If your OS support multi-core, then one stack for each CPU. When handling interrupts, first switch to interrupt stack, then switch back.

Since interrupt stack does not belong to any task (thread), your can achieve task preemption. Find a new task to run in the ISR, then retrieve its rip saved during last interrupt, and switch to that stack.

For exceptions and syscall, they don't cause task preemtpion. So no need to switch stack (only switch stack when CPL=3)

_________________
Reinventing the Wheel, code: https://github.com/songziming/wheel


Top
 Profile  
 
 Post subject: Re: Kernel preemption during software interrupts
PostPosted: Fri Mar 01, 2019 10:01 am 
Offline
Member
Member
User avatar

Joined: Wed Oct 27, 2004 11:00 pm
Posts: 846
Location: WA
task preemption requires absolutely nothing on your part -- it just works

there is really no problems here, it is a lot simpler than you are thinking:

Code:
user stack:
systemCall -> switch to kernel stack

kernel stack:
<-user return CS:r/eIP, eflags, etc.
<-user preserved registers, segment registers, etc.

<-kernel using stack

--Interrupt occurs--
<-return information (r/eIP, etc.)
<-interrupt preserved information

<-interrupt usage

--interrupt decides to task switch--

--IMPORTANT: be sure to satisfy the hardware (whatever required to prevent repeated interrupts and allow the hardware to proceed) and send EIO before moving on to the next point

<-interrupt CALLs task switch program (return state information stored on stack)
STACK SWITCH : task switching code switches to another task, switching stacks in the process

<-task switch code returns to whatever called in [i]in the new process[/i], old process is suspended

here we will assume it was called by another ISR

<-interrupt completes, and local variables are removed from stack (note: this is the interrupt from the new task, not the old task)

<-ISR restores preserved information
<-IRET returns to kernel code which was interrupted by the interrupt

<-kernel syscall finishes processing, and removes locals from stack

<-kernel syscall restores preserved registers
<-kernel syscall returns to user code

:at this point the kernel stack is empty (or rather, at the same point it was before the syscall)

_________________
## ---- ----- ------ Intel Manuals
OSdev wiki


Top
 Profile  
 
 Post subject: Re: Kernel preemption during software interrupts
PostPosted: Fri Mar 01, 2019 12:41 pm 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 169
max wrote:
What I can't wrap my head around, how does it work if this is interrupted again, say by a timer interrupt? It would be a Ring 0 -> Ring 0 switch so the stack would not be switched.. where would you store the registers now? On top of the stack again? Like...


Yes, it is all pushed on top of the stack again. One difficulty you may encounter if you enable interrupts too early is that some of the work of switching to kernel space might not be done when the registers indicate as such. For instance on x86_64, you may need to run the SWAPGS instruction, but you only need to run it on entry from userspace. Which you can typically identify by the privilege level of the pushed CS in the interrupt frame. But if you enable interrupts too early, and then get interrupted between entering the kernel and running swapgs, in the interrupt, you will see that kernel mode was interrupted because the CS will be the kernel mode CS. But the GS is not swapped yet.

And in 32-bit mode you can have the same fun switching out the data segment registers.

For this reason, and because running out of kernel stack is generally a bad thing, you should always use interrupt gates in your IDT (or task gates where the corresponding TSS's EFLAGS has IF set to 0). Then use STI judiciously (i.e. only with plenty of stack left, and only after finishing the context switch).


Top
 Profile  
 
 Post subject: Re: Kernel preemption during software interrupts
PostPosted: Sat Mar 02, 2019 12:22 pm 
Offline
Member
Member
User avatar

Joined: Mon Mar 05, 2012 11:23 am
Posts: 580
Location: Germany
Thanks for your replies, it was pretty helpful to understand it better. I'll have to adjust my mutex implementation though so it will also disable interrupts when one or more locks are taken on a processor, so it can't get interrupted and deadlock the processor :P

_________________
Ghost OS - GitHub


Top
 Profile  
 
 Post subject: Re: Kernel preemption during software interrupts
PostPosted: Sat Mar 02, 2019 12:50 pm 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 169
Easiest implementation of that: Your lock function returns the previous EFLAGS register before disabling interrupts. Your unlock function then restores that register. That way, interrupts can be enabled when the outermost lock is taken. Of course, that requires the first lock to be taken to also be the last lock to be released, but that is generally a reasonable requirement.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 7 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group