OSDev.org

I've been working (a.k. banging my head against a brick wall) on adding x86 user mode to my kernel for the past couple of weeks. I designed the task switching protocol, stack layouts and everything makes sense, but I keep getting inconsistent faults some time after switching to a user task. Switching between kernel tasks works perfectly so I'm baffled as to what could be causing switching to user tasks to be causing faults.

My task switching protocol is as follows:

1. On a timer interrupt, get the next process as per some scheduling policy.
2. If it's a kernel task, then call arch_switch_to_kernel_task (source attached), else call arch_switch_to_user_task
arch_switch_to_kernel_task:
Save current cpu state into process' saved kernel state
Restore next process' CPU state from process' saved kernel state
Return to the address stored at the top of the kernel stack, effectively continuing on from where the task was halted.

arch_switch_to_user_task:
Save current cpu state into process' saved kernel state
Restore next process' CPU state from process' saved kernel state
Return to the address stored at the top of the kernel stack, which eventually finds itself in the irq handler. This pops off the saved user state and enters user mode with an iret

To facilitate this protocol, each task is initialised:

kernel task:
the registers in the saved kernel state are set to 0
stack:
top: exit function address
top - 4: entry function address <-- kernel_state.esp

user task:
the registers in both the kernel state and user state are set to 0
kernel stack:
top: exit function address
top - 4: initial user cpu state
top - 4 - sizeof(arch_cpu_state_t): Unused eax value
top - 8 - sizeof(arch_cpu_state_t): Return address <-- kernel_state.esp
user stack:
top: exit function address <-- user_state.useresp

Sometimes I get a page fault at c0104450 in:


with the CPU state being



Then sometimes it fails at c01095c4 in:


with the CPU state being


This always happens after a seemingly arbitrary number of task switches and it all works perfectly if I make the "cleaner" task a kernel task rather than a user one.

I have a feeling that it could be to do with my setting of ebp/esp, or me missing a part of the switching protocol, but I can't for the life of me figure it out. Does anybody have any idea?

Source for reference:
* Boot file
* Where the switching happens
* Where a process is initialised
* IRQ and ISR handling
* The shceduler
* CPU state definition

Without looking further at your code: In both examples the faulting instruction dereferenced EAX, and EAX did not contain a kernel pointer (I guess all your kernel pointers start with a C, right?). So therefore, either the state was corrupted in memory, or EAX was changed before it could be used. Since your interrupt handling seems solid enough at first glance, I guess it's the former.

Yeah that's right, all kernel symbols and heap are in the upper 1GB of memory.

Your point about memory state corruption seems the most likely and is made more likely by another fault that I couldn't reproduce just before posting this thread, but has happened again now:

It sometimes triggers an invalid opcode fault when a task does a ret and jumps to an address in the heap, which makes sense since the heap doesn't have contain instructions. Below is the output of "qemu -d int,in_asm":



As you can see, a kernel task is interrupted by the timer so it switches to the user task which eventually does a ret into the heap, at which point the CPU sees a stream of arbitrary instructions and faults on the one that is invalid. This screams stack corruption to me but I can't see where I've gone wrong.

If the stack size is 1024 bytes (or 256 4-byte entries), ARCH_INIT_PROCESS_STATE's writing to the stack[256] entry is generally considered as a buffer overflow.

ARCH_INIT_PROCESS_STATE sets esp to entry 251, with the intention of storing 6 (ebp, edi, esi, ebx, entry, exit) entries beginning at 251, but it can only store 5 entries (251 to 255).

Unless the above is deliberately as designed, it could be a potential cause.

That is a good observation and isn't how it was designed, so I will look into that and report back. it could certainly explain the stack corruption.

Edit: I think you may be looking at some old code since I removed the ARCH_INIT_PROCESS_STATE macro and replaced it with the arch_init_process_state function. Your observation with buffer overflow still applies to the new code though.

True. I went where the Github search on the name took me, and that, I now realize, was a commit on 16th Feb 2019. The code search on github seems to be quite limited in power.

Yeah you're right! I don't like it either.

Addressing your observations has gotten rid of the first two issues that I reported (thanks!) but the issue where the code is jumping into the heap is still occuring. Below is an excerpt from GDB I got when breaking at the isr14 handler, does it reveal anything helpful (perhaps the "corrupt stack?" message :p)?

No .

But, I believe the arch_switch_to_*_task functions are causing stack underflow by forcing the cpu to set the esp progressively lower and lower every time an interrupt/exception arrives.

tss.esp0 is set when switching to a task, and cpu reads it back at the time of
interrupt/exception in order to push parameters on it. But the value tss.esp0 arrives from the esp value of the same task when it was going away a few seconds ago. Although the stack unwinds properly because of the return mechanisms, when tss.esp0 is set, the stack unwind (or part of it) is not taken into account.

I hope my words above made some kind of sense about the circular nature of the processing.

This should be easy to verify by progressively increasing the cleaner's stacks from 1kb to say 512kb in chunks of 1kb, and then seeing if the failure takes longer and longer to hit. (Edit: Or, since you have a working debugger setup, you can put a conditional break-print-and-go on the instructions which store
esp into (tss+4) to see if the value settles for each thread, or it runs into
a progression.)

The older output shows that the stacks become misaligned. Does kmalloc, etc. return aligned buffers?

You're precisely correct. I kept doubling both the user stack and kernel stack and it went on longer and longer before hitting the page fault. Well spotted!



kmalloc doesn't return aligned buffers but kmalloc_a does. The only place I remember using kmalloc_a was in my paging code.



I think I understand. Which part of the stack is not taken into account? Is it the parameters passed to arch_switch_to_*_task? If it won't take too much time, I think an ascii graphic/example would be of help

If kmalloc does not return an aligned buffer, then cleaner's user and kernel stacks are likely misaligned, although I can't recall at the moment what restrictions x86 places on esp, but that should be easy to find.


Some diagrams below. To see if the situation described below occurs, a breakpoint can be placed at 'mov %esp, (tss + 4)' instructions inside arch_switch_to_user_task (for now) and see the %esp value being stored. Since the tasks should preserve the stack, and since they do not perform (or can be made to perform nothing more than an idle, tight loop), the esp value must settle to some constant (one constant for each of the tasks).


Now we can see that situation (C) loops back to situation (A), and V continues to decrement.

I'll investigate, experiment and report back.



Ah thank you for that, it makes a lot of sense. Using your suggestion to set a break point does reveal that it never reaches a constant and keeps decrementing. To address this, would you suggest setting TSS.esp0 to some predefined constant or setting it in some other part of the switching tree?

If so, once the %esp goes below the lower address of the kernel stack, you have the proof of the corruption.


The value set in tss.esp0 depends on the reason the control needs to go out of the kernel mode to the user mode or to a different (interrupt/exception) context.

For instance, it is possible for a thread to interweave its activity in both the modes. In such a case, a backtrace might look like below:



Each kmode_func that decided to relinquish control to a umode_func must set tss.esp0 to such a value as to both preserve the currently active kernel mode stack frames, and also allow further calls into the kernel mode (through syscalls/interrupts/excptns) as long as there's stack space.


In jaq, it can help to separate the two concepts:
(1) switching between threads, and
(2) transitioning between the modes/contexts for a single thread.

That separation then moves the responsibility of determining the appropriate
value for tss.esp0 into part (2).

Part (1) does not need to particularly worry about tss.esp0, except copy pasting from the incoming thread's state to cpu.tss.esp0, as a part of the context switch.

In jaq, if we always return to usermode after completely unwinding the kernel stack, then setting tss.esp0 to the base of the kernel stack looks sufficient to me. That means committing to the fact that as long as the task is running in the usermode, its kernel stack does not contain any active frames (it may continue to hold state, etc. which lie beyond the base as we define it).

There might be other ways to handle this situation. For instance, linux has a per-cpu entry stack to which tss.esp0/sysenter_esp points, and the interrupt/exception handling switches back and forth between the entry stack and the task's kernel stack.

I can't believe I didn't think of that! Setting tss.esp to the base of the kernel stack worked perfectly. Thank you for your help. I learnt a thing or two from this.