[SOLVED] Getting into ring3 on x86_64

bluemoon · **Posted:** Thu May 03, 2012 8:51 am

I have a piece of code on test application:

Code:

_start:
    ud2
    dd      0xdeadbeef
    jmp     _start

    xor     rbp, rbp
    mov     edi, 0
    mov     rsi, env
    call    main

    xor     rdi, rdi
    call    _exit
    ret

If I execute such code directly in ring0, #UD as expected:

Code:

PSTUB   : FFFFFFFF:8012D740 Executing [/initrd/testapp]
PSTUB   : FFFFFFFF:8012D740 Program entry: 00000000:001000D0 -> F8EBDEAD:BEEF0B0F
INT06 : #UD Invalid Opcode Exception. RIP: 00000000:001000D0

However, if I do switch to ring3, it seems do not execute at all and only repeating process switch happens.

Code:

SCHED : Current Process: FFFFFFFF:8012E740 Next Process: FFFFFFFF:8012F150, CR3: 00000000:01FEE000 Remain: 10
SCHED : Current Process: FFFFFFFF:8012F150 Next Process: FFFFFFFF:8012E740, CR3: 00000000:01FF1000 Remain: 10
SCHED : Current Process: FFFFFFFF:8012E740 Next Process: FFFFFFFF:8012F150, CR3: 00000000:01FEE000 Remain: 10
SCHED : Current Process: FFFFFFFF:8012F150 Next Process: FFFFFFFF:8012FB60, CR3: 00000000:00000000 Remain: 10
KMAIN : kthread(pid=3): echo #2
SCHED : Current Process: FFFFFFFF:8012FB60 Next Process: FFFFFFFF:8012E740, CR3: 00000000:01FF1000 Remain: 10
SCHED : Current Process: FFFFFFFF:8012E740 Next Process: FFFFFFFF:8012F150, CR3: 00000000:01FEE000 Remain: 10
SCHED : Current Process: FFFFFFFF:8012F150 Next Process: FFFFFFFF:8012E740, CR3: 00000000:01FF1000 Remain: 10

8012E740 and 8012F150 are two process of testapp, and nothing happen even the code to be executed is ud2
however, other kthread seems not disturbed and run normally.

my code to get ring3:

Code:

; void enter_ring3  ( unsigned long ring3_ip, unsigned long ring3_sp );
enter_ring3:
    ; jmp     rdi
    mov     ecx, SEG_DATA64_3 +3
    ;mov     ds, cx
    ;mov     es, cx
    push    rcx
    push    rsi
    push    0x0202          ; rflags
    push    SEG_CODE64_3 +3
    push    rdi
    iretq

What do I miss?

ps. cpu exception (INT00-13) has IST=1, and PIC timer has IST=2.
I have kernel stack per each thread, and swap rsp0 on TSS upon reschedule.

EDIT:solved

bluemoon · **Posted:** Thu May 03, 2012 10:32 am

I also tried sysret, it also "nothing happen", no cpu exception, no sign of execution...

Code:

; void enter_ring3  ( unsigned long ring3_ip, unsigned long ring3_sp );
enter_ring3:
    ; jmp     rdi
    mov     rcx, rdi
    mov     rsp, rsi
    mov     r11, 0x0202
    sysret

I must have missed something obvious...

xenos · **Posted:** Thu May 03, 2012 10:45 am

Have you tried single-stepping through your switch to ring 3 in bochs debugger or similar?

bluemoon · **Posted:** Thu May 03, 2012 11:15 am

Not yet, I used to debug with qemu-gdb but it has problem on 64-bit target (the famous g packet too long), so I lost debugging facilities.
Now I'm building bochs on mac and see what happen.

iansjack · **Posted:** Thu May 03, 2012 12:10 pm

Just as a matter of interest, the gdb business is easily solved. Start the program, interrupt it (once it has transferred to 64-bit mode) and "load" a 64-bit executable.

But this probably won't solve your current problem as you won't be able to stop at the correct point to single-step. With my OS I wait until I get the command prompt for my shell, interrupt that, and then I can debug any further code.

May I recommend SimNow from AMD? This is an emulator rather than a simulator, so is a little slow, but is great for single-stepping assembler code in a situation like this. You can set a breakpoint in the debugger before booting your kernel and then single-step from there on.

I can't think what your problem is. A "sysret" should do the trick (except perhaps it should be "sysretq"?).

bluemoon · **Posted:** Thu May 03, 2012 12:36 pm

The ring3 process seems to come to a hang state. I dump the "Resume IP" from the scheduler as it read:

Code:

SCHED : Current Process: FFFFFFFF:8012D740 Next Process: FFFFFFFF:8012D740, EIP: 00000000:001000D0 Remain: 10
SCHED : Current Process: FFFFFFFF:8012D740 Next Process: FFFFFFFF:8012D740, EIP: 00000000:001000D0 Remain: 10
SCHED : Current Process: FFFFFFFF:8012D740 Next Process: FFFFFFFF:8012D740, EIP: 00000000:001000D0 Remain: 10
SCHED : Current Process: FFFFFFFF:8012D740 Next Process: FFFFFFFF:8012D740, EIP: 00000000:001000D0 Remain: 10

And for qemu-gdb with 64bit, I tried set architecture i386:x86-64 with no luck.

Quote:

I can't think what your problem is. A "sysret" should do the trick (except perhaps it should be "sysretq"?).

I tried sysret and it successfully returned to compatibile mode with correct descriptor (I can check this with qemu-gdb)
I also tried REX.w sysret, which returns to 64-bit code, and it seems freeze there.

bluemoon · **Posted:** Thu May 03, 2012 1:12 pm

OK Finally I catch the bug.

my PF handler unconditionally set entry to page +3, which obviously can't access by ring3.
somehow the CPU fault at 0x001000D0 but cannot resolve the handler, and i don't know what happen anyway.

Now I changed to

Code:

prot = pt[MMU_PT_INDEX(addr)] & MMU_PROT_MASK;
pt[MMU_PT_INDEX(addr)] = page | prot | MMU_PROT_PRESENT;

and the ring3 code successfully trigger #UD.

So I learnt that a lot of seemingly un-related and weird things can happen with bug in MMU code #-o

OSDev.org

[SOLVED] Getting into ring3 on x86_64

Who is online