OSDev.org

Hello,
In my kernel, at one point, I would get occasional #PFs and hangs. I compiled with -O3, and it works now. Why would -O3 fix this problem?
Thanks,
nexos

Are you on x86_64, and if so, have you setup the compiler to not use the red zone? https://wiki.osdev.org/Libgcc_without_red_zone

It's possible you have a buggy section of code somewhere that -O3 is optimizing into nothing, but that is triggering #PF when it's there.

This isn't a good example, because it's a useless function, but as an illustration:


That would cause #PF if x is an invalid pointer, but if you use -O3, the entire function becomes a NOP. The optimizer decides since you didn't use y, there's no reason to set it.

What'd you have to do is 1) track whether your #PFs are happening in the same place(s), and 2) analyze that section of code with and without -O3 to see if you can tell what's going wrong. (if the #PFs are in random places, obviously that makes it harder to find)

What is strange is that it only does it occasionaly. I don't think libgcc was compiled with -mno-red-zone. The code probably got interrupted and it corrupted the stack.

I'm certain you have an UB somewhere. Things like "it works with -O0 but not with -O3 (or vice versa)" almost all the time caused by an UB somewhere. The optimizer generating code that occasionaly doesn't work also a good indicator for an UB.

Cheers,
bzt

I meant it does it occasionaly with -O0, amd never with -O3. I probably had to do with the red zone. I realized today I wasn't passing -mno-red-zone to the compiler .