It's possible you have a buggy section of code somewhere that -O3 is optimizing into nothing, but that is triggering #PF when it's there.
This isn't a good example, because it's a useless function, but as an illustration:
Code:
void func(int *x) {
int y;
y = *x;
}
That would cause #PF if x is an invalid pointer, but if you use -O3, the entire function becomes a NOP. The optimizer decides since you didn't use y, there's no reason to set it.
What'd you have to do is 1) track whether your #PFs are happening in the same place(s), and 2) analyze that section of code with and without -O3 to see if you can tell what's going wrong. (if the #PFs are in random places, obviously that makes it harder to find)