(Fixed) Array Triple Fault

[simeonz]

I think you should first narrow the location of the problem in the source and then seek the cause. You seem to be probing with brute-force for indication of the root cause right off the bat.

Think what actually is the problem you are trying to eliminate. The other forum members assisted you and you found out that the page directory pointer in cr3 is null. Why is that? Which code is responsible to set it? You should know that.

Assuming that you think the code that is responsible should work as intended and cr3 should be properly set, you must use gdb to trace the respective code and check what's wrong. If that is insufficient and there are other unknown pieces of code involved, use any instruments that make sense. Use access breakpoints, or set variables (preferably "volatile") in various places in the code just to track executed branches, then read them with gdb, comment code paths to reduce the scope of the bug search, print diagnostic messages. You must locate the line or couple of lines that violate your intents. Then, assuming you don't find a bug in them, you can disassemble them, and if the assembly does not correspond to the language specification, you can infer that the compiler is buggy. But this is unlikely and usually you will find your own mistake.

[LtG]

Here are some things I discovered.

-O2 enabled, PMM works perfectly, returns page aligned addresses, all good.
-O2 disabled, PMM broken as hell, only returns 0x0.

I am so mad at the compiler. Why does it keep breaking my code! Then that means, no PMM -> no VMM. Thus it is probably not VMM's fault.

Edit:
Manged to track down the issue.
You would never have guessed...
GRUB only reports 16 KILOBYTES of memory available with -02 disabled. 16 freaking kilobytes, and then I wonder why my memory managers don't work.
Now I need to find why is that so.

The problem isn't the compiler, it's you. The compiler doesn't break your code, the code was broken to begin with. Optimizations affect the code produced and so with broken code it may sometimes seemingly break or fix things, but the cause is that the underlying code (your code) is fundamentally broken.

Essentially your code may never rely on anything outside the code, so even if you set some pointer to point to some memory address where there's some data left by GRUB/multiboot/BIOS/etc the compiler might _NOT_ read it and might do anything it wants. Your C code does not exist in a real physical machine not even in a real virtual machine, it exists in an abstract machine. When you need to break the abstraction you need to inform the compiler, for instance use the keyword volatile to instruct the compiler that it must actually read from memory and not assume things.

For instance:

Code: Select all

int* ptr = 0x1234;
*ptr = 555;
int b = 5 + *ptr;

Can be translated to:

Code: Select all

int b = 560;

Of course if there's not other code besides that, then the compiler can remove that "int b = 560;" line as well because it has no side effects and therefore is irrelevant.

Let's say you had memory mapped IO, where 0x1234 is actually some device and you wanted to write 555 to it so the device would do something, that write never happens, however as far as your code is concerned, internally it still knows the values 555 and 560 and will use them in all locations where they are needed, but outside of your code, the 555 may never be written to 0x1234.

Why don't you just debug the issue, once you know where and how it happens it should be relatively trivial to figure out why it happens. If it's not, you can post the relevant asm and C code here and we can help you figure it out.

At this point it seems that your O2 vs !O2 have differing behavior with trying to figure out how much RAM there is.. Though why are you even detecting the _amount_ of RAM, instead of getting a memory map? How do you attempt to get the amount of memory?

[Octacone]

I think you should first narrow the location of the problem in the source and then seek the cause. You seem to be probing with brute-force for indication of the root cause right off the bat.

Think what actually is the problem you are trying to eliminate. The other forum members assisted you and you found out that the page directory pointer in cr3 is null. Why is that? Which code is responsible to set it? You should know that.

Assuming that you think the code that is responsible should work as intended and cr3 should be properly set, you must use gdb to trace the respective code and check what's wrong. If that is insufficient and there are other unknown pieces of code involved, use any instruments that make sense. Use access breakpoints, or set variables (preferably "volatile") in various places in the code just to track executed branches, then read them with gdb, comment code paths to reduce the scope of the bug search, print diagnostic messages. You must locate the line or couple of lines that violate your intents. Then, assuming you don't find a bug in them, you can disassemble them, and if the assembly does not correspond to the language specification, you can infer that the compiler is buggy. But this is unlikely and usually you will find your own mistake.

I actually know what is going on. Looks like you didn't read my update. So the PMM should provide an address for the VMM to use but it is only providing 0x0, why? Because GRUB returns only 16 KB of total RAM, which crashes the entire OS memory system. So the question is why does it happen. I can set CR3 to any value I want, that is not questionable.

[Octacone]

Here are some things I discovered.

-O2 enabled, PMM works perfectly, returns page aligned addresses, all good.
-O2 disabled, PMM broken as hell, only returns 0x0.

I am so mad at the compiler. Why does it keep breaking my code! Then that means, no PMM -> no VMM. Thus it is probably not VMM's fault.

Edit:
Manged to track down the issue.
You would never have guessed...
GRUB only reports 16 KILOBYTES of memory available with -02 disabled. 16 freaking kilobytes, and then I wonder why my memory managers don't work.
Now I need to find why is that so.

The problem isn't the compiler, it's you. The compiler doesn't break your code, the code was broken to begin with. Optimizations affect the code produced and so with broken code it may sometimes seemingly break or fix things, but the cause is that the underlying code (your code) is fundamentally broken.

Essentially your code may never rely on anything outside the code, so even if you set some pointer to point to some memory address where there's some data left by GRUB/multiboot/BIOS/etc the compiler might _NOT_ read it and might do anything it wants. Your C code does not exist in a real physical machine not even in a real virtual machine, it exists in an abstract machine. When you need to break the abstraction you need to inform the compiler, for instance use the keyword volatile to instruct the compiler that it must actually read from memory and not assume things.

For instance:

Code: Select all

int* ptr = 0x1234;
*ptr = 555;
int b = 5 + *ptr;

Can be translated to:

Code: Select all

int b = 560;

Of course if there's not other code besides that, then the compiler can remove that "int b = 560;" line as well because it has no side effects and therefore is irrelevant.

Let's say you had memory mapped IO, where 0x1234 is actually some device and you wanted to write 555 to it so the device would do something, that write never happens, however as far as your code is concerned, internally it still knows the values 555 and 560 and will use them in all locations where they are needed, but outside of your code, the 555 may never be written to 0x1234.

Why don't you just debug the issue, once you know where and how it happens it should be relatively trivial to figure out why it happens. If it's not, you can post the relevant asm and C code here and we can help you figure it out.

At this point it seems that your O2 vs !O2 have differing behavior with trying to figure out how much RAM there is.. Though why are you even detecting the _amount_ of RAM, instead of getting a memory map? How do you attempt to get the amount of memory?

So the compiler can do anything it wants without caring if it will work. That is just awesome...
Basically total_memory = memory_lower (GRUB) + memory_upper (GRUB), can't be more simple than that. The problem is that it only returns 16 KB instead of "n" GB.
Forget about the memory map, it is implemented, but used for some other mapping stuff. Don't focus on that right now, it is irrelevant.

[LtG]

So the compiler can do anything it wants without caring if it will work. That is just awesome...
Basically total_memory = memory_lower (GRUB) + memory_upper (GRUB), can't be more simple than that. The problem is that it only returns 16 KB instead of "n" GB.
Forget about the memory map, it is implemented, but used for some other mapping stuff. Don't focus on that right now, it is irrelevant.

I was "focusing" on the memory map because doing it wrong (lower+upper) is more difficult than doing it right (memory map), so wasting time fixing the wrong way, just so you can later drop it and do it the right way is just a waste of time.

If you adamantly want to do it the wrong way, sure, I can still try to help you.

Now, assuming there's no difference in GRUB between the two versions (and the VM's, etc), then we can safely say that GRUB provides you with the correct values, you are not fetching them correctly.

The compiler can do anything it wants within the confines of the language, without caring if it works. _YOU_ need to care enough to write _CORRECT_ code, not random stuff and expect the compiler to fix it. Your code is broken, it is wrong, that's the cause of your problems.

Can you show the code how you get these (GRUB provided memory) values in C? Since you now know they are wrong, you could also have already used gdb to check why you are getting invalid values, and if you don't understand it yourself, you could have posted the relevant C and disasm here after you used gdb to check what the reason is.

[Octacone]

So the compiler can do anything it wants without caring if it will work. That is just awesome...
Basically total_memory = memory_lower (GRUB) + memory_upper (GRUB), can't be more simple than that. The problem is that it only returns 16 KB instead of "n" GB.
Forget about the memory map, it is implemented, but used for some other mapping stuff. Don't focus on that right now, it is irrelevant.

I was "focusing" on the memory map because doing it wrong (lower+upper) is more difficult than doing it right (memory map), so wasting time fixing the wrong way, just so you can later drop it and do it the right way is just a waste of time.

If you adamantly want to do it the wrong way, sure, I can still try to help you.

Now, assuming there's no difference in GRUB between the two versions (and the VM's, etc), then we can safely say that GRUB provides you with the correct values, you are not fetching them correctly.

The compiler can do anything it wants within the confines of the language, without caring if it works. _YOU_ need to care enough to write _CORRECT_ code, not random stuff and expect the compiler to fix it. Your code is broken, it is wrong, that's the cause of your problems.

Can you show the code how you get these (GRUB provided memory) values in C? Since you now know they are wrong, you could also have already used gdb to check why you are getting invalid values, and if you don't understand it yourself, you could have posted the relevant C and disasm here after you used gdb to check what the reason is.

Actually it is C++. I don't know how can that be harder. So you think I should do something like total_memory = last memory map address? It is not "random stuff", I hope you are not talking about my code...
I'll post everything relevant, first I need to check it once more with GDB.

Edit: oops that won't work, can you explain your memory map idea?
Edit 2: forget about that idea until we fix the main issue, memory map is also "destroyed"...

[Octacone]

None of the GRUB flags are being set.
Here is some relevant code:

Bootloader.asm:

Code: Select all

bits 32
section .multiboot 
multiboot_align			equ	1 << 0
multiboot_memory_info	equ	1 << 1
multiboot_flags			equ	multiboot_align | multiboot_memory_info
multiboot_header_magic  equ	0x1BADB002
multiboot_checksum		equ	- (multiboot_header_magic + multiboot_flags)

align 4
dd multiboot_header_magic 
dd multiboot_flags 
dd multiboot_checksum

...
Bootloader_Main:
      ...
      push ebx
      ...

Kernel.cpp

Code: Select all

extern "C" void Kernel_Main(uint32_t multiboot_eax_magic, multiboot_info_t* multiboot_info)
{
      Memory_Core.Initialize(multiboot_info); //this part is less relevant, sort of
      //this matters:
      //no matter where/how I do this, it returns wrong values:
      TUI.Put_String("\nMemory upper: ", 0x07);
      TUI.Put_Hex(multiboot_info->memory_upper, 0x07); -> instead of 2 GB (aka 2 097 152 KB) it displays 16 KB only.
      Also multiboot_info->flags --->>> all zeros no bytes being set
}

So little code, so much troubles.

Edit:
Kernel_Main (C++) dissasembly:

[LtG]

Kernel_Main (C++) dissasembly:

Is that one disassembly or two? What's the difference between left and right? You did notice that only one of them calls "Call_Constructors", right? Is Memory_Core a global object? How did you initialize it?

You have read the osdev wiki's relating to C++ and the stuff you need to take care of, right?

[Ch4ozz]

Show us ALL of your code of the entrypoint in the kernel asm file.
Im pretty sure your pushing it in the wrong order

[Octacone]

Kernel_Main (C++) dissasembly:

Is that one disassembly or two? What's the difference between left and right? You did notice that only one of them calls "Call_Constructors", right? Is Memory_Core a global object? How did you initialize it?

You have read the osdev wiki's relating to C++ and the stuff you need to take care of, right?

Those are two different dissasemblies. Yes I did notice it. Yes it is global. I initialized it as everything else, note initialize != constructed.
Yes I know what is going on with C++ constructors etc..

[Octacone]

Show us ALL of your code of the entrypoint in the kernel asm file.
Im pretty sure your pushing it in the wrong order

This one?

Code: Select all

Bootloader_Main:
	mov esp, stack_top
	push ebx
	push eax
	call Kernel_Main
	jmp $

[Ch4ozz]

Push order is correct, is your stack well aligned? (stack_top)
Do you do all these tests with paging turned off?
Btw GAS (AT&T) syntax is cancer, I dont think alot of people like to read that

[Octacone]

Push order is correct, is your stack well aligned? (stack_top)
Do you do all these tests with paging turned off?
Btw GAS syntax is cancer, I dont think alot of people like to read that

GAS syntax is worse than a cancer. *puke*
That is why all of my assembly code is AT&T compatible.
I hope there is a way to change it. It would be much easier for me to understand.

Here:

Code: Select all

section .bss
align 16
stack_bottom:
	resb 1024 * 64
stack_top:

[LtG]

So what happens if you single-step/breakpoint at the first instruction in Kernel_Main, what's the value of EAX, EBX, and the two multiboot variables you are passing onto the function?

Test with both versions, are they both correct? Does the MBI structure that is being pointed at by EBX look valid with both versions?

Also, which of the two side-by-side versions of disasm you showed works and which doesn't?

How do you handle global objects?

[Octacone]

So what happens if you single-step/breakpoint at the first instruction in Kernel_Main, what's the value of EAX, EBX, and the two multiboot variables you are passing onto the function?

Test with both versions, are they both correct? Does the MBI structure that is being pointed at by EBX look valid with both versions?

Also, which of the two side-by-side versions of disasm you showed works and which doesn't?

How do you handle global objects?

Left one works, right one doesn't.

Actually I found out what is going on.
When I comment out Call_Constructors(); GRUB doesn't report 16 KB any more.

Code: Select all

//"Borrowed" from @max, was lazy to consider looking at it myself
typedef void (*function_pointer)();
extern "C" void* start_constructors;
extern "C" void* end_constructors;

extern "C" void Call_Constructors()
{
	function_pointer* start = (function_pointer*) & start_constructors;
	function_pointer* end = (function_pointer*) & end_constructors;
	for(function_pointer* current = start; current != end; current++)
	{
		(*current)();
	}
}

Inside linker's data block:

Code: Select all

start_constructors = .;
*(.ctor*)
KEEP(*(.init_array));
KEEP(*(SORT_BY_INIT_PRIORITY(.init_array.*)));
end_constructors = .;