OSDev.org

The Place to Start for Operating System Developers
It is currently Mon Mar 18, 2024 10:45 pm

All times are UTC - 6 hours




Post new topic Reply to topic  [ 95 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6, 7  Next
Author Message
 Post subject: Re: Undefined Array Triple Fault
PostPosted: Sun Jul 16, 2017 7:31 pm 
Offline
Member
Member

Joined: Fri Aug 19, 2016 10:28 pm
Posts: 360
I think you should first narrow the location of the problem in the source and then seek the cause. You seem to be probing with brute-force for indication of the root cause right off the bat.

Think what actually is the problem you are trying to eliminate. The other forum members assisted you and you found out that the page directory pointer in cr3 is null. Why is that? Which code is responsible to set it? You should know that.

Assuming that you think the code that is responsible should work as intended and cr3 should be properly set, you must use gdb to trace the respective code and check what's wrong. If that is insufficient and there are other unknown pieces of code involved, use any instruments that make sense. Use access breakpoints, or set variables (preferably "volatile") in various places in the code just to track executed branches, then read them with gdb, comment code paths to reduce the scope of the bug search, print diagnostic messages. You must locate the line or couple of lines that violate your intents. Then, assuming you don't find a bug in them, you can disassemble them, and if the assembly does not correspond to the language specification, you can infer that the compiler is buggy. But this is unlikely and usually you will find your own mistake.


Top
 Profile  
 
 Post subject: Re: Undefined Array Triple Fault
PostPosted: Mon Jul 17, 2017 1:20 am 
Offline
Member
Member

Joined: Thu Aug 13, 2015 4:57 pm
Posts: 384
Octacone wrote:
Here are some things I discovered.

-O2 enabled, PMM works perfectly, returns page aligned addresses, all good.
-O2 disabled, PMM broken as hell, only returns 0x0.

I am so mad at the compiler. Why does it keep breaking my code! Then that means, no PMM -> no VMM. Thus it is probably not VMM's fault.

Edit:
Manged to track down the issue.
You would never have guessed...
GRUB only reports 16 KILOBYTES of memory available with -02 disabled. 16 freaking kilobytes, and then I wonder why my memory managers don't work.
Now I need to find why is that so.


The problem isn't the compiler, it's you. The compiler doesn't break your code, the code was broken to begin with. Optimizations affect the code produced and so with broken code it may sometimes seemingly break or fix things, but the cause is that the underlying code (your code) is fundamentally broken.

Essentially your code may never rely on anything outside the code, so even if you set some pointer to point to some memory address where there's some data left by GRUB/multiboot/BIOS/etc the compiler might _NOT_ read it and might do anything it wants. Your C code does not exist in a real physical machine not even in a real virtual machine, it exists in an abstract machine. When you need to break the abstraction you need to inform the compiler, for instance use the keyword volatile to instruct the compiler that it must actually read from memory and not assume things.

For instance:
Code:
int* ptr = 0x1234;
*ptr = 555;
int b = 5 + *ptr;


Can be translated to:
Code:
int b = 560;


Of course if there's not other code besides that, then the compiler can remove that "int b = 560;" line as well because it has no side effects and therefore is irrelevant.

Let's say you had memory mapped IO, where 0x1234 is actually some device and you wanted to write 555 to it so the device would do something, that write never happens, however as far as your code is concerned, internally it still knows the values 555 and 560 and will use them in all locations where they are needed, but outside of your code, the 555 may never be written to 0x1234.

Why don't you just debug the issue, once you know where and how it happens it should be relatively trivial to figure out why it happens. If it's not, you can post the relevant asm and C code here and we can help you figure it out.

At this point it seems that your O2 vs !O2 have differing behavior with trying to figure out how much RAM there is.. Though why are you even detecting the _amount_ of RAM, instead of getting a memory map? How do you attempt to get the amount of memory?


Top
 Profile  
 
 Post subject: Re: Undefined Array Triple Fault
PostPosted: Mon Jul 17, 2017 3:52 am 
Offline
Member
Member
User avatar

Joined: Fri Aug 07, 2015 6:13 am
Posts: 1134
simeonz wrote:
I think you should first narrow the location of the problem in the source and then seek the cause. You seem to be probing with brute-force for indication of the root cause right off the bat.

Think what actually is the problem you are trying to eliminate. The other forum members assisted you and you found out that the page directory pointer in cr3 is null. Why is that? Which code is responsible to set it? You should know that.

Assuming that you think the code that is responsible should work as intended and cr3 should be properly set, you must use gdb to trace the respective code and check what's wrong. If that is insufficient and there are other unknown pieces of code involved, use any instruments that make sense. Use access breakpoints, or set variables (preferably "volatile") in various places in the code just to track executed branches, then read them with gdb, comment code paths to reduce the scope of the bug search, print diagnostic messages. You must locate the line or couple of lines that violate your intents. Then, assuming you don't find a bug in them, you can disassemble them, and if the assembly does not correspond to the language specification, you can infer that the compiler is buggy. But this is unlikely and usually you will find your own mistake.


I actually know what is going on. Looks like you didn't read my update. So the PMM should provide an address for the VMM to use but it is only providing 0x0, why? Because GRUB returns only 16 KB of total RAM, which crashes the entire OS memory system. So the question is why does it happen. I can set CR3 to any value I want, that is not questionable.

_________________
OS: Basic OS
About: 32 Bit Monolithic Kernel Written in C++ and Assembly, Custom FAT 32 Bootloader


Top
 Profile  
 
 Post subject: Re: Undefined Array Triple Fault
PostPosted: Mon Jul 17, 2017 4:10 am 
Offline
Member
Member
User avatar

Joined: Fri Aug 07, 2015 6:13 am
Posts: 1134
LtG wrote:
Octacone wrote:
Here are some things I discovered.

-O2 enabled, PMM works perfectly, returns page aligned addresses, all good.
-O2 disabled, PMM broken as hell, only returns 0x0.

I am so mad at the compiler. Why does it keep breaking my code! Then that means, no PMM -> no VMM. Thus it is probably not VMM's fault.

Edit:
Manged to track down the issue.
You would never have guessed...
GRUB only reports 16 KILOBYTES of memory available with -02 disabled. 16 freaking kilobytes, and then I wonder why my memory managers don't work.
Now I need to find why is that so.


The problem isn't the compiler, it's you. The compiler doesn't break your code, the code was broken to begin with. Optimizations affect the code produced and so with broken code it may sometimes seemingly break or fix things, but the cause is that the underlying code (your code) is fundamentally broken.

Essentially your code may never rely on anything outside the code, so even if you set some pointer to point to some memory address where there's some data left by GRUB/multiboot/BIOS/etc the compiler might _NOT_ read it and might do anything it wants. Your C code does not exist in a real physical machine not even in a real virtual machine, it exists in an abstract machine. When you need to break the abstraction you need to inform the compiler, for instance use the keyword volatile to instruct the compiler that it must actually read from memory and not assume things.

For instance:
Code:
int* ptr = 0x1234;
*ptr = 555;
int b = 5 + *ptr;


Can be translated to:
Code:
int b = 560;


Of course if there's not other code besides that, then the compiler can remove that "int b = 560;" line as well because it has no side effects and therefore is irrelevant.

Let's say you had memory mapped IO, where 0x1234 is actually some device and you wanted to write 555 to it so the device would do something, that write never happens, however as far as your code is concerned, internally it still knows the values 555 and 560 and will use them in all locations where they are needed, but outside of your code, the 555 may never be written to 0x1234.

Why don't you just debug the issue, once you know where and how it happens it should be relatively trivial to figure out why it happens. If it's not, you can post the relevant asm and C code here and we can help you figure it out.

At this point it seems that your O2 vs !O2 have differing behavior with trying to figure out how much RAM there is.. Though why are you even detecting the _amount_ of RAM, instead of getting a memory map? How do you attempt to get the amount of memory?


So the compiler can do anything it wants without caring if it will work. That is just awesome...
Basically total_memory = memory_lower (GRUB) + memory_upper (GRUB), can't be more simple than that. The problem is that it only returns 16 KB instead of "n" GB.
Forget about the memory map, it is implemented, but used for some other mapping stuff. Don't focus on that right now, it is irrelevant.

_________________
OS: Basic OS
About: 32 Bit Monolithic Kernel Written in C++ and Assembly, Custom FAT 32 Bootloader


Top
 Profile  
 
 Post subject: Re: Undefined Array Triple Fault
PostPosted: Mon Jul 17, 2017 4:30 am 
Offline
Member
Member

Joined: Thu Aug 13, 2015 4:57 pm
Posts: 384
Octacone wrote:
So the compiler can do anything it wants without caring if it will work. That is just awesome...
Basically total_memory = memory_lower (GRUB) + memory_upper (GRUB), can't be more simple than that. The problem is that it only returns 16 KB instead of "n" GB.
Forget about the memory map, it is implemented, but used for some other mapping stuff. Don't focus on that right now, it is irrelevant.

I was "focusing" on the memory map because doing it wrong (lower+upper) is more difficult than doing it right (memory map), so wasting time fixing the wrong way, just so you can later drop it and do it the right way is just a waste of time.

If you adamantly want to do it the wrong way, sure, I can still try to help you.

Now, assuming there's no difference in GRUB between the two versions (and the VM's, etc), then we can safely say that GRUB provides you with the correct values, you are not fetching them correctly.

The compiler can do anything it wants within the confines of the language, without caring if it works. _YOU_ need to care enough to write _CORRECT_ code, not random stuff and expect the compiler to fix it. Your code is broken, it is wrong, that's the cause of your problems.

Can you show the code how you get these (GRUB provided memory) values in C? Since you now know they are wrong, you could also have already used gdb to check why you are getting invalid values, and if you don't understand it yourself, you could have posted the relevant C and disasm here after you used gdb to check what the reason is.


Top
 Profile  
 
 Post subject: Re: Undefined Array Triple Fault
PostPosted: Mon Jul 17, 2017 4:40 am 
Offline
Member
Member
User avatar

Joined: Fri Aug 07, 2015 6:13 am
Posts: 1134
LtG wrote:
Octacone wrote:
So the compiler can do anything it wants without caring if it will work. That is just awesome...
Basically total_memory = memory_lower (GRUB) + memory_upper (GRUB), can't be more simple than that. The problem is that it only returns 16 KB instead of "n" GB.
Forget about the memory map, it is implemented, but used for some other mapping stuff. Don't focus on that right now, it is irrelevant.

I was "focusing" on the memory map because doing it wrong (lower+upper) is more difficult than doing it right (memory map), so wasting time fixing the wrong way, just so you can later drop it and do it the right way is just a waste of time.

If you adamantly want to do it the wrong way, sure, I can still try to help you.

Now, assuming there's no difference in GRUB between the two versions (and the VM's, etc), then we can safely say that GRUB provides you with the correct values, you are not fetching them correctly.

The compiler can do anything it wants within the confines of the language, without caring if it works. _YOU_ need to care enough to write _CORRECT_ code, not random stuff and expect the compiler to fix it. Your code is broken, it is wrong, that's the cause of your problems.

Can you show the code how you get these (GRUB provided memory) values in C? Since you now know they are wrong, you could also have already used gdb to check why you are getting invalid values, and if you don't understand it yourself, you could have posted the relevant C and disasm here after you used gdb to check what the reason is.


Actually it is C++. I don't know how can that be harder. So you think I should do something like total_memory = last memory map address? It is not "random stuff", I hope you are not talking about my code...
I'll post everything relevant, first I need to check it once more with GDB.

Edit: oops that won't work, can you explain your memory map idea?
Edit 2: forget about that idea until we fix the main issue, memory map is also "destroyed"...

_________________
OS: Basic OS
About: 32 Bit Monolithic Kernel Written in C++ and Assembly, Custom FAT 32 Bootloader


Top
 Profile  
 
 Post subject: Re: Undefined Array Triple Fault
PostPosted: Mon Jul 17, 2017 5:24 am 
Offline
Member
Member
User avatar

Joined: Fri Aug 07, 2015 6:13 am
Posts: 1134
None of the GRUB flags are being set.
Here is some relevant code:

Bootloader.asm:
Code:
bits 32
section .multiboot
multiboot_align         equ   1 << 0
multiboot_memory_info   equ   1 << 1
multiboot_flags         equ   multiboot_align | multiboot_memory_info
multiboot_header_magic  equ   0x1BADB002
multiboot_checksum      equ   - (multiboot_header_magic + multiboot_flags)

align 4
dd multiboot_header_magic
dd multiboot_flags
dd multiboot_checksum

...
Bootloader_Main:
      ...
      push ebx
      ...


Kernel.cpp
Code:
extern "C" void Kernel_Main(uint32_t multiboot_eax_magic, multiboot_info_t* multiboot_info)
{
      Memory_Core.Initialize(multiboot_info); //this part is less relevant, sort of
      //this matters:
      //no matter where/how I do this, it returns wrong values:
      TUI.Put_String("\nMemory upper: ", 0x07);
      TUI.Put_Hex(multiboot_info->memory_upper, 0x07); -> instead of 2 GB (aka 2 097 152 KB) it displays 16 KB only.
      Also multiboot_info->flags --->>> all zeros no bytes being set
}


So little code, so much troubles.

Edit:
Kernel_Main (C++) dissasembly:


Attachments:
KernelMainASM.png
KernelMainASM.png [ 118.98 KiB | Viewed 3496 times ]

_________________
OS: Basic OS
About: 32 Bit Monolithic Kernel Written in C++ and Assembly, Custom FAT 32 Bootloader
Top
 Profile  
 
 Post subject: Re: Undefined Array Triple Fault
PostPosted: Mon Jul 17, 2017 6:10 am 
Offline
Member
Member

Joined: Thu Aug 13, 2015 4:57 pm
Posts: 384
Octacone wrote:
Kernel_Main (C++) dissasembly:

Is that one disassembly or two? What's the difference between left and right? You did notice that only one of them calls "Call_Constructors", right? Is Memory_Core a global object? How did you initialize it?

You have read the osdev wiki's relating to C++ and the stuff you need to take care of, right?


Top
 Profile  
 
 Post subject: Re: Undefined Array Triple Fault
PostPosted: Mon Jul 17, 2017 6:13 am 
Offline
Member
Member
User avatar

Joined: Mon Jul 18, 2016 2:46 pm
Posts: 170
Show us ALL of your code of the entrypoint in the kernel asm file.
Im pretty sure your pushing it in the wrong order


Top
 Profile  
 
 Post subject: Re: Undefined Array Triple Fault
PostPosted: Mon Jul 17, 2017 6:23 am 
Offline
Member
Member
User avatar

Joined: Fri Aug 07, 2015 6:13 am
Posts: 1134
LtG wrote:
Octacone wrote:
Kernel_Main (C++) dissasembly:

Is that one disassembly or two? What's the difference between left and right? You did notice that only one of them calls "Call_Constructors", right? Is Memory_Core a global object? How did you initialize it?

You have read the osdev wiki's relating to C++ and the stuff you need to take care of, right?


Those are two different dissasemblies. Yes I did notice it. Yes it is global. I initialized it as everything else, note initialize != constructed.
Yes I know what is going on with C++ constructors etc..

_________________
OS: Basic OS
About: 32 Bit Monolithic Kernel Written in C++ and Assembly, Custom FAT 32 Bootloader


Top
 Profile  
 
 Post subject: Re: Undefined Array Triple Fault
PostPosted: Mon Jul 17, 2017 6:25 am 
Offline
Member
Member
User avatar

Joined: Fri Aug 07, 2015 6:13 am
Posts: 1134
Ch4ozz wrote:
Show us ALL of your code of the entrypoint in the kernel asm file.
Im pretty sure your pushing it in the wrong order


This one?
Code:
Bootloader_Main:
   mov esp, stack_top
   push ebx
   push eax
   call Kernel_Main
   jmp $

_________________
OS: Basic OS
About: 32 Bit Monolithic Kernel Written in C++ and Assembly, Custom FAT 32 Bootloader


Top
 Profile  
 
 Post subject: Re: Undefined Array Triple Fault
PostPosted: Mon Jul 17, 2017 6:28 am 
Offline
Member
Member
User avatar

Joined: Mon Jul 18, 2016 2:46 pm
Posts: 170
Push order is correct, is your stack well aligned? (stack_top)
Do you do all these tests with paging turned off?
Btw GAS (AT&T) syntax is cancer, I dont think alot of people like to read that


Top
 Profile  
 
 Post subject: Re: Undefined Array Triple Fault
PostPosted: Mon Jul 17, 2017 6:37 am 
Offline
Member
Member
User avatar

Joined: Fri Aug 07, 2015 6:13 am
Posts: 1134
Ch4ozz wrote:
Push order is correct, is your stack well aligned? (stack_top)
Do you do all these tests with paging turned off?
Btw GAS syntax is cancer, I dont think alot of people like to read that


GAS syntax is worse than a cancer. *puke*
That is why all of my assembly code is AT&T compatible.
I hope there is a way to change it. It would be much easier for me to understand.

Here:
Code:
section .bss
align 16
stack_bottom:
   resb 1024 * 64
stack_top:

_________________
OS: Basic OS
About: 32 Bit Monolithic Kernel Written in C++ and Assembly, Custom FAT 32 Bootloader


Top
 Profile  
 
 Post subject: Re: Undefined Array Triple Fault
PostPosted: Mon Jul 17, 2017 7:02 am 
Offline
Member
Member

Joined: Thu Aug 13, 2015 4:57 pm
Posts: 384
So what happens if you single-step/breakpoint at the first instruction in Kernel_Main, what's the value of EAX, EBX, and the two multiboot variables you are passing onto the function?

Test with both versions, are they both correct? Does the MBI structure that is being pointed at by EBX look valid with both versions?

Also, which of the two side-by-side versions of disasm you showed works and which doesn't?

How do you handle global objects?


Top
 Profile  
 
 Post subject: Re: Undefined Array Triple Fault
PostPosted: Mon Jul 17, 2017 7:22 am 
Offline
Member
Member
User avatar

Joined: Fri Aug 07, 2015 6:13 am
Posts: 1134
LtG wrote:
So what happens if you single-step/breakpoint at the first instruction in Kernel_Main, what's the value of EAX, EBX, and the two multiboot variables you are passing onto the function?

Test with both versions, are they both correct? Does the MBI structure that is being pointed at by EBX look valid with both versions?

Also, which of the two side-by-side versions of disasm you showed works and which doesn't?

How do you handle global objects?


Left one works, right one doesn't.

Actually I found out what is going on.
When I comment out Call_Constructors(); GRUB doesn't report 16 KB any more.

Code:
//"Borrowed" from @max, was lazy to consider looking at it myself
typedef void (*function_pointer)();
extern "C" void* start_constructors;
extern "C" void* end_constructors;

extern "C" void Call_Constructors()
{
   function_pointer* start = (function_pointer*) & start_constructors;
   function_pointer* end = (function_pointer*) & end_constructors;
   for(function_pointer* current = start; current != end; current++)
   {
      (*current)();
   }
}


Inside linker's data block:
Code:
start_constructors = .;
*(.ctor*)
KEEP(*(.init_array));
KEEP(*(SORT_BY_INIT_PRIORITY(.init_array.*)));
end_constructors = .;

_________________
OS: Basic OS
About: 32 Bit Monolithic Kernel Written in C++ and Assembly, Custom FAT 32 Bootloader


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 95 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6, 7  Next

All times are UTC - 6 hours


Who is online

Users browsing this forum: Bing [Bot], DotBot [Bot] and 10 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group