OSDev.org

The Place to Start for Operating System Developers
It is currently Mon Oct 14, 2019 7:38 pm

All times are UTC - 6 hours




Post new topic Reply to topic  [ 14 posts ] 
Author Message
 Post subject: Out Of Memory error by GRUB when loading Higher Half kernel
PostPosted: Sun Jan 28, 2018 11:55 pm 
Offline

Joined: Fri Sep 09, 2016 5:52 pm
Posts: 24
Location: Australia
I have been trying to make a Higher Half Kernel and looked at the wiki page on doing so and when i got to testing GRUB gave me this error:
Image
I have tried with different amounts of RAM in my VM from 128 to 1024 MB and have tried with both Hyper-V and QEMU.

Entry file:
Code:
BITS    32

global entry
extern kmain

MBALIGN     equ 1<<0
MEMINFO     equ 1<<1
MAGIC       equ 0x1BADB002
FLAGS       equ MBALIGN | MEMINFO
CHECKSUM    equ -(MAGIC + FLAGS)

KERNEL_VIRTUAL_BASE equ 0xC0000000
KERNEL_PAGE_NUMBER equ KERNEL_VIRTUAL_BASE >> 22

section .data
align 4096
KernelPageDirectory:
   dd 0x00000083
    times (KERNEL_PAGE_NUMBER - 1) dd 0         ; Pages before kernel space
    dd 0x00000083
    times (1024 - KERNEL_PAGE_NUMBER - 1) dd 0   ; Pages after kernel space

section .text
align 4 ; Multiboot Header
dd MAGIC
dd FLAGS
dd CHECKSUM

entry:
   mov ecx, (KernelPageDirectory - KERNEL_VIRTUAL_BASE)
   mov cr3, ecx

   mov ecx, cr4
   or ecx, 0x00000010
   mov cr0, ecx

   lea ecx, [entry_higher_half]
   jmp ecx ; Jump to higher half

entry_higher_half:
   mov dword [KernelPageDirectory],0
   invlpg [0]

    mov esp, stack_top
    push eax

   add ebx, KERNEL_VIRTUAL_BASE
    push ebx
    call kmain ; Load C++ part of kernel
   
    cli
    hlt

section .bss
align 4
stack_bottom:
resb 16384
stack_top:


Linker Script:

Code:
ENTRY(entry)
OUTPUT_FORMAT(elf32-i386)

SECTIONS {
   . = 0xC0100000;

   .text : AT(ADDR(.text) - 0xC0000000) {
       *(.text)
       
      . = ALIGN(4096);
   }

   .data : AT(ADDR(.data) - 0xC0000000) {
       *(.data)
      *(.rodata*)
      . = ALIGN(4096);
   }

   .bss : AT(ADDR(.bss) - 0xC0000000) {
       *(.bss)
      . = ALIGN(4096);
   }
}


Top
 Profile  
 
 Post subject: Re: Out Of Memory error by GRUB when loading Higher Half ker
PostPosted: Fri Feb 02, 2018 6:44 pm 
Offline
Member
Member

Joined: Fri Aug 26, 2016 1:41 pm
Posts: 485
Personally I believe that the higher half barebones wiki that you are referencing is flawed. GRUB and a Multiboot compliant bootloader in all likelihood will not understand an entry point in the higher half. If it does you just got lucky. I believe it is better to ensure that the Multiboot entry point is in lower memory, not in higher memory. This should prevent GRUB from getting thoroughly confused. Most of the changes to make this work are mostly with the liner script and the addition of some new sections to separate what is needed when the kernel runs in lower half and then what is needed in the higher half.

A version of the linker script could look like:
Code:
ENTRY(entry)
OUTPUT_FORMAT(elf32-i386)

KERNEL_VIRTUAL_BASE = 0xC0000000;

SECTIONS {
   /* The multiboot data and code will exist in low memory
      starting at 0x100000 */

   . = 0x00100000;
   .multiboot.data : {
       *(.multiboot.data)
   }

   .multiboot.text : {
       *(.multiboot.text)
   }

   /* The kernel will live at 3GB + 1MB in the virtual
      address space, which will be mapped to 1MB in the
      physical address space. */

   . += KERNEL_VIRTUAL_BASE;
   .text ALIGN(4096) : AT(ADDR(.text) - KERNEL_VIRTUAL_BASE) {
       *(.text)
   }

   .data ALIGN (4096) : AT(ADDR(.data) - KERNEL_VIRTUAL_BASE) {
       *(.data)
       *(.rodata*)
   }

   .bss ALIGN (4096) : AT(ADDR(.bss) - KERNEL_VIRTUAL_BASE) {
       _sbss = .;
       *(COMMON)
       *(.bss)
       _ebss = .;
   }

   /DISCARD/ : {
       *(.eh_frame);
       *(.comment*);
   }
}
The loader assembly file could look like:
Code:
extern kmain
global entry

; Multiboot header in lower memory
MBALIGN  equ  1<<0
MEMINFO  equ  1<<1
FLAGS    equ  MBALIGN | MEMINFO
MAGIC    equ  0x1BADB002
CHECKSUM equ -(MAGIC + FLAGS)

KERNEL_VIRTUAL_BASE equ 0xC0000000     ; 3GB
KERNEL_PAGE_NUMBER equ (KERNEL_VIRTUAL_BASE >> 22)

section .multiboot.data
align 4
MultiBootHeader:
    dd MAGIC
    dd FLAGS
    dd CHECKSUM

; Stack size
STACKSIZE equ 16*1024

align 0x1000
BootPageDirectory:
    dd 0x00000083
    times (KERNEL_PAGE_NUMBER - 1) dd 0
    ; This page directory entry defines a 4MB page containing the kernel.
    dd 0x00000083
    times (1024 - KERNEL_PAGE_NUMBER - 1) dd 0

; Multiboot code entry point in lower memory
section .multiboot.text progbits alloc exec nowrite align=16

; setting up entry point for linker
entry:
    mov ecx, BootPageDirectory
    mov cr3, ecx

    mov ecx, cr4
    or ecx, 0x00000010                 ; Set PSE bit in to enable 4MB pages
    mov cr4, ecx

    mov ecx, cr0
    or ecx, 0x80000000                 ; Set PG bit to enable paging.
    mov cr0, ecx

    jmp .StartInHigherHalf

; .text will be put in the higher half by the linker script
section .text
.StartInHigherHalf:

    ; Unmap the identity-mapped first 4MB
    mov dword [BootPageDirectory], 0
    invlpg [0]

    mov esp, stack+STACKSIZE           ; set up the stack

    push eax                           ; Multiboot magic number
    ; Map the multiboot structure to higher half
    add ebx, KERNEL_VIRTUAL_BASE
    push ebx
    call  kmain                        ; call kernel entry point

    cli                                ; Disable interrupts
.endloop:
    hlt
    jmp .endloop                       ; If we return here infinite loop with hlt

section .bss
align 32
stack:
    resb STACKSIZE                     ; 16kb stack
This code also makes the false assumption that the entire Mulltiboot structure is in the first 4MB. The multiboot spec doesn't actually guarantee this. Doing add ebx, KERNEL_VIRTUAL_BASE also doesn't account for the pointers inside the Multiboot structure that also need to be fixed up and made higher half. To simplify things you could just keep the first 4MB identity mapped after jumping to the higher half side of the kernel. Then these pointers would not have to be fixed up at all. Once you are finished retrieving the needed info in the Multiboot structure then you could then unmap it.


Top
 Profile  
 
 Post subject: Re: Out Of Memory error by GRUB when loading Higher Half ker
PostPosted: Sat Feb 03, 2018 4:28 pm 
Offline

Joined: Fri Sep 09, 2016 5:52 pm
Posts: 24
Location: Australia
Trying that, I am now getting
Code:
error: no multiboot header found.

I ran objdump and it still however says that the mutliboot header is at 1M
Code:
00100000 l    d  .multiboot.hdr   00000000 .multiboot.hdr

_________________
My OS:
https://github.com/fido2020/Lemon-OS


Top
 Profile  
 
 Post subject: Re: Out Of Memory error by GRUB when loading Higher Half ker
PostPosted: Sat Feb 03, 2018 4:44 pm 
Offline
Member
Member

Joined: Fri Aug 26, 2016 1:41 pm
Posts: 485
First, what are you building this on? Windows? Linux? MacOS? 32-bit or 64-bit compiler? What commands do you use to compile and link? How do you build the kernel image? How do you run it I suspect what you are now experiencing is specific to your build environment and/or your build process. What is the entire output of `objdump -x` show on your elf executable?

If you could put your project on github or some other repository we might get a better sense of what may be going wrong. I suspect that whatever is causing the multiboot header not to be found is because something else has been placed in front of it in the elf executable pushing it outside the first 8k of the file. If the multiboot header is not in the first 8k of the file then a Multiboot compliant loader (like GRUB) likely won't see it.


Top
 Profile  
 
 Post subject: Re: Out Of Memory error by GRUB when loading Higher Half ker
PostPosted: Sun Feb 04, 2018 1:37 am 
Offline

Joined: Fri Sep 09, 2016 5:52 pm
Posts: 24
Location: Australia
https://github.com/fido2020/Lemon-OS

My output from objdump -x:
Attachment:
File comment: output from objdump
dump.txt [11.61 KiB]
Downloaded 18 times


I am using Windows Subsystem for Linux with Visual Studio:
These are the commands it uses to build and link:
Code:
g++ -std=c++14 -Wall" -fno-rtti -fno-exceptions -o "D:\OneDrive\Documents\Lemon\Kernel\obj\x86\%(filename).o" -m32 --freestanding -Wno-write-strings -I $(RemoteProjectDir)/src/arch/x86/include -I $(RemoteProjectDir)/src/include

Code:
g++ -o "<output>" -m32 -T ~/projects/Kernel/linkscript.ld -ffreestanding -O2 -nostdlib ~/projects/Kernel/obj/x86/*.asm.o <other input files>

_________________
My OS:
https://github.com/fido2020/Lemon-OS


Top
 Profile  
 
 Post subject: Re: Out Of Memory error by GRUB when loading Higher Half ker
PostPosted: Sun Feb 04, 2018 2:10 am 
Offline
Member
Member

Joined: Fri Aug 26, 2016 1:41 pm
Posts: 485
I notice that you aren't using a cross compiler. You should. with that being said if you look at the objdump output you'll notice that the multiboot.hdr is actually placed at offset 0x2000 in the file. This is beyond the first 8k and why it can't be found by GRUB. You should observe that in the section just before it there is a build-id section that was placed there:
Code:
Sections:

     Idx Name          Size      VMA       LMA       File off  Algn
  0 .note.gnu.build-id 00000024  00000000  00000000  00001000  2**2
                       CONTENTS, ALLOC, LOAD, READONLY, DATA
  1 .multiboot.hdr     0000000c  00100000  00100000  00002000  2**2
                       CONTENTS, ALLOC, LOAD, READONLY, DATA
You can eliminate this by modifying your linking line (g++) by adding the option -Wl,--build-id=none
Code:
g++ -o "<output>" -m32 -T ~/projects/Kernel/linkscript.ld -Wl,--build-id=none -ffreestanding -O2 -nostdlib ~/projects/Kernel/obj/x86/*.asm.o <other input files>
By eliminating the build id during the linking process we can get rid of this section. This may allow your multiboot header to be placed starting at 0x1000 instead of 0x2000. As well you have a typo in this line section .mutliboot.data it should be:
Code:
section .multiboot.data
In your updated code you have a bug setting up paging:
Code:
mov ecx, (KernelPageDirectory - KERNEL_VIRTUAL_BASE)
needs to be:
Code:
mov ecx, KernelPageDirectory
. This is because with the linker script I gave you KernelPageDirectory isalready a low memory address so you don't adjust it. There is a typo here:
Code:
mov ecx, cr4
or ecx, 0x00000010
mov cr0, ecx
should be
Code:
mov ecx, cr4
or ecx, 0x00000010
mov cr4, ecx
In your VGA.CPP use use the low memory address 0xb8000 for video memory. Since you unmapped the identity mapping for the first page of RAM you need to use the high memory address. The address now is:
Code:
uint8_t* video_memory = (uint8_t*)0xC00B8000;
You also have created a large sized structure for your page tables. This line in paging.cpp:
Code:
page_table_t page_tables[1024];
will require a fair amount of BSS space and will make your code and data exceed the 4MB page that has been mapped. You are now in a position where if you keep the code this way you'll need to map more than the first 4MB of ram to high memory. If you don't you'll fault the processor when it tries to access any data outside the first 4MB of the kernel. You could add an additional 4MB to the initial mapping by changing the initial kernel page table to appear as:
Code:
KERNEL_NUM_PAGES equ 2
section .multiboot.data
align 4096
KernelPageDirectory:
    dd 0x00000083 | 0<<22                       ; Identity map 0x00000000-0x003fffff
    times (KERNEL_PAGE_NUMBER - 1) dd 0         ; Pages before kernel space
    dd 0x00000083 | 0<<22                       ; Map 0xc0000000-0xc03fffff to 0x00000000-0x003fffff
    dd 0x00000083 | 1<<22                       ; Map 0xc0400000-0xc07fffff to 0x00400000-0x007fffff
    times (1024 - KERNEL_PAGE_NUMBER - KERNEL_NUM_PAGES) dd 0
                                                ; Pages after kernel space

---
When compiling your files without a cross compiler you may find that the code generated will be position independent. That will cause issues for the code generated. You should add the option -fno-pic along side -ffreestanding. I also noticed that you have a typo in your answer above. The line you are using to compile is using --freestanding when it should be -ffreestanding


Top
 Profile  
 
 Post subject: Re: Out Of Memory error by GRUB when loading Higher Half ker
PostPosted: Sat Feb 10, 2018 1:28 am 
Offline

Joined: Fri Sep 09, 2016 5:52 pm
Posts: 24
Location: Australia
Thanks a lot, I got the Higher Half working but I found out that the paging code i was using after entering the C++ Kernel was causing a fault in QEMU but hanging in VirtualBox without writing anything to the serial debug file - even before i switched the directory and i managed to fix the fault but now in both QEMU and VirtualBox it hangs whilst corrupting the video memory.


Attachments:
corruptedvidmem.PNG
corruptedvidmem.PNG [ 11.13 KiB | Viewed 1503 times ]

_________________
My OS:
https://github.com/fido2020/Lemon-OS
Top
 Profile  
 
 Post subject: Re: Out Of Memory error by GRUB when loading Higher Half ker
PostPosted: Sat Feb 10, 2018 2:11 am 
Offline
Member
Member

Joined: Fri Aug 26, 2016 1:41 pm
Posts: 485
My recommendation would be to use Bochs to debug this. Bochs doesn't have symbolic debugging but you can use objdump to dump out the symbols and addresses. Find an address of an instruction that is right after you do the video mapping. Run until that break point. Use the command info tab to dump the page tables. Make sure that the mappings that you did are correct.


Top
 Profile  
 
 Post subject: Re: Out Of Memory error by GRUB when loading Higher Half ker
PostPosted: Sat Feb 10, 2018 9:26 pm 
Offline

Joined: Fri Sep 09, 2016 5:52 pm
Posts: 24
Location: Australia
I have found out that for some reason paging is getting disabled as when i ran that command in the bochs debugger it told me paging was off.
Here are the logs from where GRUB loaded the kernel
Code:
00299904720i[WINGUI] dimension update x=720 y=400 fontheight=16 fontwidth=9 bpp=8
00359292309e[CPU0  ] interrupt(): gate.type(2) != {5,6,7,14,15}
00359292309e[CPU0  ] interrupt(): gate not present
00359292309i[CPU0  ] CPU is in protected mode (active)
00359292309i[CPU0  ] CS.mode = 32 bit
00359292309i[CPU0  ] SS.mode = 32 bit
00359292309i[CPU0  ] EFER   = 0x00000000
00359292309i[CPU0  ] | EAX=00000000  EBX=c0010000  ECX=0000002f  EDX=000003f8
00359292309i[CPU0  ] | ESP=c010b030  EBP=c010b040  ESI=00000000  EDI=00000000
00359292309i[CPU0  ] | IOPL=0 id vip vif ac vm RF nt of df if tf sf ZF af PF cf
00359292309i[CPU0  ] | SEG sltr(index|ti|rpl)     base    limit G D
00359292309i[CPU0  ] |  CS:0008( 0001| 0|  0) 00000000 ffffffff 1 1
00359292309i[CPU0  ] |  DS:0010( 0002| 0|  0) 00000000 ffffffff 1 1
00359292309i[CPU0  ] |  SS:0010( 0002| 0|  0) 00000000 ffffffff 1 1
00359292309i[CPU0  ] |  ES:0010( 0002| 0|  0) 00000000 ffffffff 1 1
00359292309i[CPU0  ] |  FS:0010( 0002| 0|  0) 00000000 ffffffff 1 1
00359292309i[CPU0  ] |  GS:0010( 0002| 0|  0) 00000000 ffffffff 1 1
00359292309i[CPU0  ] | EIP=c010406a (c010406a)
00359292309i[CPU0  ] | CR0=0xe0000011 CR2=0x80000f5b
00359292309i[CPU0  ] | CR3=0x00101000 CR4=0x00000000
(0).[359292309] [0x00000000006a] 0008:00000000c010406a (unk. ctxt): add byte ptr ds:[eax-2147479717], al ; 00805b0f0080
00359292309e[CPU0  ] exception(): 3rd (11) exception with no resolution, shutdown status is 00h, resetting
00359292309i[SYS   ] bx_pc_system_c::Reset(HARDWARE) called
00359292309i[CPU0  ] cpu hardware reset
00359292309i[APIC0 ] allocate APIC id=0 (MMIO enabled) to 0x0000fee00000
00359292309i[CPU0  ] CPUID[0x00000000]: 00000005 756e6547 6c65746e 49656e69
00359292309i[CPU0  ] CPUID[0x00000001]: 00000633 00010800 00002028 1fcbfbff
00359292309i[CPU0  ] CPUID[0x00000002]: 00410601 00000000 00000000 00000000
00359292309i[CPU0  ] CPUID[0x00000003]: 00000000 00000000 00000000 00000000
00359292309i[CPU0  ] CPUID[0x00000004]: 00000000 00000000 00000000 00000000
00359292309i[CPU0  ] CPUID[0x00000005]: 00000040 00000040 00000003 00000020
00359292309i[CPU0  ] CPUID[0x80000000]: 80000008 00000000 00000000 00000000
00359292309i[CPU0  ] CPUID[0x80000001]: 00000000 00000000 00000101 2a100000
00359292309i[CPU0  ] CPUID[0x80000002]: 20202020 20202020 20202020 6e492020
00359292309i[CPU0  ] CPUID[0x80000003]: 286c6574 50202952 69746e65 52286d75
00359292309i[CPU0  ] CPUID[0x80000004]: 20342029 20555043 20202020 00202020
00359292309i[CPU0  ] CPUID[0x80000005]: 01ff01ff 01ff01ff 40020140 40020140
00359292309i[CPU0  ] CPUID[0x80000006]: 00000000 42004200 02008140 00000000
00359292309i[CPU0  ] CPUID[0x80000007]: 00000000 00000000 00000000 00000000
00359292309i[CPU0  ] CPUID[0x80000008]: 00003028 00000000 00000000 00000000
00359292309i[PLUGIN] reset of 'pci' plugin device by virtual method
00359292309i[PLUGIN] reset of 'pci2isa' plugin device by virtual method
00359292309i[PLUGIN] reset of 'cmos' plugin device by virtual method
00359292309i[PLUGIN] reset of 'dma' plugin device by virtual method
00359292309i[PLUGIN] reset of 'pic' plugin device by virtual method
00359292309i[PLUGIN] reset of 'pit' plugin device by virtual method
00359292309i[PLUGIN] reset of 'vga' plugin device by virtual method
00359292309i[PLUGIN] reset of 'floppy' plugin device by virtual method
00359292309i[PLUGIN] reset of 'acpi' plugin device by virtual method
00359292309i[PLUGIN] reset of 'ioapic' plugin device by virtual method
00359292309i[PLUGIN] reset of 'keyboard' plugin device by virtual method
00359292309i[PLUGIN] reset of 'harddrv' plugin device by virtual method
00359292309i[PLUGIN] reset of 'pci_ide' plugin device by virtual method
00359292309i[PLUGIN] reset of 'unmapped' plugin device by virtual method
00359292309i[PLUGIN] reset of 'biosdev' plugin device by virtual method
00359292309i[PLUGIN] reset of 'speaker' plugin device by virtual method
00359292309i[PLUGIN] reset of 'extfpuirq' plugin device by virtual method
00359292309i[PLUGIN] reset of 'parallel' plugin device by virtual method
00359292309i[PLUGIN] reset of 'serial' plugin device by virtual method
00359292309i[PLUGIN] reset of 'gameport' plugin device by virtual method
00359292309i[PLUGIN] reset of 'iodebug' plugin device by virtual method
00359292309i[PLUGIN] reset of 'usb_uhci' plugin device by virtual method
Next at t=359292310
(0) [0x0000fffffff0] f000:fff0 (unk. ctxt): jmpf 0xf000:e05b          ; ea5be000f0
<bochs:3> info tab
paging off

_________________
My OS:
https://github.com/fido2020/Lemon-OS


Top
 Profile  
 
 Post subject: Re: Out Of Memory error by GRUB when loading Higher Half ker
PostPosted: Sat Feb 10, 2018 9:32 pm 
Offline
Member
Member

Joined: Fri Aug 26, 2016 1:41 pm
Posts: 485
Paging is off because you have triple faulted and are back in real mode as a result. You need to step into your code to a point just after you set the video ram paging. The output (screenshot) suggests the possibility that you have done something wrong with the paging. I may be wrong, but that's my gut feel.


Top
 Profile  
 
 Post subject: Re: Out Of Memory error by GRUB when loading Higher Half ker
PostPosted: Sat Feb 10, 2018 10:19 pm 
Offline

Joined: Fri Sep 09, 2016 5:52 pm
Posts: 24
Location: Australia
Yes, i am quite sure it is a problem with the paging as it also happens with VGA Text Mode and right after i switch the page directory

_________________
My OS:
https://github.com/fido2020/Lemon-OS


Top
 Profile  
 
 Post subject: Re: Out Of Memory error by GRUB when loading Higher Half ker
PostPosted: Sun Feb 11, 2018 1:44 pm 
Offline
Member
Member

Joined: Fri Aug 26, 2016 1:41 pm
Posts: 485
For one thing you defined kernel_end in paging.cpp as uint32_t. If you take the address of it and then add 1 it will add 4 (the size of uint32_t). In your code you have (uint32_t)(&kernel_end-KERNEL_VIRTUAL_BASE). I think you want (((uint32_t)&kernel_end)-KERNEL_VIRTUAL_BASE).Cast the pointer to a non pointer uint32_t and then do math on that.


Top
 Profile  
 
 Post subject: Re: Out Of Memory error by GRUB when loading Higher Half ker
PostPosted: Wed Feb 14, 2018 10:44 pm 
Offline

Joined: Fri Sep 09, 2016 5:52 pm
Posts: 24
Location: Australia
Changing it has caused to go back to faulting

_________________
My OS:
https://github.com/fido2020/Lemon-OS


Top
 Profile  
 
 Post subject: Re: Out Of Memory error by GRUB when loading Higher Half ker
PostPosted: Thu Feb 15, 2018 12:18 am 
Offline
Member
Member

Joined: Fri Aug 26, 2016 1:41 pm
Posts: 485
I didn't say it is the only issue, but it is clearly one of your issues. Can tell you as it is now that one mistake will make your program fail. There may be others but that is an obvious problem to fix (you effectively added 0xC0000000 *4 (and was truncated) so it was the same as adding 0. You should use Bochs to step through the code to the point just before the failure. You can look at the registers and even the page table if necessary. Learning to use a debugger for this kind of thing is basically a necessary skill in doing development.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 14 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: Bing [Bot], MSN [Bot] and 7 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group