OSDev.org

The Place to Start for Operating System Developers
It is currently Thu Mar 28, 2024 8:19 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 9 posts ] 
Author Message
 Post subject: Passing data block from boot loader to the kernel
PostPosted: Thu Aug 25, 2022 9:23 pm 
Offline
Member
Member
User avatar

Joined: Fri Oct 27, 2006 9:42 am
Posts: 1925
Location: Athens, GA, USA
Just as a heads' up for those who have been helping me so far, I am now having an issue where the second stage loader is supposed to pass some data to the kernel, but the kernel isn't finding the data block at the memory address I expected it to be at.

The plan had been to place the data block on the stack, but I quickly realized that a) this conflicted with how I was intending page map the stack, b) it would leave the kernel with an extremely small usable stack, and c) passing the large amount of data in question would present a problem when defining the function signature for the kernel's main function. I may revisit that idea later, but for now, I am just passing the data as a block at the end of the first 64KiB of the kernel memory. A more complete solution would almost certainly involve a more elaborate setup for the page mapping.

Right now, I am - despite some misgivings on this - hard-coding the address I expect the data block to be at. I do define an enum representing the offsets into this block, but I am not certain if it is a correct reflection of the data block in memory.

The code in the kernel right now is:
Code:
/* kernel.c */
#include <stdint.h>
#include "terminal.h"
#include "mem_map.h"

#define KDATA_OFFSET 0xc000fffc

enum KData
{
    mmap_cnt = 0,
    mmap = mmap_cnt + 1 + (24 * 16),
    drive_id,
    fat = drive_id + 1 + (9 * (512 / 4))
};


void kernel_main()
{
    clear_screen();
    kprints("Starting Kernel...\n\n", CYAN, BLACK, 0);

    uint32_t* fs_data = (uint32_t *) KDATA_OFFSET;

    kprints("Passed Data offset ", WHITE, BLACK, 0);
    kprintu((uint32_t)(fs_data + 1), 16, WHITE, BLACK, 0);

    kprints("\nDrive ID: ", WHITE, BLACK, 0);
    kprintu(*(fs_data - drive_id), 16, WHITE, BLACK, 0);
    kprints("\nFAT offset ", WHITE, BLACK, 0);
    kprintu((uint32_t)(fs_data - drive_id - 1), 16, WHITE, BLACK, 0);

    kprints("\nMemory Map Table offset ", WHITE, BLACK, 0);
    kprintu((uint32_t)(fs_data - mmap_cnt - 1), 16, WHITE, BLACK, 0);
    print_mmap(*(fs_data - mmap_cnt), (struct memory_map_entry*)(fs_data - mmap));
}


The code which moves the data block to FFFF:FFFC (which should be mapped to 0xC000FFFC in virtual memory) is here:

Code:
kernel_base        equ 0xffff

kdata_offset       equ 0xfffc

struc KData
    .mmap_cnt      resd 1
    .mmap          resd High_Mem_Map_size * 16
    .drive         resd 1
    .fat           resd fat_size
endstruc


...

Code:
;; Attempt to get the full physical memory map for the system
;; this must be done before the move to protected mode
get_mem_maps:
        ; get the low memory map
        write low_mem
        int LMBIOS
        mov si, print_buffer
        call print_decimal_word
        write kbytes
        ; get the high memory map
        push es
        push si
        push di
        push bp
        mov ax, kernel_base
        mov es, ax
        mov di, (kdata_offset - KData.mmap - mem_map_buffer_size)
        mov si, (kdata_offset - KData.mmap_cnt)
        call get_hi_memory_map
        mov di, (kdata_offset - KData.mmap - mem_map_buffer_size)
        mov bp, es:[kdata_offset - KData.mmap_cnt]
        call print_hi_mem_map
        pop bp
        pop di
        pop si
        pop es

load_kernel_data:
        push ax
        push es
        mov ax, kernel_base
        mov es, ax
        mov dx, word [bp - stg2_parameters.drive]
        mov es:[kdata_offset - KData.drive], edx
        lea ax, es:[kdata_offset - KData.fat - fat_size]
        memcopy_rm ax, [bp - stg2_parameters.fat_0], fat_size
        pop es
        pop ax


The goal of the transfer code is to write first the number of entries in the memory map, followed by the memory map itself, followed by the disk ID, and then finally followed by the entire FAT for the floppy disk.

Right now, it isn't finding any data at that location at all - it's all zeroed memory.

_________________
Rev. First Speaker Schol-R-LEA;2 LCF ELF JAM POEE KoR KCO PPWMTF
Ordo OS Project
Lisp programmers tend to seem very odd to outsiders, just like anyone else who has had a religious experience they can't quite explain to others.


Top
 Profile  
 
 Post subject: Re: Passing data block from boot loader to the kernel
PostPosted: Thu Aug 25, 2022 9:50 pm 
Offline
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 5099
Schol-R-LEA wrote:
I do define an enum representing the offsets into this block, but I am not certain if it is a correct reflection of the data block in memory.

Maybe it is, maybe it isn't. But what I want to know is: why aren't you using a struct? It would make the pointer arithmetic easy:

Code:
struct kdata * data_block = ((struct kdata *)KDATA_OFFSET) - 1;


Schol-R-LEA wrote:
FFFF:FFFC (which should be mapped to 0xC000FFFC in virtual memory)

It can't be. Real mode address FFFF:FFFC is linear address 0x10FFEC.

Schol-R-LEA wrote:
Code:
        lea ax, es:[kdata_offset - KData.fat - fat_size]

What exactly is the purpose of the ES segment override on this instruction?


Top
 Profile  
 
 Post subject: Re: Passing data block from boot loader to the kernel
PostPosted: Thu Aug 25, 2022 10:13 pm 
Offline
Member
Member
User avatar

Joined: Fri Oct 27, 2006 9:42 am
Posts: 1925
Location: Athens, GA, USA
Octocontrabass wrote:
Schol-R-LEA wrote:
I do define an enum representing the offsets into this block, but I am not certain if it is a correct reflection of the data block in memory.

Maybe it is, maybe it isn't. But what I want to know is: why aren't you using a struct? It would make the pointer arithmetic easy:

Code:
struct kdata * data_block = ((struct kdata *)KDATA_OFFSET) - 1;


For some reason, I was thinking that it wouldn't be feasible but yes, that should be possible. Let me see about reworking it that way. EDIT: this should do:

Code:
struct kdata
{
    uint32_t mmap_cnt;
    struct memory_map_entry mem_table[16];
    uint32_t drive_id;
    uint8_t fat[9 * 512];
};


Octocontrabass wrote:
Schol-R-LEA wrote:
FFFF:FFFC (which should be mapped to 0xC000FFFC in virtual memory)

It can't be. Real mode address FFFF:FFFC is linear address 0x10FFEC.


I have the virtual addresses from 0xC0000000-0xC003FFFF mapped to physical addresses 0x00010000-0x0004FFFF, or vice versa. Or at least, I thought I did.

Octocontrabass wrote:
Schol-R-LEA wrote:
Code:
        lea ax, es:[kdata_offset - KData.fat - fat_size]

What exactly is the purpose of the ES segment override on this instruction?

That's... a good question. I think I was thrashing a bit when I added the segment marker there.

I may need to take a break again.

_________________
Rev. First Speaker Schol-R-LEA;2 LCF ELF JAM POEE KoR KCO PPWMTF
Ordo OS Project
Lisp programmers tend to seem very odd to outsiders, just like anyone else who has had a religious experience they can't quite explain to others.


Top
 Profile  
 
 Post subject: Re: Passing data block from boot loader to the kernel
PostPosted: Thu Aug 25, 2022 11:12 pm 
Offline
Member
Member
User avatar

Joined: Fri Oct 27, 2006 9:42 am
Posts: 1925
Location: Athens, GA, USA
Updated code (pushed to repo):

Code:
/* kernel.c */
#include <stdint.h>
#include "terminal.h"
#include "mem_map.h"

#define KDATA_OFFSET 0xc0010000

struct kdata
{
    uint8_t fat[9 * 512];
    uint32_t drive_id;
    struct memory_map_entry mem_table[16];
    uint32_t mmap_cnt;
};

void kernel_main()
{
    clear_screen();
    kprints("Starting Kernel...\n\n", CYAN, BLACK, 0);

    struct kdata* fs_data = (struct kdata *) (KDATA_OFFSET - sizeof(struct kdata));

    kprints("Passed Data offset ", WHITE, BLACK, 0);
    kprintu((uint32_t)(fs_data), 16, WHITE, BLACK, 0);

    kprints("\nDrive ID: ", WHITE, BLACK, 0);
    kprintu(fs_data->drive_id, 16, WHITE, BLACK, 0);
    kprints("\nFAT offset ", WHITE, BLACK, 0);
    kprintu((uint32_t) &fs_data->fat, 16, WHITE, BLACK, 0);

    kprints("\nMemory Map Table offset ", WHITE, BLACK, 0);
    kprintu((uint32_t) &fs_data->mem_table, 16, WHITE, BLACK, 0);
    print_mmap(fs_data->mmap_cnt, fs_data->mem_table);
}


Code:
kernel_base        equ 0xffff

struc KData
    .fat           resb fat_size
    .drive         resd 1
    .mmap          resb High_Mem_Map_size * 16
    .mmap_cnt      resd 1
endstruc

kdata_offset       equ 0xffff - KData_size


Code:
;; Attempt to get the full physical memory map for the system
;; this must be done before the move to protected mode
get_mem_maps:
        ; get the low memory map
        write low_mem
        int LMBIOS
        mov si, print_buffer
        call print_decimal_word
        write kbytes
        ; get the high memory map
        push es
        push si
        push di
        push bp
        mov ax, kernel_base
        mov es, ax
        mov di, (kdata_offset + KData.mmap)
        mov si, (kdata_offset + KData.mmap_cnt)
        call get_hi_memory_map
        mov di, (kdata_offset + KData.mmap)
        mov bp, es:[kdata_offset + KData.mmap_cnt]
        call print_hi_mem_map
        pop bp
        pop di
        pop si
        pop es

load_kernel_data:
        push ax
        push es
        mov ax, kernel_base
        mov es, ax
        zero(edx)
        mov dx, word [bp - stg2_parameters.drive]
        mov es:[kdata_offset + KData.drive], edx
        mov ax, kdata_offset + KData.fat
        memcopy_rm ax, [bp - stg2_parameters.fat_0], fat_size
        pop es
        pop ax

_________________
Rev. First Speaker Schol-R-LEA;2 LCF ELF JAM POEE KoR KCO PPWMTF
Ordo OS Project
Lisp programmers tend to seem very odd to outsiders, just like anyone else who has had a religious experience they can't quite explain to others.


Top
 Profile  
 
 Post subject: Re: Passing data block from boot loader to the kernel
PostPosted: Fri Aug 26, 2022 8:59 am 
Offline
Member
Member
User avatar

Joined: Fri Oct 27, 2006 9:42 am
Posts: 1925
Location: Athens, GA, USA
OK, so I figured it out way too early this morning (I was unable to sleep so I ended up obsessing over this).

While thrashing about, I struck on the idea of using the print_mmap() routine to probe the memory a bit. By forcing the count to 2, I got the following results:
Image

This showed that the data was in fact getting loaded, but not where I was anticipating it to be - the data shown was correct, but in the wrong location.

After a bit of experimentation and some more thrashing, I found that the data was offset from the area I expected it to be in by 16 bytes. Adding a negative offset to the code

Code:
    struct kdata* fs_data = (struct kdata *) (KDATA_OFFSET - sizeof(struct kdata) - 16);


get the data correctly:

Image

It took me a bit of thinking to realize that this was the 16 bits which are 'missing' in the 8086 High Memory Area.

I also noted that I am not loading non-constant initialized data correctly when I load the ELF file, but that is a problem I can address later - I am pretty sure that the wiki explains why this can happen, but I will put off reviewing that until I have the time.

So basically, as far as what is being displayed, this puts me where I was a few weeks ago with the second stage boot loader, but now I am in the kernel and can actually do something with that memory map.

I also noted that the data block in question is exactly 5000 bytes in size (0x1388 in hex) but that's not especially relevant right now.

_________________
Rev. First Speaker Schol-R-LEA;2 LCF ELF JAM POEE KoR KCO PPWMTF
Ordo OS Project
Lisp programmers tend to seem very odd to outsiders, just like anyone else who has had a religious experience they can't quite explain to others.


Top
 Profile  
 
 Post subject: Re: Passing data block from boot loader to the kernel
PostPosted: Fri Aug 26, 2022 11:36 am 
Offline
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 5099
Schol-R-LEA wrote:
I have the virtual addresses from 0xC0000000-0xC003FFFF mapped to physical addresses 0x00010000-0x0004FFFF, or vice versa. Or at least, I thought I did.

You do. Paging doesn't change the low 12 bits of the address, so it's impossible to map 0x10FFEC to 0xC000FFFC.


Top
 Profile  
 
 Post subject: Re: Passing data block from boot loader to the kernel
PostPosted: Fri Aug 26, 2022 3:04 pm 
Offline
Member
Member
User avatar

Joined: Fri Oct 27, 2006 9:42 am
Posts: 1925
Location: Athens, GA, USA
Ah, I misunderstood what Octocontrabass was saying. Yes, you are right.

Hence the 16 byte difference, then.

_________________
Rev. First Speaker Schol-R-LEA;2 LCF ELF JAM POEE KoR KCO PPWMTF
Ordo OS Project
Lisp programmers tend to seem very odd to outsiders, just like anyone else who has had a religious experience they can't quite explain to others.


Top
 Profile  
 
 Post subject: Re: Passing data block from boot loader to the kernel
PostPosted: Thu Sep 01, 2022 7:45 am 
Offline
Member
Member

Joined: Fri Feb 11, 2022 4:55 am
Posts: 435
Location: behind the keyboard
Since you are a beginner, I will give you how I do it.
My kernel is in the PE64 Format, so I create a section named "INITDATA" and allocate the initdata structure to it. Then the bootloader detects it and fills it. I got this idea when I was inspecting the Windows NT Kernel Executable Image "ntoskrnl.exe" which in Windows 11 is still at Version 10 (Windows 10 As I understand) :)

When you are planning to boot on UEFI and Legacy BIOS, these OFFSET Declaration would not work on all computers, because the Memory Map changes. So my kernel fixes that by being relocated by the bootloader to Conventional Memory, and all its structures are relocated. So never expect that Address 0x100000 is RAM on all PCs.

Then when it boots, my kernel relocates it self to the top of virtual memory (System Space) and continues the work.

Devc1,


Top
 Profile  
 
 Post subject: Re: Passing data block from boot loader to the kernel
PostPosted: Thu Sep 01, 2022 8:47 am 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 1593
devc1 wrote:
Since you are a beginner, I will give you how I do it.
My kernel is in the PE64 Format, so I create a section named "INITDATA" and allocate the initdata structure to it. Then the bootloader detects it and fills it.
Well, that is certainly one way, but requires absolute synchronicity between bootloader and kernel. Passing data on the boot stack affords more flexibility. And time has taught that flexibility will be needed. You will forget something, or need something different in the future.

devc1 wrote:
When you are planning to boot on UEFI and Legacy BIOS, these OFFSET Declaration would not work on all computers, because the Memory Map changes. So my kernel fixes that by being relocated by the bootloader to Conventional Memory, and all its structures are relocated. So never expect that Address 0x100000 is RAM on all PCs.
The kernel itself can abstract its position in physical memory by way of virtual memory. And the bootloader can be made responsible for setting up the page tables. As I have often stated before, my kernel does not care about its position in physical memory, and could in theory be in pieces strewn all across physical memory. Now, all bootloader adapters I have written load the kernel en bloc, but the kernel itself does not depend on that fact.

However, for a legacy BIOS bootloader, it is normal to expect the 1MB area to hold at least some memory. Linux will in fact only run from that address (on x86; this is an architectural detail). The kernel itself obviously should not make that same assumption, at least not in architecture-independent code.

It is a good idea to relax assumptions and check them if possible, but sometimes the architecture defines things for you. There being RAM at 1MB has been a defining feature of PCs ever since the PC-AT. No manufacturer can build a PC that works any differently without running headlong into problems. And there would be no point. There is no reason not to have RAM at 1MB. There would be more of a reason not to have an I/O memory hole at 640kB (e.g. if you are building a legacy-free system. System boot can happen from flash mapped to 0xffff0000, and BIOS can copy itself to 0x000f0000 one DRAM is set up. Indeed, x86 memory controllers have lock bits to prevent write access to the BIOS copy area for precisely such a feature)

_________________
Carpe diem!


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 9 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 53 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group