Memory Heap Value Seems Incorrect

TheGameMaker90 · **Joined:** Thu Jan 07, 2021 2:01 pm **Posts:** 67

So for this one, I am trying to get started with modifying the heap I have in place for a proprietary heap implementation. My problem is that I seem to be getting the wrong address back from my initialization code.

Example:

Code:

uint32_t *test = (uint32_t *)0xC010000;
printf("Address: 0x%X.\n", &test);

Output:
Address: 0x10DC48

...

I believe I pointed the address to 0xC0100000, as indicated as the start address of a higher half kernel. Is this the intended result or am I missing something here?

iansjack · **Posted:** Sun Jan 17, 2021 9:27 am

Did you mean to print the address of the variable test rather than the pointer value that you initialized it with? What you print will be wherever on the stack the variable is located. What is the address range of your stack?

TheGameMaker90 · **Joined:** Thu Jan 07, 2021 2:01 pm **Posts:** 67

iansjack wrote:

Did you mean to print the address of the variable test rather than the pointer value that you initialized it with? What you print will be wherever on the stack the variable is located. What is the address range of your stack?

Goodness, I'm still figuring all of this stuff out. Do you mean in the bootloader? Not sure. It's very simple but the bootloader stack is defined like this:

Code:

.section .bss
.align 16
stack_bottom:
     .skip 16384
stack_top:

Also I'm using GRUB, and in the linker it starts at the . = 1M; (So 1MB) boundary. Also all sections are set using BLOCK(4K) : ALIGN(4K). (in the linker that is.)

Hope all of that is useful to help me.

foliagecanine · **Joined:** Sun Aug 23, 2020 4:35 pm **Posts:** 148

I believe this is the intended result.

Code:

uint32_t *test = (uint32_t *)0xC010000;

initializes a new 32-bit variable (likely located somewhere in the data segment, which makes sense with the address of 0x10DC48) with the value 0xC010000. Pointers are just like any other variable stored in memory. The only difference is how they are used in the programming language.

Code:

printf("Address: 0x%X.\n", &test);

takes the address of the new 32-bit variable (test) and prints it out. It happens to be placed at 0x10DC48 by the compiler.

If you use

Code:

printf("Address: 0x%X.\n", test);

you will find that it prints out "Address: 0xC010000"

Although it will complain about implicitly casting from a pointer to an integer unless you do something like this:

Code:

printf("Address: 0x%X.\n",(uint32_t)(intptr_t)test);

------------------------------------------

TheGameMaker90 wrote:

Goodness, I'm still figuring all of this stuff out. Do you mean in the bootloader? Not sure. It's very simple but the bootloader stack is defined like this:

Code:

.section .bss
.align 16
stack_bottom:
     .skip 16384
stack_top:

No, that is not the bootloader. That code is just the start code that first runs when the bootloader (GRUB) loads your OS. However, that (along with your linker) does show where your stack is.

TheGameMaker90 · **Joined:** Thu Jan 07, 2021 2:01 pm **Posts:** 67

foliagecanine wrote:

I believe this is the intended result.

Code:

uint32_t *test = (uint32_t *)0xC010000;

initializes a new 32-bit variable (likely located somewhere in the data segment, which makes sense with the address of 0x10DC48) with the value 0xC010000. Pointers are just like any other variable stored in memory. The only difference is how they are used in the programming language.

Code:

printf("Address: 0x%X.\n", &test);

takes the address of the new 32-bit variable (test) and prints it out. It happens to be placed at 0x10DC48 by the compiler.

If you use

Code:

printf("Address: 0x%X.\n", test);

you will find that it prints out "Address: 0xC010000"

Although it will complain about implicitly casting from a pointer to an integer unless you do something like this:

Code:

printf("Address: 0x%X.\n",(uint32_t)(intptr_t)test);

------------------------------------------

TheGameMaker90 wrote:

Goodness, I'm still figuring all of this stuff out. Do you mean in the bootloader? Not sure. It's very simple but the bootloader stack is defined like this:

Code:

.section .bss
.align 16
stack_bottom:
     .skip 16384
stack_top:

No, that is not the bootloader. That code is just the start code that first runs when the bootloader (GRUB) loads your OS. However, that (along with your linker) does show where your stack is.

Oh if only it were as simple as that... If you can believe it, I get a completely different result. No such complaint and it prints out 0x3FF00000 now. That is without the address of (&) operator. At least I now know why I was getting a page fault at that address before. I suppose though, I should remove the current heap implementation and paging first though...

Aaaaand... No such luck. I thought that it might have been a virtual address location designated by the current heap implementation. I can't understand why it would be so off...

foliagecanine · **Joined:** Sun Aug 23, 2020 4:35 pm **Posts:** 148

If you could supply your code (i.e. on GitHub) for us to look at, that would be helpful for diagnosing the problem.
Specifically, if you are willing, please post the linker script and the file in which this code is running.

It may be other code around it or the linker causing the problem, since compiling

Code:

#include <stdio.h>
#include <stdint.h>

int main() {
    uint32_t *test = (uint32_t *)0xC010000;
    printf("Address: 0x%X.\n",(uint32_t)(intptr_t)test);
    return 0;
}

with a C compiler and run it, it outputs "Address: 0xC010000."

Make sure you aren't modifying the value of test anywhere else as well.

TheGameMaker90 · **Joined:** Thu Jan 07, 2021 2:01 pm **Posts:** 67

foliagecanine wrote:

If you could supply your code (i.e. on GitHub) for us to look at, that would be helpful for diagnosing the problem.
Specifically, if you are willing, please post the linker script and the file in which this code is running.

It may be other code around it or the linker causing the problem, since compiling

Code:

#include <stdio.h>
#include <stdint.h>

int main() {
    uint32_t *test = (uint32_t *)0xC010000;
    printf("Address: 0x%X.\n",(uint32_t)(intptr_t)test);
    return 0;
}

with a C compiler and run it, it outputs "Address: 0xC010000."

Make sure you aren't modifying the value of test anywhere else as well.

I wish I could post the link, but it's on a private repo for a reason... It's privileged code only for my company Psionix Softworks, despite the fact that I work "alone" on it. I can provide snippets here and there, but most things are renamed to protect actual source base. I have created a function called "test_code" for the purposes of testing before implementation. So no, it isn't used elsewhere. Here's my linker (temporary, with unimportant things left out/omitted):

Code:

...
. = 1M;
.text BLOCK(4K) : ALIGN(4K)
{
    *(.multiboot)
    *(.text)
}
.data BLOCK(4K) : ALIGN(4K)
{
     *(.data)
}
.rodata BLOCK(4K) : ALIGN(4K)
{
    *(.rodata)
}
.bss BLOCK(4K) : ALIGN(4K)
{
    *(COMMON)
    *(.bss)
}
kernel_end = .;
...

As for the code that the address value is assigned, it's literally after the initialization of the other modules with my memory module and paging code commented off as to not interfere.

Paraphrased in my main module:

Code:

<kernel-type> kmain(void)
{
    ...
    test_code();
}

static inline void test_code(void)
{
    uint32_t *test = (uint32_t *)0xC0100000;
    printf("Address: 0x%X.\n", test); // Also tried "&test"
}

Literal Output:
...
Address: 0x3FF00000. // Without '&'

And:
...
Address: 0x10DC4C. // With '&'

Nothing too special really. This code was tested with minimal modules to avoid conflicts.
Also I noticed tou missed a zero. It's a uint32_t, as such, it should have 8 digits in the hex value: 0xC010 0000.

Edit:
I feel that I should also mention that this is in an attempt to replace my current (working) heap. I borrowed it from elsewhere and now want to replace it with my own code. As such, I am in the process of building my heap from the ground up. So this location in memory should not be influenced by other code since it is most definitely independent. And in Visual Studio 2017 I got the same result as you foliagecanine. I just can't understand why all of a sudden the addresses are all so far off. Let me try something else...

Edit2:
No luck. I thought that if I pulled the address of an array index it would work. Yesterday I was using smaller values like 0x4000 and it was working. Now it seems to not work. I can't remember if it was because I removed the old heap or left it in, but for some reason it doesn't seem to want to work right...

Okay so I tested with just 0xC010, could it be a "number too large" case? I feel like that's what it is. test[0] = 0xC010, test[1] = C014, which makes sense if it's aligned by 4 bytes. How would I fix that if that's what it is?

Edit3:
Okay so here's something interesting. I decided to test a different way...

Code:

if (&test[0] == 0xC0100000)
    printf("&test = 0xC0100000");

At which point it printed:
&test = 0xC0100000

Perhaps it's the way large numbers are being represented in hexadecimal. Maybe indeed...
I just tried something:

Code:

printf("%d\n", 0xFFFFFFFF);

And it prints: -1...
Hmm. That can't be normal...

Figured it out actually... It wasn't that the address was misplaced, it was how I printed hex values to the screen in my print function. Now it prints the correct address. However, without the array index it still prints a 6 digit hex value starting at 10XXXX. Is that normal?
Also, would it be a bad idea to consider starting at 0x7FFFFFFF?

Octocontrabass · **Joined:** Mon Mar 25, 2013 7:01 pm **Posts:** 5137

TheGameMaker90 wrote:

However, without the array index it still prints a 6 digit hex value starting at 10XXXX. Is that normal?

According to your linker script, yes.

TheGameMaker90 wrote:

Also, would it be a bad idea to consider starting at 0x7FFFFFFF?

The typical higher-half example places the kernel's code and data in the 0xC0000000-0xFFFFFFFF range to make page table management easier, since that address range is covered by a single PDPTE. If you think reserving a different portion of the address space for the kernel makes more sense, you can do that instead.

TheGameMaker90 · **Joined:** Thu Jan 07, 2021 2:01 pm **Posts:** 67

Octocontrabass wrote:

TheGameMaker90 wrote:

However, without the array index it still prints a 6 digit hex value starting at 10XXXX. Is that normal?

According to your linker script, yes.

TheGameMaker90 wrote:

Also, would it be a bad idea to consider starting at 0x7FFFFFFF?

The typical higher-half example places the kernel's code and data in the 0xC0000000-0xFFFFFFFF range to make page table management easier, since that address range is covered by a single PDPTE. If you think reserving a different portion of the address space for the kernel makes more sense, you can do that instead.

Fair enough, but what part of my linker script dictates that.

And That makes sense. Here's something I've also been wondering: Is the kernel heap itself supposed to be the max size of RAM? Like I have 16GB of RAM in my (custom built) computer and was hoping to be able to allocate up to that 16GB for later use if possible. Perhaps even when I finally get to UserLand.
If not, then how (besides in 16 bit real mode since I'm using GRUB as a bootloader) would I detect how much RAM my computer has. Or is it already stored somewhere I can pull it from. Is it something I should worry about now or later?

Octocontrabass · **Joined:** Mon Mar 25, 2013 7:01 pm **Posts:** 5137

TheGameMaker90 wrote:

Is the kernel heap itself supposed to be the max size of RAM?

Typically, no. The kernel heap is usually built on top of another allocator specifically for physical memory in page-sized units (sometimes called "physical memory manager" or "PMM"). The wiki has several pages on memory management.

TheGameMaker90 wrote:

If not, then how (besides in 16 bit real mode since I'm using GRUB as a bootloader) would I detect how much RAM my computer has. Or is it already stored somewhere I can pull it from.

You're using GRUB, which means you're using Multiboot or Multiboot2, which means GRUB has prepared a memory map for you. The Multiboot and Multiboot2 specifications explain how to get the memory map.

TheGameMaker90 wrote:

Is it something I should worry about now or later?

You can't allocate memory if you don't know where memory is.

TheGameMaker90 · **Joined:** Thu Jan 07, 2021 2:01 pm **Posts:** 67

Octocontrabass wrote:

TheGameMaker90 wrote:

Is the kernel heap itself supposed to be the max size of RAM?

Typically, no. The kernel heap is usually built on top of another allocator specifically for physical memory in page-sized units (sometimes called "physical memory manager" or "PMM"). The wiki has several pages on memory management.

TheGameMaker90 wrote:

If not, then how (besides in 16 bit real mode since I'm using GRUB as a bootloader) would I detect how much RAM my computer has. Or is it already stored somewhere I can pull it from.

You're using GRUB, which means you're using Multiboot or Multiboot2, which means GRUB has prepared a memory map for you. The Multiboot and Multiboot2 specifications explain how to get the memory map.

TheGameMaker90 wrote:

Is it something I should worry about now or later?

You can't allocate memory if you don't know where memory is.

I've been on that page. As we speak I'm looking at "Pancake's" examples on heap implementations. Hopefully after enough studying I'll be able to do it the way I have in mind.

Okay, I'll have to look into that. In truth, GRUB is (hopefully) temporary, and I hope to write my own bootloader at some point. I've already created a separate project (literally called OtherOS) for the purpose of testing it. It works,m but has flaws and is missing a good chunk of things I know Grub has. Putting that aside, I will definitely check it out. Perhaps you could provide a link??

Haha, this is true. I kind of felt stupid asking this one, but I like to have a thorough understanding of things. Looks like I have a bit of homework to do...

Oh and one other thing: is it possible (in the off chance it's needed) to go beyond the address 0xFFFFFFFF? I know that's the max unsigned integer value, but perhaps virtual memory could be mapped beyond that point?

Octocontrabass · **Joined:** Mon Mar 25, 2013 7:01 pm **Posts:** 5137

TheGameMaker90 wrote:

Perhaps you could provide a link??

Here is the Multiboot specification and here is the Multiboot2 specification. Without seeing your code, I can't tell you which one you're using right now.

TheGameMaker90 wrote:

Oh and one other thing: is it possible (in the off chance it's needed) to go beyond the address 0xFFFFFFFF? I know that's the max unsigned integer value, but perhaps virtual memory could be mapped beyond that point?

In protected mode, virtual addresses have 32 bits. In long mode, virtual addresses have 64 bits, but only 48 of those bits are usable on current CPUs.

In both modes, physical addresses may have up to 52 bits. In protected mode, you must use PAE to access all 52 bits.

TheGameMaker90 · **Joined:** Thu Jan 07, 2021 2:01 pm **Posts:** 67

Octocontrabass wrote:

TheGameMaker90 wrote:

Perhaps you could provide a link??

Here is the Multiboot specification and here is the Multiboot2 specification. Without seeing your code, I can't tell you which one you're using right now.

TheGameMaker90 wrote:

Oh and one other thing: is it possible (in the off chance it's needed) to go beyond the address 0xFFFFFFFF? I know that's the max unsigned integer value, but perhaps virtual memory could be mapped beyond that point?

In protected mode, virtual addresses have 32 bits. In long mode, virtual addresses have 64 bits, but only 48 of those bits are usable on current CPUs.

In both modes, physical addresses may have up to 52 bits. In protected mode, you must use PAE to access all 52 bits.

Sort of makes sense, but how can you have 64 bits on a 32 bit OS?

iansjack · **Posted:** Tue Jan 19, 2021 1:05 pm

Have you read the Intel Programmer's Manual? If this is a project you are doing for your employer then you really need to do your homework. Learn how to program in C, in particular be sure you fully understand pointers, and learn about the hardware you are working with.

TheGameMaker90 · **Joined:** Thu Jan 07, 2021 2:01 pm **Posts:** 67

iansjack wrote:

Have you read the Intel Programmer's Manual? If this is a project you are doing for your employer then you really need to do your homework. Learn how to program in C, in particular be sure you fully understand pointers, and learn about the hardware you are working with.

No I haven't in detail. And employer?? I believe I mentioned "my company." I'm the CEO and this is (until it gets into a usable state) a personal project.

OSDev.org

Memory Heap Value Seems Incorrect

Who is online