OSDev.org

The Place to Start for Operating System Developers
It is currently Thu Mar 28, 2024 3:55 pm

All times are UTC - 6 hours




Post new topic Reply to topic  [ 10 posts ] 
Author Message
 Post subject: How can you map more than 256 TiB in long mode.
PostPosted: Sun Sep 05, 2021 1:31 pm 
Offline
Member
Member

Joined: Sun May 24, 2020 9:11 am
Posts: 61
Location: /dev/null
I'm switching to BOOTBOOT. For a long time I developed my bottom half kernel but now I want to try a higher half kernel.

The problem is that it's code segment for example is mapped to 0xFFFFFFFFFFE02000. However, using 4-level paging you can only map up to 256 TiB (max. range 0x1000000000000) of memory!

For now I use this function to get physical address of some page:

Code:
intptr_t vm_get_physaddr(struct vm_space *vm_space, void *virtualaddr)
{
   uint64_t pml4 = vm_space->pml4;
   uint64_t ptindex = (uint64_t) virtualaddr / 0x1000 % 512;
   uint64_t pdindex = ((uint64_t) virtualaddr / 0x200000) % 512;
   uint64_t pdpindex = ((uint64_t) virtualaddr / 0x40000000) % 512;
   uint64_t pml4index = ((uint64_t) virtualaddr / 0x8000000000);
   uint64_t *entry = (uint64_t*) pml4;

   if (!(entry[pml4index] & 1))
      return -1;
   entry = (uint64_t*) (entry[pml4index] & ~0xFFF);

   if (!(entry[pdpindex] & 1))
      return -1;
   entry = (uint64_t*) (entry[pdpindex] & ~0xFFF);

   if (!(entry[pdindex] & 1))
      return -1;
   entry = (uint64_t*) (entry[pdindex] & ~0xFFF);

   if (!(entry[ptindex] & 1))
      return -1;
   entry = (uint64_t*) (entry[ptindex] & ~0xFFF);

   return entry;
}


I was able to successfully get address of first 2GiB (or more if available) of memory which are identity mapped in BOOTBOOT protocol. However, obviously this method did not work with higher addresses. I wonder how they were even mapped.


Top
 Profile  
 
 Post subject: Re: How can you map more than 256 TiB in long mode.
PostPosted: Sun Sep 05, 2021 2:01 pm 
Offline
Member
Member
User avatar

Joined: Sun Apr 30, 2017 12:16 pm
Posts: 68
Location: Poland
The 48-bit address is sign extended into 64-bits, so bits 63-48 are supposed to have the same value as bit 47, which turns 0x800000000000 into 0xFFFF800000000000, and so on. If you try accessing an address where bits 63-48 don't have the same value as bit 47, you'll get a general protection fault with error code 0.

_________________
Working on managarm.


Top
 Profile  
 
 Post subject: Re: How can you map more than 256 TiB in long mode.
PostPosted: Sun Sep 05, 2021 2:50 pm 
Offline
Member
Member
User avatar

Joined: Sat Mar 31, 2012 3:07 am
Posts: 4591
Location: Chichester, UK
Check out "canonical address" in the Programmer's Manual.


Top
 Profile  
 
 Post subject: Re: How can you map more than 256 TiB in long mode.
PostPosted: Sun Sep 05, 2021 2:55 pm 
Offline
Member
Member

Joined: Sun May 24, 2020 9:11 am
Posts: 61
Location: /dev/null
I'm not sure how exactly this works... I can't access this same address in non-sign-extended form.

Code:
(gdb) x/b 0xffffffffffe07d40
0xffffffffffe07d40 <buf>:   0x57
(gdb) x/b 0xffffffe07d40
0xffffffe07d40:   Cannot access memory at address 0xffffffe07d40
(gdb)


Top
 Profile  
 
 Post subject: Re: How can you map more than 256 TiB in long mode.
PostPosted: Sun Sep 05, 2021 9:29 pm 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 1593
antoni wrote:
I'm not sure how exactly this works... I can't access this same address in non-sign-extended form.
Yes, because those would be non-canonical. The CPU actually checks the sign extension and generates #GP(0) if it is wrong.

_________________
Carpe diem!


Top
 Profile  
 
 Post subject: Re: How can you map more than 256 TiB in long mode.
PostPosted: Mon Sep 06, 2021 3:05 am 
Offline
Member
Member
User avatar

Joined: Sat Mar 31, 2012 3:07 am
Posts: 4591
Location: Chichester, UK
0xffffffe07d40 = 0000 0000 0000 0000 1111 1111 1111 1111 1111 1111 1110 0000 0111 1101 0010 0000 in binary.

Bit 47 is 1. This is not extended to bits 48 - 63, which are all 0, so the address is not canonical, and raises an error when you attempt to access it.

0xffffffffffe07d40 = 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1110 0000 0111 1101 0010 0000 in binary.

Bit 47 is 1. This is extended to bits 48 - 63, which are all 1, so the address is canonical and can be accessed without an error.


Top
 Profile  
 
 Post subject: Re: How can you map more than 256 TiB in long mode.
PostPosted: Mon Sep 06, 2021 2:58 pm 
Offline
Member
Member

Joined: Sun Jun 23, 2019 5:36 pm
Posts: 618
Location: North Dakota, United States
If you want to map 256-Tbytes, check for five-level paging and then set CR4.LA57. Then set up your PML5 table and fill it with 512 PML5Es.Then canonicalize your addresses by sign-extending bit 56 to bits 57-64 (if memory serves).


Top
 Profile  
 
 Post subject: Re: How can you map more than 256 TiB in long mode.
PostPosted: Tue Sep 07, 2021 1:55 am 
Offline
Member
Member
User avatar

Joined: Sat Mar 31, 2012 3:07 am
Posts: 4591
Location: Chichester, UK
Restricting your OS to the latest Xeon processors is probably not a good move for a hobby OS.


Top
 Profile  
 
 Post subject: Re: How can you map more than 256 TiB in long mode.
PostPosted: Tue Sep 07, 2021 3:21 am 
Offline
Member
Member

Joined: Wed Mar 30, 2011 12:31 am
Posts: 676
It's also not what the OP is asking for...

Quote:
I was able to successfully get address of first 2GiB (or more if available) of memory which are identity mapped in BOOTBOOT protocol. However, obviously this method did not work with higher addresses. I wonder how they were even mapped.


You need to walk the page tables, just like the MMU does, to translate a virtual address to a physical address.

_________________
toaruos on github | toaruos.org | gitlab | twitter | bim - a text editor


Top
 Profile  
 
 Post subject: Re: How can you map more than 256 TiB in long mode.
PostPosted: Tue Sep 07, 2021 2:28 pm 
Offline
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 5100
antoni wrote:
The problem is that it's code segment for example is mapped to 0xFFFFFFFFFFE02000. However, using 4-level paging you can only map up to 256 TiB (max. range 0x1000000000000) of memory!

You can map 256 TiB total, but it's split into two 128 TiB pieces. The lower 128TiB is addresses from 0 to 0x00007FFFFFFFFFFF, and the upper 128 TiB is addresses from 0xFFFF800000000000 to 0xFFFFFFFFFFFFFFFF. Your example address 0xFFFFFFFFFFE02000 is in the upper 128 TiB.

With 4-level paging, you can't map anything from 0x0000800000000000 to 0xFFFF7FFFFFFFFFFF. Addresses in that range will always be invalid.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 10 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: Bing [Bot], Google [Bot], Majestic-12 [Bot] and 61 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group