OSDev.org

The Place to Start for Operating System Developers
It is currently Thu Mar 28, 2024 11:17 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 15 posts ] 
Author Message
 Post subject: Need ideas on paging in long mode
PostPosted: Sun Dec 26, 2021 2:53 am 
Offline
Member
Member

Joined: Wed Oct 20, 2021 6:00 pm
Posts: 102
Location: Paraguay
Hi,
I need someone to consult, fix and comment on paging in long mode. I have implemented some functions here: https://github.com/NeonLightions/Amore-OS-x64, but it's quite confused. And can you please suggest me some resources or articles that are helpful in 4-level paging? Thank you very much!


Top
 Profile  
 
 Post subject: Re: Need ideas on paging in long mode
PostPosted: Sun Dec 26, 2021 4:11 am 
Online
Member
Member
User avatar

Joined: Sat Mar 31, 2012 3:07 am
Posts: 4591
Location: Chichester, UK
I would suggest that your question is a little too open ended.

You have implemented paging functions. So, do they work? If not, what errors are you getting? If you are getting errors, what steps have you taken towards debugging these errors?

As for resources - the AMD and Intel manuals provide detailed explantations of the paging mechanism.

In summary, what exactly is your problem?


Top
 Profile  
 
 Post subject: Re: Need ideas on paging in long mode
PostPosted: Sun Dec 26, 2021 4:56 am 
Offline
Member
Member

Joined: Wed Oct 20, 2021 6:00 pm
Posts: 102
Location: Paraguay
iansjack wrote:
I would suggest that your question is a little too open ended.

You have implemented paging functions. So, do they work? If not, what errors are you getting? If you are getting errors, what steps have you taken towards debugging these errors?

As for resources - the AMD and Intel manuals provide detailed explantations of the paging mechanism.

In summary, what exactly is your problem?

Hi, i got triple fault when enable paging. I have used debugger and realized that i got error at walking on pml4 to look for pdpt. I think the problem is my macros. Can you help me? Also is there any topic or thread that show how to walk with 4-level paging?


Top
 Profile  
 
 Post subject: Re: Need ideas on paging in long mode
PostPosted: Sun Dec 26, 2021 4:56 am 
Offline
Member
Member

Joined: Wed Oct 20, 2021 6:00 pm
Posts: 102
Location: Paraguay
iansjack wrote:
I would suggest that your question is a little too open ended.

You have implemented paging functions. So, do they work? If not, what errors are you getting? If you are getting errors, what steps have you taken towards debugging these errors?

As for resources - the AMD and Intel manuals provide detailed explantations of the paging mechanism.

In summary, what exactly is your problem?

Hi, i got triple fault when enable paging. I have used debugger and realized that i got error at walking on pml4 to look for pdpt. I think the problem is my macros. Can you help me? Also is there any topic or thread that show how to walk with 4-level paging?


Top
 Profile  
 
 Post subject: Re: Need ideas on paging in long mode
PostPosted: Sun Dec 26, 2021 5:07 am 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 1593
You are using code for 32-bit PAE paging. This will not work in 64-bit mode. In 32-bit PAE paging, there are indeed 32-byte page directory pointer tables, but in 64-bit mode, all layers of the paging tree are naturally aligned pages.

_________________
Carpe diem!


Top
 Profile  
 
 Post subject: Re: Need ideas on paging in long mode
PostPosted: Sun Dec 26, 2021 5:21 am 
Offline
Member
Member

Joined: Wed Oct 20, 2021 6:00 pm
Posts: 102
Location: Paraguay
nullplan wrote:
You are using code for 32-bit PAE paging. This will not work in 64-bit mode. In 32-bit PAE paging, there are indeed 32-byte page directory pointer tables, but in 64-bit mode, all layers of the paging tree are naturally aligned pages.

I see, but that is just temporary solution :D. Now things I need are: Is there anything wrong with my index calculator macros? If so, what should I need to fix? How to calculate index in 4-level paging?


Top
 Profile  
 
 Post subject: Re: Need ideas on paging in long mode
PostPosted: Sun Dec 26, 2021 6:29 am 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 1593
NeonLightions wrote:
I see, but that is just temporary solution
It is not any solution. The CPU will look at that misaligned address in CR3 and throw it back into your face in the form of a tripple fault. By setting CR3 to a misaligned address, or any address that is not a proper PML4, you make it impossible to resolve any addresses once paging is enabled. So the next instruction address cannot be resolved, nor can the address of the IDT, that of the GDT, of the TSS, or any stack in the TSS. That is why you get a tripple fault.

NeonLightions wrote:
Now things I need are: Is there anything wrong with my index calculator macros?
No, those are fine, albeit written weirdly. All those leading zeroes can be omitted, they serve absolutely no purpose, except to take up space and hinder readability.

There is ADDR_PAGE_ROUND_UP, which as written always returns 0. As used, I should think you meant
Code:
#define ADDR_PAGE_ROUND_UP(addr) (((addr) + 0xfff) & -0x1000)
/* or possibly: */
#define ADDR_PAGE_ROUND_UP(addr) ADDR_PAGE_ROUND_DN((addr) + 0xfff)


Besides, __paging_init() simply cannot work, because it is writing virtual addresses into paging structures and into CR3, but those all require physical addresses. Unless you are currently identity-mapped, you should not be doing that.

_________________
Carpe diem!


Top
 Profile  
 
 Post subject: Re: Need ideas on paging in long mode
PostPosted: Sun Dec 26, 2021 6:51 am 
Offline
Member
Member

Joined: Wed Oct 20, 2021 6:00 pm
Posts: 102
Location: Paraguay
nullplan wrote:
NeonLightions wrote:
I see, but that is just temporary solution
It is not any solution. The CPU will look at that misaligned address in CR3 and throw it back into your face in the form of a tripple fault. By setting CR3 to a misaligned address, or any address that is not a proper PML4, you make it impossible to resolve any addresses once paging is enabled. So the next instruction address cannot be resolved, nor can the address of the IDT, that of the GDT, of the TSS, or any stack in the TSS. That is why you get a tripple fault.

NeonLightions wrote:
Now things I need are: Is there anything wrong with my index calculator macros?
No, those are fine, albeit written weirdly. All those leading zeroes can be omitted, they serve absolutely no purpose, except to take up space and hinder readability.

There is ADDR_PAGE_ROUND_UP, which as written always returns 0. As used, I should think you meant
Code:
#define ADDR_PAGE_ROUND_UP(addr) (((addr) + 0xfff) & -0x1000)
/* or possibly: */
#define ADDR_PAGE_ROUND_UP(addr) ADDR_PAGE_ROUND_DN((addr) + 0xfff)


Besides, __paging_init() simply cannot work, because it is writing virtual addresses into paging structures and into CR3, but those all require physical addresses. Unless you are currently identity-mapped, you should not be doing that.

Okay, that solution has been removed (just not post to repo). Is there a way to make the calculation faster and neater? So, the address of those page structures are naturally aligned so I don't need to align it?


Top
 Profile  
 
 Post subject: Re: Need ideas on paging in long mode
PostPosted: Sun Dec 26, 2021 8:08 am 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 1593
NeonLightions wrote:
Is there a way to make the calculation faster and neater? So, the address of those page structures are naturally aligned so I don't need to align it?
I wouldn't worry about speed at this juncture. No, this is basically about design. I have a physical allocator that can by its very nature only return pages. Therefore I don't need to align the addresses that fall out of the allocator. But that means I have to make the general purpose allocator more complicated. I have a "kmalloc()" for allocating miscellaneous small things, but its memory pool can only grow, so for large things, directly calling the physical allocator is encouraged. Later on, I might experiment with demand paging, and then things get interesting.

_________________
Carpe diem!


Top
 Profile  
 
 Post subject: Re: Need ideas on paging in long mode
PostPosted: Mon Dec 27, 2021 6:54 am 
Offline
Member
Member

Joined: Wed Oct 20, 2021 6:00 pm
Posts: 102
Location: Paraguay
Hi,
I have looked through this tutorial: https://github.com/josehu07/hux-kernel/wiki/10.-Two-Level-Paging-Setup and I found this line:
Code:
/** Helper macro on getting the pointed-to address stored in an entry. */
#define ENTRY_FRAME_ADDR(entry) ((uint32_t) (entry).frame << 12)

What's that line mean?


Top
 Profile  
 
 Post subject: Re: Need ideas on paging in long mode
PostPosted: Mon Dec 27, 2021 11:15 am 
Offline
Member
Member

Joined: Tue Apr 03, 2018 2:44 am
Posts: 401
NeonLightions wrote:
Hi,
I have looked through this tutorial: https://github.com/josehu07/hux-kernel/wiki/10.-Two-Level-Paging-Setup and I found this line:
Code:
/** Helper macro on getting the pointed-to address stored in an entry. */
#define ENTRY_FRAME_ADDR(entry) ((uint32_t) (entry).frame << 12)

What's that line mean?


(entry).frame is the physical page number (though, not physical address). So the first physical page would be page number 0, the next page would be page number 1 etc.

To get the physical address, we shift the page number left by 12 bits, which leaves the lower 12 bits as 0, and therefore leaving the resulting address aligned on a 4096 byte boundary (page size).

So, for page number 1:

  • Page address in binary: 0000 0000 0000 0000 0001 0000 0000 0000
  • Page address in hex: 0x00001000


Top
 Profile  
 
 Post subject: Re: Need ideas on paging in long mode
PostPosted: Mon Dec 27, 2021 5:33 pm 
Offline
Member
Member

Joined: Wed Oct 20, 2021 6:00 pm
Posts: 102
Location: Paraguay
thewrongchristian wrote:
NeonLightions wrote:
Hi,
I have looked through this tutorial: https://github.com/josehu07/hux-kernel/wiki/10.-Two-Level-Paging-Setup and I found this line:
Code:
/** Helper macro on getting the pointed-to address stored in an entry. */
#define ENTRY_FRAME_ADDR(entry) ((uint32_t) (entry).frame << 12)

What's that line mean?


(entry).frame is the physical page number (though, not physical address). So the first physical page would be page number 0, the next page would be page number 1 etc.

To get the physical address, we shift the page number left by 12 bits, which leaves the lower 12 bits as 0, and therefore leaving the resulting address aligned on a 4096 byte boundary (page size).

So, for page number 1:

  • Page address in binary: 0000 0000 0000 0000 0001 0000 0000 0000
  • Page address in hex: 0x00001000

Oh, I get it. But will it work on 4-level paging? Cause that tutorial is for 2 level paging.


Top
 Profile  
 
 Post subject: Re: Need ideas on paging in long mode
PostPosted: Tue Dec 28, 2021 12:47 am 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 1593
NeonLightions wrote:
Oh, I get it. But will it work on 4-level paging? Cause that tutorial is for 2 level paging.
And this is where you need to know the big difference between non-PAE and PAE paging: The size of the page table entries. In non-PAE (2-level) paging, the entries are 32 bits, and in PAE and 4-level paging, they are 64 bits. So if you defined your page table entry type correctly, the macro would almost work, except that the result would have to be a 64-bit number, not a 32-bit one.

I don't use bit fields. Ever. It has so far never been worthwhile for me to do so. In case of page table entries, each entry simply contains the physical address of either the next layer or the page itself, with some flags ORed in. This avoids the issue of shifting just to get back to where you were.

_________________
Carpe diem!


Top
 Profile  
 
 Post subject: Re: Need ideas on paging in long mode
PostPosted: Tue Dec 28, 2021 4:23 am 
Offline
Member
Member

Joined: Wed Oct 20, 2021 6:00 pm
Posts: 102
Location: Paraguay
Ok, just 1 more question: In Intel 64 and IA-32 Architectures Software Developer's Manual, Volume 3, Chapter 2, Section 2.1.5.1, it said:
Code:
With 4-level paging, there is only one PML4 table and its base physical address is stored in CR3
. Is it said only 1 PML4 table per process or the entire OS use only 1 PML4 table?


Top
 Profile  
 
 Post subject: Re: Need ideas on paging in long mode
PostPosted: Tue Dec 28, 2021 5:44 am 
Online
Member
Member
User avatar

Joined: Sat Mar 31, 2012 3:07 am
Posts: 4591
Location: Chichester, UK
One per process.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 15 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: Google [Bot] and 77 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group