OSDev.org

The Place to Start for Operating System Developers
It is currently Fri Jan 28, 2022 2:01 pm

All times are UTC - 6 hours




Post new topic Reply to topic  [ 20 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: My OS gets stuck switching tasks
PostPosted: Thu Nov 25, 2021 7:33 pm 
Offline

Joined: Thu Nov 25, 2021 7:11 pm
Posts: 17
I have a physical and virtual memory allocator set up, and now I'm working on getting multitasking to function, (co-operative for now) but there's a problem. The assembly code on the wiki that saves state to one location and loads from another isn't working. It gets stuck somewhere in that function, and I'm not quite sure why.

Does QEMU have an instruction stepped debugger?


Top
 Profile  
 
 Post subject: Re: My OS gets stuck switching tasks
PostPosted: Thu Nov 25, 2021 11:21 pm 
Online
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 3590
You can connect GDB to QEMU for instruction-level debugging.


Top
 Profile  
 
 Post subject: Re: My OS gets stuck switching tasks
PostPosted: Fri Nov 26, 2021 5:27 pm 
Offline

Joined: Thu Nov 25, 2021 7:11 pm
Posts: 17
Cool, now I've been able to find all sorts of issues unrelated to just the task switching.
Oh dear...
  • My physical memory allocator is seemingly working fine
  • My virtual memory allocator is doing all sorts of wrong things I don't understand
  • I have the first 4MB of memory identity paged, and while it seems like sharing the page table responsible between all new page directories should work, something is going horribly wrong.
  • When I peek at the first and last pairs of PDEs, it always gives me the same wrong values of B9660BC0 001800B0 392F3332 00FC0039
well, here's the current code if anyone wants to see how messy it has gotten


Top
 Profile  
 
 Post subject: Re: My OS gets stuck switching tasks
PostPosted: Fri Nov 26, 2021 10:03 pm 
Offline

Joined: Thu Nov 25, 2021 7:11 pm
Posts: 17
Good news; I found the problem. It seems that using PDEs directly just doesn't work properly. Allocating a new page table to identity map the first 4MB again seems to work.
Bad news; the cycling through tasks isn't quite working right. I'll figure that out tomorrow though.


Top
 Profile  
 
 Post subject: Re: My OS gets stuck switching tasks
PostPosted: Sat Nov 27, 2021 3:44 pm 
Offline

Joined: Thu Nov 25, 2021 7:11 pm
Posts: 17
It appears the issue is stack pointer related. I tried to fix it and now I'm getting Debug exceptions, and occasionally QEMU just outright crashed.
I would push the current code to github but it's down now? (it's actually up, just very slow somehow)


Top
 Profile  
 
 Post subject: Re: My OS gets stuck switching tasks
PostPosted: Sat Nov 27, 2021 6:14 pm 
Online
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 3590
sed4906h wrote:
It seems that using PDEs directly just doesn't work properly.

Are you trying to use 4MiB pages without setting CR4.PSE?

sed4906h wrote:
Allocating a new page table to identity map the first 4MB again seems to work.

You probably shouldn't use large pages for that anyway. Using a large page to map a range of addresses with multiple effective memory types is undefined behavior.


Top
 Profile  
 
 Post subject: Re: My OS gets stuck switching tasks
PostPosted: Sat Nov 27, 2021 7:30 pm 
Offline

Joined: Thu Nov 25, 2021 7:11 pm
Posts: 17
I'm just using 4KB pages.
My approach to creating new tasks involves temporarily switching into the new task's page directory (which should have been properly set up beforehand) so that some virtual memory can be allocated for where the stack pointer should go, and then switching back to the previous page directory.


Top
 Profile  
 
 Post subject: Re: My OS gets stuck switching tasks
PostPosted: Sun Nov 28, 2021 3:39 pm 
Offline

Joined: Thu Nov 25, 2021 7:11 pm
Posts: 17
Okay, I'm really stumped here. I tried to change my method for adding new page tables when needed to one that involves handling page faults, but it seems that I'm not getting page faults to occur, even on purpose.
When I create a page directory, I set every entry I haven't allocated memory in to 0, and set the last entry to point to the page directory itself.
My code that maps a virtual page to a physical page currently looks like this.
Code:
uint32_t *const pt = (uint32_t *)0xFFC00000;

/* There's stuff in between */

uint32_t *kmmap(uint32_t *vaddr, uint32_t *paddr) {
    uint32_t ptindex = (uint32_t)vaddr >> 12;

    if(pt[ptindex] & 1) {
        return 0;
    }

    pt[ptindex] = (uint32_t)paddr & ~0xFFF;
    pt[ptindex] |= 3;

    return vaddr;
}

Ideally, this should cause a page fault if the page table to put the mapping in doesn't exist yet. I would handle it by adding the entry before continuing.

But I don't get page faults from this. I just get garbage data. How did I manage to mess up recursive paging?


Top
 Profile  
 
 Post subject: Re: My OS gets stuck switching tasks
PostPosted: Sun Nov 28, 2021 11:24 pm 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 1168
Well, that code is definitely false. You neglect to separate the "ptindex" further into the page directory index and the page table index. Even in the oldest version of paging, you have two levels to contend with. Your code makes me presume you are working with 32-bit non-PAE paging using the recursive paging trick on page directory 1023. In that case, you need something more like this:

Code:
#define PGD_RECURSE 1023
#define MAKE_PGD_ADDR(pgi, pti) ((uint32_t*)(PGD_RECURSE << 22 | pgi << 12 | pti << 2))
int kmap(void *vaddr, uint32_t paddr) {
  uint32_t va = (uint32_t)vaddr;
  int pgi = va >> 22;
  int pti = (va >> 12) & 0x3ff;
  uint32_t *pgde = MAKE_PGD_ADDR(PGD_RECURSE, pgi);
  if (!(*pgde & 1))
  {
    uint32_t pt = phys_zalloc(); /* or whatever your physical allocator is called. Page needs to be zeroed out. */
    if (!pt) return -ENOMEM;
    *pgde = pt | 3;
  }
  uint32_t *pte = MAKE_PGD_ADDR(pgi, pti);
  int invalidate = *pte & 1;
  *pte = (paddr & 0xfffff000) | 3;
  if (invalidate)
    asm("invlpg %0" : : "r"(vaddr) : "memory");
  return 0;
}
So you first must ensure you have a page table for the given region in the page directory, and this is a step that can fail if you have no memory left. Then after that you can set the page table.

For PAE paging, the principle is the same, but it is one more layer, and for 64-bit paging it is yet another. Also, the entries are 64-bit, and you only use 9 index bits per level.

_________________
Carpe diem!


Top
 Profile  
 
 Post subject: Re: My OS gets stuck switching tasks
PostPosted: Mon Nov 29, 2021 9:00 am 
Offline

Joined: Thu Nov 25, 2021 7:11 pm
Posts: 17
Using your code (which is similar to my previous code), the check for whether a page table exists always returns true and tries to make a new page table. What? Another related question... How should I approach zeroing out the new page table before mapping it?


Top
 Profile  
 
 Post subject: Re: My OS gets stuck switching tasks
PostPosted: Mon Nov 29, 2021 10:54 am 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 1168
sed4906h wrote:
Using your code (which is similar to my previous code), the check for whether a page table exists always returns true and tries to make a new page table. What?
Hm. I concur with the question. Does my macro work correctly? I just wrote that code off the cuff. Have you tried multiple addresses in the same aligned 4MB range?
sed4906h wrote:
Another related question... How should I approach zeroing out the new page table before mapping it?
Ah, the problems I don't have. See, I'm working in 64-bit mode, where I have more than enough virtual space to map all memory linearly and then still have enough left over for logical mappings. I use that for my implementation of virt_from_phys(), which merely has to add a base address.

Well, the solution I would go with is to declare one page directory the "temporary" directory that is always filled. For example, directory 1022, why not. When you initialize paging, you clear a page of RAM and set page directory 1022 to that page. That way, you can map up to 1024 pages temporarily. For the purpose of allocating a zeroed page, you only need one, so I would make that one temporary address per CPU. That way, you don't need TLB shootdowns for this purpose, at least. Then a zeroed allocation would just be
Code:
#define PGD_TEMP 1022
#define TEMP_ADDR ((void*)(PGD_TEMP << 22 | this_cpu()->nr << 12))
uint32_t phys_zalloc(void) {
  uint32_t pg = phys_alloc();
  if (!pg)
    return pg;
  MAKE_PGD_ADDR(PGD_TEMP, this_cpu()->nr) = pg | 3;
  asm("" ::: "memory"); /* make GCC not reorder the memset above the assignment */
  memset(TEMP_ADDR, 0, PAGE_SIZE);
  asm("" ::: "memory"); /* make GCC not reorder the assignment above the memset */
  MAKE_PGD_ADDR(PGD_TEMP, this_cpu()->nr) = 0;
  asm("invlpg %0" :: "r"(TEMP_ADDR) : "memory");
  return pg;
}


Even if the other CPUs ever get a TLB for the current CPU's TEMP_ADDR, it is never accessed, so no page fault is possible. Or if it happens, it is spurious.

I had briefly considered just mapping the new page table to its final place and zeroing it out through the recursive mapping, but then you need to invalidate all 1024 TLBs in that 4MB region. And nobody wants that.

_________________
Carpe diem!


Top
 Profile  
 
 Post subject: Re: My OS gets stuck switching tasks
PostPosted: Mon Nov 29, 2021 3:55 pm 
Offline
Member
Member
User avatar

Joined: Sun Feb 18, 2007 7:28 pm
Posts: 1535
Hi,

We use recursive paging as well but am not sure what the issues are here. It doesnt make sense to zero out the page table before mapping it so idk where that is coming from. You already know the virtual address to use for the page table (as defined by the recursive mapping) so just ask for a free Page Frame Number from the physical allocator and map it. You can clear the new page out after.

_________________
OS Development Series | Wiki | os | ncc
char c[2]={"\x90\xC3"};int main(){void(*f)()=(void(__cdecl*)(void))(void*)&c;f();}


Top
 Profile  
 
 Post subject: Re: My OS gets stuck switching tasks
PostPosted: Mon Nov 29, 2021 8:16 pm 
Offline

Joined: Thu Nov 25, 2021 7:11 pm
Posts: 17
The overarching issue I'm having is that reading back entries from the recursively mapped page tables is giving me back garbage values. I'm certain the physical memory being used is memory that can be used, I make sure that the only physical memory pages that the physical memory allocator can allocate or free are above 0x00400000 and in regions marked as available by the Multiboot map.


Top
 Profile  
 
 Post subject: Re: My OS gets stuck switching tasks
PostPosted: Mon Nov 29, 2021 8:46 pm 
Online
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 3590
neon wrote:
It doesnt make sense to zero out the page table before mapping it so idk where that is coming from.

If you map it before you clear it, the CPU may use it to load nonsense into the TLB, and you'll need to flush that nonsense from the TLB.


Top
 Profile  
 
 Post subject: Re: My OS gets stuck switching tasks
PostPosted: Tue Nov 30, 2021 3:20 am 
Offline
Member
Member
User avatar

Joined: Sun Feb 18, 2007 7:28 pm
Posts: 1535
Hi,

Without code for reference cant be sure what the issue is. This is what we do (slightly modified) for page tables, however there really isn't much to it that hasn't already been posted above. The recursive mapping places all page tables relative to 0xffc00000 so we just map it to a frame and invalidate the TLB before using it.
Code:
#define MM_PAGE_TABLE_BASE 0xffc00000
#define MM_GET_PAGE_TABLE(addr) ((MM_PAGE_TABLE_BASE) + (MM_PAGE_TABLE_SIZE * MM_PAGE_DIRECTORY_INDEX((addr))))
PUBLIC BOOL MmPrepareAddressMapping(IN VIRTUAL_ADDRESS address) {

   MMPDE*          pde;
   PFN_NUMBER      pageTablePfn;
   VIRTUAL_ADDRESS pageTableAddress;

   pde = MmAddressToPde(address);
   if (pde->u.device.valid == 0) {

      pageTablePfn = MmGetFreeFrame ();
      pageTableAddress = MM_GET_PAGE_TABLE (address);

      pde->u.device.pageFrameNumber = pageTablePfn;
      pde->u.device.valid = 1;

      MmAllocateFrame (pageTableAddress);
      MmInvalidateAddress (pageTableAddress);
   }
   return TRUE;
}

_________________
OS Development Series | Wiki | os | ncc
char c[2]={"\x90\xC3"};int main(){void(*f)()=(void(__cdecl*)(void))(void*)&c;f();}


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 20 posts ]  Go to page 1, 2  Next

All times are UTC - 6 hours


Who is online

Users browsing this forum: Bing [Bot], DotBot [Bot], Majestic-12 [Bot], Octocontrabass and 15 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group