OSDev.org

Hullo,
I am here again. Probably some of you may be angry at me for posting so rapidly, with these noobish kind of problems, but I guess, you would have to bear with me.
So, I am developing my own OS, and even though I use QEMU, I love testing it on real hardware. So today, I implemented Paging, learning how to do it by JamesM's tutorial. I thought to test it on real hardware, and when I started it up, it automatically rebooted, without showing anything. Has a triple fault occured?
When I tried it on QEMU, it booted up properly, and showed me what it had to.
So I have attached the source code, and want a little help, on knowing what is causing the problem.

Triple-fault is very likely.

But dumping the source code and expecting others to do the debugging?

First order of the day: Put "pings" into your code. Make your bootloader print an "A" before continuing. Make your bootloader print a "B" before jumping into the kernel. Make your kernel print a "C" before doing anything. And so on.

Then put an endless loop after the "A" ping. If that works, put one after the "B" ping. And so on.

Find out, as precisely as possible, which part of your code makes it reboot.

Then look at it, perhaps you'll find the problem yourself.

Then come back.

That's debugging 101. It's so menial a task that you cannot really expect everyone who wants to help you to do it for you.

Hi sir,
Was quite waiting for a response. Thank you for it. I would surely do the debugging 101. I have a extra stack of CDs so would put few of these versions in those, and would then debug it.
Regards,
Shikhin

I'd think about a CD-RW in that case...

Or better yet, an USB floppy drive.

Or better yet, making a partition for your own OS and booting directly from HD...

I have CD RW but who would write to them a million times. Instead, I would write few of them, so that I dont have to re boot, to write, another version with one less infinite loop, again.

As far as I seem to gather, the problem is occuring in this line:

Which tries, to put the updated value of the register cr0, into it.

Here is the complete function:


Can anyone tell me what the problem is. I am working on the debugging part since morning, as I want to complete this part till night, and then start unravelling it.

Again, I as well as other OS devers) suggest testing iso files with virtual machines. Burning CDs for testing a loop is a huge waste.

And about the rebooting problem, it is definitely a Triple-Fault.

Hi Sir,
I always test it with a virtual machine, such as QEMU. However I like testing it on some real hardware once in a while, and I have got a whole stock of RW CDs, so its no problem for me.

I have posted the part which is causing the problem. Can you tell me why is it causing it?

Erm... I'm not the inline ASM expert here, but... are you using AT&T (default) or Intel notation?

Your triple-fault is probably coming from that piece of code, but then again it isn't. What I mean is: this piece of code is directly causing the triple fault, but it's not the fault of that piece of code. There's a very big chance your page tables are layed out incorrectly (e.g. you mapped some incorrect regions, you forgot to identity map the kernel, etc.). Make sure you completely understand paging before you try to fix it. I know paging is hard (especially if this is your first time and you perhaps don't fully understand it yet), but once you do, you'll most likely find your problem.

One of the possible causes is that you forgot to identity map the kernel (or did it incorrectly). This means you forgot to set the first n pages of the first page table to the correct physical addresses. You must make sure that once paging is activated, your kernel still "exists". If you wouldn't identity map your kernel, it would reside at e.g. 1 MB physical addressing. However, once paging is activated, your using virtual addresses. These have to be explicitely "mapped" (or specified, if you will). This is why you must ensure that the first range of virtual addresses point to the physical addresses of your kernel (e.g. 1 MB virtual points to 1 MB physical, 2 MB virtual to 2 MB physical, etc.). Your kernel could reside at another virtual address, but simply ignore that fact for now. An example: you have some variable located at adres 0x100005 (slightly above 1 MB, where your kernel is most likely loaded by GRUB). When your kernel boots, this adres exists in memory (it is a physical adres). However, when paging is on, you must use virtual memory, so you must make sure that adres 0x100005 in virtual memory is the same as 0x100005 in physical memory. You do this by simply mapping the first page table entries to the correct physical addresses (e.g. firsttable[0] covers 0 - 4 kB, firsttable[1] covers 4 - 8 kB, etc.). You must do this for the entire region your kernel will need (this will always be above 1 MB as you probably also want to identity map the kernel).

I hope this clears some things up for you. It took me a very (very!) long time before I properly got a grasp of paging, and a friend of mine had trouble with it as well. So if you have any questions, I will try to explain it again.

Hi Creature,
I read your post completely. Now correct me if I am wrong:

I have to map the (PYHSICAL) address space used by the kernel to the SAME virtual address?

Also, can you explain me paging, in its simplest form, and how it is carried out and developed.

Regards,
Shikhin

Well, usually we refer to virtual addresses being mapped to physical addresses, but I suppose you could put it the other way around. In x86 paging (your standard paging, so no PAE or such involved), an "address space" is simply "a" page directory. Your entire (virtual) address space in this case is 4 GB long, regardless of how much (physical) memory you have installed. This means that there is usually more virtual memory than physical memory. When you activate paging, you use a so-called page directory. This page directory contains 1024 page tables, which each contain 1024 pages. page_table_count * page_count_for_each_table * size_covered_by_each_page = 1024 * 1024 * 4096 = 4 GB. A "page" is usually just an index into a page table and can point to a "physical frame", which is usually 4 kB. You should track what "physical frames" are already in use when you need more of them, and you can do this in various ways. JamesM does it by using a bitmap.

Now, before you activate paging, you still use physical addresses. These are the actual places in memory. If you have 32 MB memory installed, your physical memory length is 32 MB. You can use most of this 32 MB to store whatever you want. When GRUB loads your kernel, it is usually loaded at 1 MB. This is all physical memory, and before you activate paging, you directly access it. However, as I said before, when paging is initialized, you use virtual addresses and have 4 GB of virtual memory available to you. Virtual address 1 MB may not be the same as physical address 1 MB. For this reason, all the pointers and memory used by your kernel "no longer exists" when you use virtual memory. And because of this, you must make sure that once virtual memory is used, 1 MB virtual points to 1 MB physical (so the kernel does still exist at its exact same place). This process is called "identity mapping the kernel". Once paging is activated, you can no longer directly access physical memory. You must first make sure somewhere in virtual memory points to the physical memory location you want to access, and then access that virtual address (which will in fact be addressing the physical address).

Identity mapping the kernel should be done before you activate paging (JamesM does this in his tutorials). The page directory, representing the entire address space, must first be initialized to point virtual memory to the kernel's physical memory. You do this by identity mapping. An easy way to start would be to assume that your kernel will not be larger than 4 MB (1 page table), and simply create the very first page table (page_dir[0]) somewhere at a low physical address that is not used, such as 0x8000. Once you did this, you still have 1024 pages inside this newly created table to map. You simply loop through the 1024 pages, doing something like this:



Note that 0x1000 is the same as 4 kB and 0x1 = PAGE_PRESENT and 0x2 = PAGE_READWRITE. This simple loop will identity map the first 4 MB (I assumsed your kernel is located at 1 MB). The hard part about paging is that all paging structures use PHYSICAL addresses, but you can only access VIRTUAL addresses once it is on. If paging is turned off, you can access physical memory as you would (so during initialization, you can tinker with all the physical addresses you want). This poses a problem: how do I update a specific page or a table when I can only use virtual addresses? I believe JamesM solves the problem by using an array, but this wastes a lot of memory, certainly if you're going to have many addresses. Another technique you could look at is the map-to-self technique, where page_directory[1023] = page_directory_physical_address. A far-fetched approach may be to disable paging every time you want to change something, but it's costly (but it may be enough when you're still learning). It is described further on the wiki. JamesM's technique will get you into trouble when you start with the heap and it needs to expand, but you can use it untill you're ready to write your own from scratch.

Again, I did my best to explain this, but if you have more questions, feel free to ask. Note also that my code may be a bit rusty so it may contain some minor mistakes (but the idea should be there, though).

Hi Creature,
I read your entire post, but I am still not following some of it. Well I understood, some of the identity mapping of the Kernel. However, I couldnt understand whole of it, as I didnt understand paging itself. Can you explain to me how paging works, how it is developed and all. I mean, not WHAT paging is, but How it works, how it is developed and all.
Further more, you said that JamesM identity maps the Kernel. I am using that tutorial only, so what is the problem then. I am copying of the exact code, so can you tell me the problem too.
Regards,
Shikhin

The best thing I can tell you is that you should write your own implementation from scratch (this way you will know its downsides and upsides and such), but I understand this may not be feasible at this time. A memory manager is usually implemented on two layers: the physical and the virtual memory manager. I'll try to tell you about an example implementation (note there are other ways, but I'll try to do it as JamesM does it).

As you know, physical memory is divided into chunks of 4 kB. So that means you take memory_amount and divide it by 4096 (or 0x1000) to find the amount of chunks you need (note that available memory may not always be 4 kB-aligned, so you might lose some chunks at the end, but you can fix that later). Then you create a bitmap where each bit represents one 4 kB-chunk (called a "physical frame"). This bitmap will now be used to indicate whether a frame in physical memory is in use or not. You will most likely need it later on when you have one or more heaps and they need to find some free memory. This is basically all the physical memory manager does. Usually it has some functions like FindFirstFreeFrame, AllocFrame, FreeFrame, and so on. You should now have a picture in your head of the memory in your computer, divided into 4 kB chunks.

On top of the previous layer, you create a virtual memory manager. It usually deals with paging (and sometimes, the heap as well). In the beginning, you will most likely only have one address space (or page directory). You may need more later when you have multiple processes running. The virtual memory manager initializes itself and identity maps the kernel. First you need to put your page directory somewhere. When that is done, you clear it and start initializing (e.g. set the first page table to point to the first 4 MB, as I explained above). Then paging is turned on.

This is basically how both are implemented. Now, I will try to explain how the MMU translates addresses and how virtual addresses are translated into physical addresses. Suppose paging is turned off, and you access adres 0x12345678. Nothing special is involved here, you simply access that memory adres as you desire (it must exist of course, i.e. you must have that amount of memory installed in your computer). With paging turned on, it is a whole different story. Different types of paging (e.g. standard, PAE, PML4, etc.) translate virtual addresses differently, but I will focus on standard x86 paging as that's what you're implementing here. So let's get back to our address 0x12345678. When using this address, the MMU will use the currently active address space or page directory (which is stored in CR3 as a physical address). Generally, virtual addresses look like this:



As you can see, the first 12 bits are an offset from the page. 12 bits means 10^12 possibilities, thus 4096 possibilities (0 - 4095). This value is usually of no use to you. It describes how far the address is from the page in question. The next 10 bits are the index into your page table. 10^10 = 1024 possibilities, what a coincidence, since there are 1024 pages in each page table! The last 10 bits are the index into your page directory, so they indicate what page table is used. Now let's go back to address 0x12345678 and see how it fits in. 0x12345678 translates to binary is 0001001000_1101000101_011001111000 (the underscores mark where the fields above end). The highest bits indicate our page directory index and the value is 000100100, which is 72, so the MMU will take your page directory and go to page_directory[72]. page_directory[72] holds the physical address of a page table, so the MMU jumps to this address and continues looking. Note that the value here contains physical_adres | page_present | other_flags. If the value were to be 0, you'll get a page fault (page not present), because the MMU doesn't know what address to use for this virtual address. Now that we have the page table we want, we continue looking: the index into the page table is 110100010, or 418, so let's see what physical address is in page_table_at_index_72[418]. This address will also be equal to some_physical_adres | page_present | other_flags (if page_present is not set, you will again get a page fault). This time, the MMU has found the physical address it needs. Suppose the value of this page is 0x4001. As you know, the first bit of a page is PAGE_PRESENT (0x1), so this is 0x4000 | 0x1. We discard the flags (we only need the physical address), and we find 0x4000. The MMU now knows that address 0x12345678 is mapped to physical frame 0x4000. The only thing remaining now are the last 12 bits, which define the offset. The offset here is 011001111000, or 1656. The MMU is now finished, the physical address you are actually using when you use virtual address 0x12345678 is 0x4000 + 1656 = 18040 or 0x4678.

The last paragraph should give you a general idea of how the MMU translates addresses. Now that you know this, you know why making sure the pages inside each page table must have the right values and that the kernel must be identity mapped to itself so it doesn't just "disappear" when paging is activated.

Note that I'm just trying to explain this to you, the error in your code is most likely just a small mistake you made somewhere (e.g. forgot to memset a paging structure, etc.)., as JamesM tutorial on paging does work properly (although most people roll their own once they understand it completely). I also assumed that you know about the bits that have to be set when you set an entry in a page directory or page table (shameless plug):



This structure is not entirely correct (as some of the unused bits aren't actually unused, so you can best not use them at this time, but JamesM doesn't do anything special to them so you probably aren't either). For you to not get the "page fault - page not present" exception, the "present" bit must always be set. You can do this by:



But you probably knew that (since you should know your bitwise operators ).

Hi Sir, (Now I feel I should call you sir )

After reading your post, I clearly understood what paging is and have got to known the following three things:

1. How paging is implemented.
2. How MMU translates the addresses.
3. Sir, YOU ROCK! I have seldom found anyone, willing to give, his precious time, to explain one particularly stubby subject to a student/beginner.

I am again now trying to implement Paging, and if any problems occur, I will post here. : Dedicated to you!