OSDev.org

The Place to Start for Operating System Developers
It is currently Thu Mar 28, 2024 11:58 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 22 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: Can't figure out paging
PostPosted: Sat Sep 02, 2017 5:17 pm 
Offline
User avatar

Joined: Wed Aug 23, 2017 1:09 pm
Posts: 20
I'm trying to add paging to my OS and I've tried several methods online, but to no avail. I already have a simple paging function (with free memory address hardcoded in) in mem.c but it's no where near as complex as the paging methods I've found on the internet. I was following this tutorial OS From Scratch but I finished his book and he doesn't have paging so next I started following JamesM's tutorial and I got stuck at paging. If somebody could point me to a paging tutorial with my own bootloader (instead of GRUB) I would like that. Or my files are at this dropbox link if you would please take a look at them.

Paging Method from libc/mem.c
Code:
uint32_t free_mem_addr = 0x10000;

uint32_t kmalloc(uint32_t size, int align, uint32_t *phys_addr) {
   if (align ==1 && (free_mem_addr & 0xFFFFF000)) {
      free_mem_addr &= 0xFFFFF000;
      free_mem_addr += 0x1000;
   }

   if (phys_addr) *phys_addr = free_mem_addr;

   uint32_t ret = free_mem_addr;
   free_mem_addr += size;
   return ret;
}

_________________
Gigaboy


Top
 Profile  
 
 Post subject: Re: Can't figure out paging
PostPosted: Sat Sep 02, 2017 6:07 pm 
Offline
Member
Member
User avatar

Joined: Fri Aug 07, 2015 6:13 am
Posts: 1134
The function from above really doesn't have anything to do with paging at all.
I would advise you against following any tutorials, because that is one of the most crucial parts of an operating system.
Intel manuals are a good way to start: https://software.intel.com/sites/default/files/managed/a4/60/325384-sdm-vol-3abcd.pdf, take a look at Chapter 4 called Paging, it has everything you will need. It might take a few (x10) reads.
Paging is something very specific and requires a good understanding of the mechanism itself, before you can code anything.
You primary goals should be:
1.Figure out how it works (in details) in your head.
2.Figure out what a page directory, page directory entry, page table and page itself is. I would suggest you using bit-fields, leave clunky arrays of integers and bit-shifts alone.
3.Figure out how to translate an address aka how to map it. Physical -> Virtual
4.Enable paging by setting PG bit in CR0.
5.Tell us how badly does it triple fault, remember: paging structures don't like unaligned addresses. It has to fail at least once, it is perfectly normal to experience some issues at the very beginning.

This was just a really really short overview. You still have a lot of decision to make: higher half, PAE or not, PSE or not (you are not ready for this yet, not a good idea)...
Feel free to ask anything you want. Currently I don't have any more time but if you want I can explain you in detail how I did it and what paths did I take.

_________________
OS: Basic OS
About: 32 Bit Monolithic Kernel Written in C++ and Assembly, Custom FAT 32 Bootloader


Top
 Profile  
 
 Post subject: Re: Can't figure out paging
PostPosted: Sat Sep 02, 2017 6:25 pm 
Offline
User avatar

Joined: Wed Aug 23, 2017 1:09 pm
Posts: 20
Octacone wrote:
Feel free to ask anything you want. Currently I don't have any more time but if you want I can explain you in detail how I did it and what paths did I take.

Do I need paging in my OS? I've seen some OSes without paging in it at all.

Thanks for the Intel link, it's very helpful.

_________________
Gigaboy


Top
 Profile  
 
 Post subject: Re: Can't figure out paging
PostPosted: Sat Sep 02, 2017 7:22 pm 
Offline
Member
Member
User avatar

Joined: Sat Dec 27, 2014 9:11 am
Posts: 901
Location: Maadi, Cairo, Egypt
Gigaboy wrote:
Do I need paging in my OS? I've seen some OSes without paging in it at all.

Definitely.

Paging is the hardware's mechanism of memory protection, virtual address space, and other modern things. Of course, it's possible to write an OS that doesn't use paging, and depends either on segmentation or a flat non-paged address space. For the initial, you're using obsolete technology. Most OSes today don't use segmentation, and even x86_64 removed support for segmentation at all (except for FS and GS registers, but that's irrelevant.) For the latter, each program must be loaded at a different address, which means programs must be aware that they are loaded at a non-standard location, or that each program must have their own linker script, which is plain ridiculous.

In any case, in order to properly implement multitasking, which is required in any modern OS, you need to implement paging. Octacone's overview on paging should get you started, and the Intel manuals will help a lot. Feel free to ask here when you need help. :)

_________________
You know your OS is advanced when you stop using the Intel programming guide as a reference.


Top
 Profile  
 
 Post subject: Re: Can't figure out paging
PostPosted: Mon Sep 04, 2017 4:47 pm 
Offline
User avatar

Joined: Wed Aug 23, 2017 1:09 pm
Posts: 20
In JamesM's tutorial he uses the variable "placement_address". It's defined in his kheap.c file
Code:
// end is defined in the linker script.
extern u32int end;
u32int placement_address = (u32int)&end;
and "end" is defined in the linker script.
Code:
end = .; _end = .; __end = .;
He's using GRUB for his bootloader and I'm using my own bootloader, my question is how to I get the value of "end" in my bootloader?

EDIT: More specifically, how do I set "placement_address" to the end of my kernel.

_________________
Gigaboy


Top
 Profile  
 
 Post subject: Re: Can't figure out paging
PostPosted: Tue Sep 05, 2017 1:33 am 
Offline
Member
Member

Joined: Wed Jun 17, 2015 9:40 am
Posts: 501
Location: Athens, Greece
Hi,


Gigaboy wrote:
In JamesM's tutorial
I suggest you don't use JamesM's tutorial. It's full of bugs, hasn't been updated in 9 years and probably hasn't even been peer reviewed.

As for paging, yes, you probably need it if you want to make a modern OS. I however don't know of a good tutorial about paging - you might as well try Octacone's advice for now.


Regards,
glauxosdever


Top
 Profile  
 
 Post subject: Re: Can't figure out paging
PostPosted: Tue Sep 05, 2017 9:50 am 
Offline
User avatar

Joined: Wed Aug 23, 2017 1:09 pm
Posts: 20
I hate low-level coding and everything in the Intel manual is Assembly but I'll try to use that.

_________________
Gigaboy


Top
 Profile  
 
 Post subject: Re: Can't figure out paging
PostPosted: Tue Sep 05, 2017 9:57 am 
Offline
Member
Member
User avatar

Joined: Sat Mar 31, 2012 3:07 am
Posts: 4591
Location: Chichester, UK
Though the Intel manuals describe concepts in terms of the underlying hardware, it doesn't mean that you have to use assembler to realize those concepts. Paging requires no assembler instructions, apart from those to enable it in the first place and the occasional instruction to flush the TLBs and to load/save register cr3.


Top
 Profile  
 
 Post subject: Re: Can't figure out paging
PostPosted: Tue Sep 05, 2017 11:46 am 
Offline
Member
Member
User avatar

Joined: Wed Aug 08, 2012 6:39 am
Posts: 42
Location: New York, NY
If you hate low-level coding, then why are you trying to do OS development? Just out of curiosity.

_________________
Cheers,

Lev


Top
 Profile  
 
 Post subject: Re: Can't figure out paging
PostPosted: Tue Sep 05, 2017 12:07 pm 
Offline
Member
Member
User avatar

Joined: Fri Aug 07, 2015 6:13 am
Posts: 1134
There is almost no low-level assembly involved when it comes to paging. You won't need more than a few lines of code. - Very simple code
You definitely need paging, it is widely used these days, provides some mandatory features, also Long Mode has paging enabled by default.
You certainly won't be able to learn/understand it over night, it takes some time and deep concentration. The main problem I see is, you don't even know why you need paging.
First figure out what you can't do without it and then try to understand how to implement it.

_________________
OS: Basic OS
About: 32 Bit Monolithic Kernel Written in C++ and Assembly, Custom FAT 32 Bootloader


Top
 Profile  
 
 Post subject: Re: Can't figure out paging
PostPosted: Tue Sep 05, 2017 2:30 pm 
Offline
User avatar

Joined: Wed Aug 23, 2017 1:09 pm
Posts: 20
lkurusa wrote:
If you hate low-level coding, then why are you trying to do OS development? Just out of curiosity.

I enjoy coding and I thought it would be fun to make my own OS. (It is fun BTW)

Octacone wrote:
There is almost no low-level assembly involved when it comes to paging. You won't need more than a few lines of code. - Very simple code
You definitely need paging, it is widely used these days, provides some mandatory features, also Long Mode has paging enabled by default.
You certainly won't be able to learn/understand it over night, it takes some time and deep concentration. The main problem I see is, you don't even know why you need paging.
First figure out what you can't do without it and then try to understand how to implement it.

I'll try to figure it out on my own but over the week I might post some questions here.

_________________
Gigaboy


Top
 Profile  
 
 Post subject: Re: Can't figure out paging
PostPosted: Thu Sep 07, 2017 1:58 pm 
Offline
User avatar

Joined: Wed Aug 23, 2017 1:09 pm
Posts: 20
After reading through 32-bit paging section in the Intel PDF file and looking at others source code I came up with the code in the files paging.c/h and kheap.c/h. All the C files compile with no errors/warnings and when I run the OS image in Qemu it doesn't work. The files I used for paging (paging.c/h, kheap.c/h, panic.c/h, and system.h) are all in the libc/ directory. Dropbox Files

_________________
Gigaboy


Top
 Profile  
 
 Post subject: Re: Can't figure out paging
PostPosted: Thu Sep 07, 2017 3:01 pm 
Offline
Member
Member
User avatar

Joined: Fri Aug 07, 2015 6:13 am
Posts: 1134
Gigaboy wrote:
After reading through 32-bit paging section in the Intel PDF file and looking at others source code I came up with the code in the files paging.c/h and kheap.c/h. All the C files compile with no errors/warnings and when I run the OS image in Qemu it doesn't work. The files I used for paging (paging.c/h, kheap.c/h, panic.c/h, and system.h) are all in the libc/ directory. Dropbox Files


It is really a bad idea to use that code, as stated above the tutorial you're following is not the best. It can be used as a reference but nothing more.
Please take a look: http://wiki.osdev.org/James_Molloy's_Tutorial_Known_Bugs, you really shouldn't be using that code.
You probably don't know it, but there is a bitmap based Physical Memory Manager integrated within that code, which is really really bad. PMM should be an entirely separate concept and it has nothing to with paging.
Why did you put your files inside a libc folder doe? That seems kind of (and is) really strange.
Please try to code it yourself. Don't copy and past code, you will learn very little by doing so.
It is much more likely that you will learn by trying to code something by yourself and encountering problems yourself.
Paging is not hard, it just required a certain level of logical understanding.

Other than that, what exactly is your problem. Doesn't work is not very descriptive.

_________________
OS: Basic OS
About: 32 Bit Monolithic Kernel Written in C++ and Assembly, Custom FAT 32 Bootloader


Top
 Profile  
 
 Post subject: Re: Can't figure out paging
PostPosted: Thu Sep 07, 2017 3:29 pm 
Offline
User avatar

Joined: Wed Aug 23, 2017 1:09 pm
Posts: 20
Quote:
...what exactly is your problem. Doesn't work is not very descriptive.


Qemu keeps looping the code up to init_page()
5 sec video

Quote:
It is really a bad idea to use that code, as stated above the tutorial you're following is not the best. It can be used as a reference but nothing more.
Please take a look: http://wiki.osdev.org/James_Molloy's_Tu ... Known_Bugs, you really shouldn't be using that code.
You probably don't know it, but there is a bitmap based Physical Memory Manager integrated within that code, which is really really bad. PMM should be an entirely separate concept and it has nothing to with paging.

I had no idea JamesM's tutorials had that many bugs (gonna stop following his tutorial).

Quote:
Why did you put your files inside a libc folder doe? That seems kind of (and is) really strange.

Yes, it's strange. I was going to make a mm/ directory for paging, heap, malloc, etc. but I didn't feel like creating a new directory if the code wasn't going to work. My Makefile only compiles the code in libc/, cpu/, or kernel/ so I had to put the files in one of those folders or it wouldn't compile them.

_________________
Gigaboy


Last edited by Gigaboy on Thu Sep 07, 2017 4:02 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Can't figure out paging
PostPosted: Thu Sep 07, 2017 3:58 pm 
Offline
Member
Member
User avatar

Joined: Fri Aug 07, 2015 6:13 am
Posts: 1134
Gigaboy wrote:
Quote:
...what exactly is your problem. Doesn't work is not very descriptive.


Qemu keeps looping the code up to init_page()
5 sec video


That is definitely a Triple Fault, possibly followed by a page fault.
It is time for some debugging. Use Bochs and tell us the exact error message.
Possible causes (some of them):
  • Page directory/entries are not page aligned.
  • CR3 doesn't point to a page directory.
  • Kernel is not mapped.
  • Paging structures are not mapped.
  • Trying to access invalid/(not mapped, not present) memory.

But we can only guess until you show us the real error.

_________________
OS: Basic OS
About: 32 Bit Monolithic Kernel Written in C++ and Assembly, Custom FAT 32 Bootloader


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 22 posts ]  Go to page 1, 2  Next

All times are UTC - 6 hours


Who is online

Users browsing this forum: Amazonbot [bot] and 68 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group