OSDev.org

hi I have a question about pe loaders. I was reading that a pe loader loads a pe file by mapping the file into memory. From what I have heard it is similar to how memory mapped files work.


1. call CreateFile to open the file you want to map.
2. call CreateFileMapping with the file handle returned by CreateFile as one of its parameter. This function creates a file mapping object from the file opened by CreateFile.
3. call MapViewOfFile to map a selected file region or the whole file to memory. This function returns a pointer to the first byte of the mapped file region.
4. Use the pointer to read or write the file
5. call UnmapViewOfFile to unmap the file.
6. call CloseHandle with the handle to the mapped file as the parameter to close the mapped file.
7. call CloseHandle again this time with the file handle returned by CreateFile to close the actual file.

Would the pe loader allocate memory on its heap for the pe file ? I came to this conclusion because if its on the heap it can make a system call with using malloc() or new in order for the operating system to update its page tables. Please correct me if i'm wrong.

Loading PE images is effectively the same as memory mapped files; it is mapped and loaded directly into the address space. This is the way we do it, anyways; there is no standard. We suspect it is also the same basic method used in Windows given our experiments; it is easy to self extract any part of your PE file directly in memory under Windows. We also don't recommend the use of a heap due to the size of PE images; the heap can be used for temporary storage but not for loading the entire image file (consider that it is not uncommon to have image sizes > 100 MB.)

I wanted to understand how something like the windows pe loader works. Where would the windows loader load the pe file if not the heap, and how would it go about updating the page table in the operating system?

All the system needs to do is load and map the image in the current process address space (or create a new one if needed) section by section. The page tables and directories would be allocated automatically as new pages are requested while mapping the image. That is,
Note the system requests new frames to allocate for the image in order to load it and directly maps it to the correct location in the process address space. Also note that the above is very basic; the Windows process manager is highly optimized; I would encourage picking up a copy of Windows Internals to read more about it.

If interested, I have recently published an article - Process Management 1 - that discusses the basic strategy in more depth. For Windows however I strongly recommend picking up the Windows Internals book; it is a great read.

I heard that the pe loader is a seperate executable that is loaded into the process's address space. How does the pe loader map the pe file outside of itself within the process address space? What I mean is mapping it outside the allocated space meant for the pe loader. I'm not sure how that works. Maybe I shouldn't be worried about it at this stage because i'm just a beginner, but I am very curious as to how it is done.

No. It is part of the loader component of the kernel image. In a pure microkernel architecture the loader component can be separate from the kernel or executive however Windows is a hybrid; monolithic in its design and places the loader component in the kernel.

Microkernel architecture specific
In a pure microkernel architecture, it is possible to place the loader component in a separate user space server by preloading it with the kernel or executive with other servers. The executive can then go through the initial server list to initialize them; the PE loader server can then provide the basic services for PE image loading and communicate with the memory manager for memory requests and mapping services via Inter Process Communication (IPI) services offered by the kernel or executive.

This is fairly complicated as with any microkernel design though and is not what is used by most operating systems due to performance reasons. As noted above, Windows places the loader component in the kernel image itself since it is monolithic (advertised as a hybrid.)

I do still recommend the Windows Internals book since it discusses Windows specific design issues in greater detail.