OSDev.org

The Place to Start for Operating System Developers
It is currently Tue Jan 31, 2023 1:54 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 6 posts ] 
Author Message
 Post subject: What to do if you need to allocate physical memory below 4gb
PostPosted: Thu Dec 08, 2022 5:00 am 
Offline
Member
Member

Joined: Fri Feb 11, 2022 4:55 am
Posts: 339
Location: behind the keyboard
Some devices only support 32 bit addressing and optionally 64 bit (EHCI, AHCI...). In my OS, I just set a flag to the page allocator to search for memory below 4GB. Does Linux and other OSes do the same thing ?


Top
 Profile  
 
 Post subject: Re: What to do if you need to allocate physical memory below
PostPosted: Thu Dec 08, 2022 9:19 am 
Offline
Member
Member

Joined: Wed Aug 30, 2017 8:24 am
Posts: 1420
Basically. Linux has the idea of a "DMA mask". It has an extra DMA allocator to allocate device memory, and that allocator is constrained by the position of the device in the device tree. EHCI can only deal with 32-bit addresses (most of the time, at least), so it gets a mask of 32 bits. It can be constrained further if a specific device is detected that only supports 31 bits. Then most EHCI devices are PCI devices, so the memory they can access is constrained to the ability of the PCI host bridge to forward requests. On the PC, the PCI host bridge typically forwards requests to all physical memory and without offset, but that doesn't need to be the case.

In the end, all memory allocated to a device has three addresses: The virtual address in the CPU, the physical address from the point of view of the CPU, and the DMA address, which is the physical address of the memory from the point of view of the device. The last two needn't be equal. They are on the PC, but not everything is a PC.

_________________
Carpe diem!


Top
 Profile  
 
 Post subject: Re: What to do if you need to allocate physical memory below
PostPosted: Fri Dec 09, 2022 4:17 am 
Offline
Member
Member

Joined: Wed Oct 01, 2008 1:55 pm
Posts: 2954
nullplan wrote:
Basically. Linux has the idea of a "DMA mask". It has an extra DMA allocator to allocate device memory, and that allocator is constrained by the position of the device in the device tree. EHCI can only deal with 32-bit addresses (most of the time, at least), so it gets a mask of 32 bits. It can be constrained further if a specific device is detected that only supports 31 bits. Then most EHCI devices are PCI devices, so the memory they can access is constrained to the ability of the PCI host bridge to forward requests. On the PC, the PCI host bridge typically forwards requests to all physical memory and without offset, but that doesn't need to be the case.

In the end, all memory allocated to a device has three addresses: The virtual address in the CPU, the physical address from the point of view of the CPU, and the DMA address, which is the physical address of the memory from the point of view of the device. The last two needn't be equal. They are on the PC, but not everything is a PC.


I don't use flags in the calls (don't like enums or flags-bits in APIs), but instead provide several allocator calls. I have variants that allocate memory below 4G (some 32-bit PCI devices) and variants that can allocate anywhere, but that allocates above 4G if there is free memory above 4G. I have variants that allocates one 4k page (typically used by page-fault handler), one that allocates one 2M page (used for large blocks), and a special function that can allocate any number of consecutive 4k pages (typically used by PCI devices that need large memory schedule blocks).

For easy translation between linear and physical addresses (and the reverse), I have an allocator built on a 4k page that is optimized for same-sized allocations. For large allocations, it will automatically add new 4k pages as needed. The header contains both the linear and the physical address, and it is quick to check if a particular address from a memory schedule is inside the allocator, and to return it's linear address.

I only support PC configurations, and assume devices see the same address space as the CPU.


Top
 Profile  
 
 Post subject: Re: What to do if you need to allocate physical memory below
PostPosted: Fri Dec 09, 2022 5:27 am 
Offline
Member
Member

Joined: Fri Feb 11, 2022 4:55 am
Posts: 339
Location: behind the keyboard
So from what you say, Linux remaps DMA addresses to be accessible by the device using 32 bits for example. If the address is 0x10000000000, which is above 1gb the CPU accesses it as 0x10000000000 as physical address but we can make it accessible to the device using address 0x12000 for example.

How can I do that too ? Should I read this : https://wiki.osdev.org/ISA_DMA


Top
 Profile  
 
 Post subject: Re: What to do if you need to allocate physical memory below
PostPosted: Fri Dec 09, 2022 12:20 pm 
Offline
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 4505
devc1 wrote:
So from what you say, Linux remaps DMA addresses to be accessible by the device using 32 bits for example.

No, Linux has a special allocator that chooses addresses that will be accessible by the device.

Linux is also capable of using an IOMMU to remap DMA addresses, but an IOMMU is not always available.

devc1 wrote:
Should I read this : https://wiki.osdev.org/ISA_DMA

Probably not. Only ancient ISA devices use the ISA DMA controllers.


Top
 Profile  
 
 Post subject: Re: What to do if you need to allocate physical memory below
PostPosted: Fri Dec 09, 2022 12:43 pm 
Offline
Member
Member

Joined: Fri Feb 11, 2022 4:55 am
Posts: 339
Location: behind the keyboard
Ok, now I understand.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 6 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: Bing [Bot], Google [Bot], Majestic-12 [Bot] and 7 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group