OSDev.org

The Place to Start for Operating System Developers
It is currently Sat Nov 27, 2021 10:53 pm

All times are UTC - 6 hours




Post new topic Reply to topic  [ 6 posts ] 
Author Message
 Post subject: Intel VT-d IOMMU passthrough
PostPosted: Fri Oct 15, 2021 2:04 am 
Offline
Member
Member

Joined: Sat Dec 22, 2012 12:01 pm
Posts: 73
Hi,

I've some confusion about the linux kernel boot options iommu=on and iommu=pt in the context of virtualization environments (e.g. QEMU/KVM). From my understanding iommu=on turns on the IOMMU device unit (if it exists in HW of course) for all uses. Adding further the option iommu=pt does mean the following:

    from Host OS point of view the IOMMU unit is actually disabled (DMA target addresses are the same as host system physical addresses)

    physical devices (e.g. NIC) configured in passthrough (e.g. NIC PCI passthrough) assigned to a Guest running in a VM, continue to use the IOMMU unit (their DMA target addresses are actually I/O virtual addresses assigned to the DMA-capable passthrough NIC by the Guest and translated by IOMMU DMARs to access memory)

Does it makes sense ? Thank you.

ps. from the Intel manual 'vt-directed-io-spec - rev 3.3' section 10.4.3 the PT bit into Extended Capability Register is set if HW does support IOMMU passthrough. For each PCI device identified by its B/D/F the looked up Context Entry (or Scalable-Mode PASID Table Entry) specifies the Translation Type in TT (or GPTT) field (see section 9.3 - 9.6). Pass-through mode (TT = 10b or GPTT = 100b) actually means IOMMU does not perform any form of DMA target address translation -- i.e. DMA target addresses are actually system physical addresses.


Top
 Profile  
 
 Post subject: Re: Intel VT-d IOMMU passthrough
PostPosted: Sun Oct 17, 2021 10:59 am 
Offline
Member
Member

Joined: Sat Dec 22, 2012 12:01 pm
Posts: 73
Any feedback ? Thank you.


Top
 Profile  
 
 Post subject: Re: Intel VT-d IOMMU passthrough
PostPosted: Sun Oct 17, 2021 12:18 pm 
Offline
Member
Member

Joined: Sun Jun 23, 2019 5:36 pm
Posts: 456
Location: North Dakota, United States
Please be patient. We are not being paid to answer your questions and we answer them in our own time.


Top
 Profile  
 
 Post subject: Re: Intel VT-d IOMMU passthrough
PostPosted: Sun Oct 17, 2021 8:50 pm 
Offline
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 3484
PCI passthrough requires an IOMMU. Without an IOMMU, the hypervisor has to emulate the device.

Accessing devices through an IOMMU adds some overhead. When you're not doing PCI passthrough, you probably don't want to use the IOMMU.

The "iommu=pt" option gives you the best (or at least fastest) of both worlds. The IOMMU is used for PCI passthrough, and it isn't used for anything else.


Top
 Profile  
 
 Post subject: Re: Intel VT-d IOMMU passthrough
PostPosted: Mon Oct 18, 2021 1:58 am 
Offline
Member
Member

Joined: Sat Dec 22, 2012 12:01 pm
Posts: 73
Octocontrabass wrote:
PCI passthrough requires an IOMMU. Without an IOMMU, the hypervisor has to emulate the device.
Accessing devices through an IOMMU adds some overhead. When you're not doing PCI passthrough, you probably don't want to use the IOMMU.

The "iommu=pt" option gives you the best (or at least fastest) of both worlds. The IOMMU is used for PCI passthrough, and it isn't used for anything else.
Thank you for the answer.

AFAIK the IOMMU HW unit is actually employed only from DMA-capable devices to perform DMA accesses either to physical memory (i.e. DRAM) or to other MMIO devices's registers.

So when you say Accessing devices through an IOMMU adds some overhead are you actually referring to DMA accesses (transactions) performed from DMA-capable devices as described in my sentence above ? Thank you.


Top
 Profile  
 
 Post subject: Re: Intel VT-d IOMMU passthrough
PostPosted: Mon Oct 18, 2021 9:51 am 
Offline
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 3484
Yes. When the IOMMU has to walk its page tables to map a DMA transaction to physical memory, the DMA transaction has to wait.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 6 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: Google [Bot], monarrk and 18 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group