I've added support for IOAPIC, MSI and MSIX interrupts, but I'm having an issue on certain hardware where an interrupt is being handled on multiple CPUs instead of just the lowest priority one. Here's my setup:
- During boot I set the logical destination register (LDR) of the boot APIC to be 0x01 << 24
- When configuring the IOAPIC, I set the appropriate redirection table entries to 0x0100000000000936, which should be lowest priority, logical mode.
- When I bring up all of the application processors, I set all of their LDR's to be 0x02 << 24
- Once all APs are up, I update the IOAPIC pins to be: 0x0300000000000936
- At boot, I set the TPR to 0 for all local APICs, and don't touch it again after.
From reading the intel docs (Vol3A chapter 10) and Brendan's post
http://forum.osdev.org/viewtopic.php?p=202258#p202258 on the subject, it was my understanding that the LDR could be used to set up groups of processors to receive IPIs or interrupts. In my case, I have one group that is just the boot processor, and another group that contains all of the APs. Then when sending a lowest priority interrupt using logical mode, setting the IOAPIC destination field to be (0x1 | 0x2) would send the interrupt to all local APICs, and they would arbitrate amongst themselves to determine who was the lowest priority CPU and subsequently handles the interrupt.
Now this seems to work on most of the systems I've tested on, including virtual box. Only a single cpu actually handles the interrupt, and it's sometimes the BSP, sometimes the AP. However, on two Xeon's server boxes I've got, a single interrupt is actually being handled by all application processors (but not on the boot processor). So my PIT timer (and therefore time of day) is incrementing at a rather amusing rate, since it's being bumped 15 times more than it should be (16 core machine).
Is my understanding of how grouping processors with the LDR works correct? Are there differences in how lowest priority mode is used on different classes of CPUs? Any light you can shine on the subject would be most appreciated. Thanks!
**edit: Only on the machines I'm having difficulty with, I noticed that bits 48:55 of the redirection table entries is being set. Took some time to track it down since this is in the reserved section according to the 82093AA spec, but in the ICH9 spec (
http://www.intel.com/content/dam/doc/datasheet/io-controller-hub-9-datasheet.pdf, pg. 477), it says these are:
"Extended Destination ID (EDID) — RO. These bits are sent to a local APIC only when in Processor System Bus mode. They become bits 11:4 of the address."
Now I'm even more confused... what address is this referring to?