[Solved] Can't get IRQs to work on RPI4

pvc · **Joined:** Mon Jan 15, 2018 2:27 pm **Posts:** 201

I am trying to get IRQs to work on Raspberry Pi 4. Without much success. One of few things I know about it (thank you Broadcom for being absolute d :evil:

cks about documentation) is that it uses GIC-400 interrupt controller.

CPU is running in secure EL1 mode (AArch64).
A, I and F bits in PSTATE are all clear.
Interrupt handling state machine seems to be working as supposed.

Here is an example:

controller state after initialization
Code:
GICC_CTLR: 00000001 GICC_PMR: 000000f8 GICC_BPR: 00000002
GICC_RPR: 000000ff GICC_HPPIR: 000003ff
GICC_ABPR: 00000003 GICC_AHPPIR: 000003ff

GICD_CTLR: 00000001 GICD_TYPER: 0000fc67 GICD_IGROUPR0: 00000000
GICD_ISENABLER0: 0000ffff GICD_ISPENDR0: 00000000 GICD_ISACTIVER0: 00000000
GICD_IPRIORITYR0: 00000000 GICD_ITARGETSR0: 01010101 GICD_ICFGR0: aaaaaaaa
GICD_PPISR: 00000000 GICD_SPISR0: 00000000 GICD_SPENSGIR0: 00000000
controller state after issuing SGI 4 (interrupt 4 changes state to PENDING)
Code:
GICC_CTLR: 00000001 GICC_PMR: 000000f8 GICC_BPR: 00000002
GICC_RPR: 000000ff GICC_HPPIR: 00000004
GICC_ABPR: 00000003 GICC_AHPPIR: 000003ff

GICD_CTLR: 00000001 GICD_TYPER: 0000fc67 GICD_IGROUPR0: 00000000
GICD_ISENABLER0: 0000ffff GICD_ISPENDR0: 00000010 GICD_ISACTIVER0: 00000000
GICD_IPRIORITYR0: 00000000 GICD_ITARGETSR0: 01010101 GICD_ICFGR0: aaaaaaaa
GICD_PPISR: 00000000 GICD_SPISR0: 00000000 GICD_SPENSGIR0: 00000000
now IRQ exception should fire but it doesn't so i try to read GICC_IAR next
Code:
GICC_IAR: 00000004
after reading IAR interrupt 4 changes state to ACTIVE
Code:
GICC_CTLR: 00000001 GICC_PMR: 000000f8 GICC_BPR: 00000002
GICC_RPR: 00000000 GICC_HPPIR: 000003ff
GICC_ABPR: 00000003 GICC_AHPPIR: 000003ff

GICD_CTLR: 00000001 GICD_TYPER: 0000fc67 GICD_IGROUPR0: 00000000
GICD_ISENABLER0: 0000ffff GICD_ISPENDR0: 00000000 GICD_ISACTIVER0: 00000010
GICD_IPRIORITYR0: 00000000 GICD_ITARGETSR0: 01010101 GICD_ICFGR0: aaaaaaaa
GICD_PPISR: 00000000 GICD_SPISR0: 00000000 GICD_SPENSGIR0: 00000000
after writing IAR value to EOIR
Code:
GICC_CTLR: 00000001 GICC_PMR: 000000f8 GICC_BPR: 00000002
GICC_RPR: 000000ff GICC_HPPIR: 000003ff
GICC_ABPR: 00000003 GICC_AHPPIR: 000003ff

GICD_CTLR: 00000001 GICD_TYPER: 0000fc67 GICD_IGROUPR0: 00000000
GICD_ISENABLER0: 0000ffff GICD_ISPENDR0: 00000000 GICD_ISACTIVER0: 00000000
GICD_IPRIORITYR0: 00000000 GICD_ITARGETSR0: 01010101 GICD_ICFGR0: aaaaaaaa
GICD_PPISR: 00000000 GICD_SPISR0: 00000000 GICD_SPENSGIR0: 00000000

This is my interrupt controller initialization routine:

Code:

GIC400::GIC400(errcode *status, ulong base) :
    base(base)
{
    errcode stat = ESUCCESS;

    // map distributor
    distrAddr = Paging::MapMMIO(&stat, base + 0x1000, (4 << 10));
    if(stat != ESUCCESS)
    {
        if(status) *status = stat;
        return;
    }

    // map CPU interface
    cpuAddr = Paging::MapMMIO(&stat, base + 0x2000, (8 << 10));
    if(stat != ESUCCESS)
    {
        Paging::FreeMMIO(distrAddr);
        if(status) *status = stat;
        return;
    }

    // make sure both CPU interface and distributor are disabled before initialization
    CPU::MMIOWrite32(cpuAddr + GICC_CTLR, 0);
    CPU::MMIOWrite32(distrAddr + GICD_CTLR, 0);

    // get interrupt count
    intCount = ((CPU::MMIORead32(distrAddr + GICD_TYPER) & 0x1F) + 1) << 5;
    if(Debug) Debug::PutFmt("[GIC-400@%p] intCount = %d\n", base, intCount);

    // enable SGIs, disable all PPI and SPIs, clear pending and active ints
    // set all ints to group 0
    for(unsigned i = 0; i < (intCount >> 5); ++i)
    {
        unsigned offs = i << 2;
        if(!i)
        {
            CPU::MMIOWrite32(distrAddr + GICD_ISENABLERn + offs, 0x0000FFFFu);
            CPU::MMIOWrite32(distrAddr + GICD_ICENABLERn + offs, 0xFFFF0000u);
        } else CPU::MMIOWrite32(distrAddr + GICD_ICENABLERn + offs, 0xFFFFFFFFu);
        CPU::MMIOWrite32(distrAddr + GICD_ICPENDRn + offs, 0xFFFFFFFFu);
        CPU::MMIOWrite32(distrAddr + GICD_ICACTIVERn + offs, 0xFFFFFFFFu);
        CPU::MMIOWrite32(distrAddr + GICD_IGROUPRn + offs, 0u);
    }

    // set affinity and priority
    for(unsigned i = 0; i < (intCount >> 2); ++i)
    {
        CPU::MMIOWrite32(distrAddr + GICD_IPRIORITYRn + (i << 2), 0x00000000);
        CPU::MMIOWrite32(distrAddr + GICD_ITARGETSRn + (i << 2), 0x01010101);
    }

    // make all ints level triggered
    for (unsigned i = 0; i < (intCount >> 4); ++i)
        CPU::MMIOWrite32(distrAddr + GICD_ICFGRn + (i << 2), 0);

    // enable distributor
    CPU::MMIOWrite32(distrAddr + GICD_CTLR, 0x1);

    // enable CPU interface
    CPU::MMIOWrite32(cpuAddr + GICC_PMR, 0xFF);
    CPU::MMIOWrite32(cpuAddr + GICC_CTLR, 0x1);

    if(status) *status = ESUCCESS;
}

I tried different IRQBypDisGrp1 and IRQBypDisGrp0 combinations. Nothing works. After all that IRQ is never fired and ISR_EL1 register stays at value 0. Am I missing something in controller initialization, am I using it wrong. Or maybe there are some extra thing to be set in CPU special registers? I have no idea. I've been using similiar code on dual Cortex-A9 (Cyclone V SoC) and it worked.

pvc · **Joined:** Mon Jan 15, 2018 2:27 pm **Posts:** 201

There is some progress.

But now I am even more confused. It seems like there is second interrupt controller in that thing. It looks like the same controller as in BCM2836. I can generate some IRQs using that, but I think I will need to use GIC as well (to handle IRQs from devices like UART, PCIe host, NIC, etc.

pvc · **Joined:** Mon Jan 15, 2018 2:27 pm **Posts:** 201

Problem finally solved. I am posting here so it might be of use to someone else.

GIC-400 controller is present and functional but doesn't seem to be used at all. There are 3 levels of IRQ/FIQ routing.

I and F flags inside the CPU core itself. Just your plain old masking bits. With some extra ELx shenanigans.
Local interrupt controller at address 0xFF800000. This one is the same as in BCM2836 and is responsible for IPIs (called mailbox interrupts here) and routing interrupts between cores. Also has built-in timer.
GPU interrupt controller at address 0xFE00B000. This one seems to be compatible with its counterpart from BCM2835. It is just a simple switch/multiplexer thingy that routes all 64 GPU interrupts to interrupt 8 (GPU interrupt) of local controller.

Any more sophisticated functionallity (that is present in most interrupt controllers; like prioritization or handler selection) has to be done in software in your common IRQ/FIQ handler. Interrupt numbers for devices seem to roughly match those in .dtb file used in Linux for RPI4/BCM2711 (after masking some bits).

EDIT: Turns out that GIC is actually used and working. There is `enable_gic` property available to use in `config.txt` which, when set to 1, well… enables GIC. After doing that, everything works exactly as expected. This property doesn't seem to be documented anywhere.

OSDev.org

[Solved] Can't get IRQs to work on RPI4

Who is online