OSDev.org

The Place to Start for Operating System Developers
It is currently Fri Aug 07, 2020 9:16 pm

All times are UTC - 6 hours




Post new topic Reply to topic  [ 3 posts ] 
Author Message
 Post subject: [Solved] Can't get IRQs to work on RPI4
PostPosted: Wed Jan 08, 2020 4:19 pm 
Offline
Member
Member
User avatar

Joined: Mon Jan 15, 2018 2:27 pm
Posts: 126
I am trying to get IRQs to work on Raspberry Pi 4. Without much success. One of few things I know about it (thank you Broadcom for being absolute d :evil: cks about documentation) is that it uses GIC-400 interrupt controller.

CPU is running in secure EL1 mode (AArch64).
A, I and F bits in PSTATE are all clear.
Interrupt handling state machine seems to be working as supposed.

Here is an example:
  • controller state after initialization
    Code:
    GICC_CTLR: 00000001 GICC_PMR:    000000f8 GICC_BPR: 00000002
    GICC_RPR:  000000ff GICC_HPPIR:  000003ff
    GICC_ABPR: 00000003 GICC_AHPPIR: 000003ff

    GICD_CTLR:        00000001 GICD_TYPER:      0000fc67 GICD_IGROUPR0:   00000000
    GICD_ISENABLER0:  0000ffff GICD_ISPENDR0:   00000000 GICD_ISACTIVER0: 00000000
    GICD_IPRIORITYR0: 00000000 GICD_ITARGETSR0: 01010101 GICD_ICFGR0:     aaaaaaaa
    GICD_PPISR:       00000000 GICD_SPISR0:     00000000 GICD_SPENSGIR0:  00000000
  • controller state after issuing SGI 4 (interrupt 4 changes state to PENDING)
    Code:
    GICC_CTLR: 00000001 GICC_PMR:    000000f8 GICC_BPR: 00000002
    GICC_RPR:  000000ff GICC_HPPIR:  00000004
    GICC_ABPR: 00000003 GICC_AHPPIR: 000003ff

    GICD_CTLR:        00000001 GICD_TYPER:      0000fc67 GICD_IGROUPR0:   00000000
    GICD_ISENABLER0:  0000ffff GICD_ISPENDR0:   00000010 GICD_ISACTIVER0: 00000000
    GICD_IPRIORITYR0: 00000000 GICD_ITARGETSR0: 01010101 GICD_ICFGR0:     aaaaaaaa
    GICD_PPISR:       00000000 GICD_SPISR0:     00000000 GICD_SPENSGIR0:  00000000
  • now IRQ exception should fire but it doesn't so i try to read GICC_IAR next
    Code:
    GICC_IAR: 00000004
  • after reading IAR interrupt 4 changes state to ACTIVE
    Code:
    GICC_CTLR: 00000001 GICC_PMR:    000000f8 GICC_BPR: 00000002
    GICC_RPR:  00000000 GICC_HPPIR:  000003ff
    GICC_ABPR: 00000003 GICC_AHPPIR: 000003ff

    GICD_CTLR:        00000001 GICD_TYPER:      0000fc67 GICD_IGROUPR0:   00000000
    GICD_ISENABLER0:  0000ffff GICD_ISPENDR0:   00000000 GICD_ISACTIVER0: 00000010
    GICD_IPRIORITYR0: 00000000 GICD_ITARGETSR0: 01010101 GICD_ICFGR0:     aaaaaaaa
    GICD_PPISR:       00000000 GICD_SPISR0:     00000000 GICD_SPENSGIR0:  00000000
  • after writing IAR value to EOIR
    Code:
    GICC_CTLR: 00000001 GICC_PMR:    000000f8 GICC_BPR: 00000002
    GICC_RPR:  000000ff GICC_HPPIR:  000003ff
    GICC_ABPR: 00000003 GICC_AHPPIR: 000003ff

    GICD_CTLR:        00000001 GICD_TYPER:      0000fc67 GICD_IGROUPR0:   00000000
    GICD_ISENABLER0:  0000ffff GICD_ISPENDR0:   00000000 GICD_ISACTIVER0: 00000000
    GICD_IPRIORITYR0: 00000000 GICD_ITARGETSR0: 01010101 GICD_ICFGR0:     aaaaaaaa
    GICD_PPISR:       00000000 GICD_SPISR0:     00000000 GICD_SPENSGIR0:  00000000

This is my interrupt controller initialization routine:
Code:
GIC400::GIC400(errcode *status, ulong base) :
    base(base)
{
    errcode stat = ESUCCESS;

    // map distributor
    distrAddr = Paging::MapMMIO(&stat, base + 0x1000, (4 << 10));
    if(stat != ESUCCESS)
    {
        if(status) *status = stat;
        return;
    }

    // map CPU interface
    cpuAddr = Paging::MapMMIO(&stat, base + 0x2000, (8 << 10));
    if(stat != ESUCCESS)
    {
        Paging::FreeMMIO(distrAddr);
        if(status) *status = stat;
        return;
    }

    // make sure both CPU interface and distributor are disabled before initialization
    CPU::MMIOWrite32(cpuAddr + GICC_CTLR, 0);
    CPU::MMIOWrite32(distrAddr + GICD_CTLR, 0);

    // get interrupt count
    intCount = ((CPU::MMIORead32(distrAddr + GICD_TYPER) & 0x1F) + 1) << 5;
    if(Debug) Debug::PutFmt("[GIC-400@%p] intCount = %d\n", base, intCount);

    // enable SGIs, disable all PPI and SPIs, clear pending and active ints
    // set all ints to group 0
    for(unsigned i = 0; i < (intCount >> 5); ++i)
    {
        unsigned offs = i << 2;
        if(!i)
        {
            CPU::MMIOWrite32(distrAddr + GICD_ISENABLERn + offs, 0x0000FFFFu);
            CPU::MMIOWrite32(distrAddr + GICD_ICENABLERn + offs, 0xFFFF0000u);
        } else CPU::MMIOWrite32(distrAddr + GICD_ICENABLERn + offs, 0xFFFFFFFFu);
        CPU::MMIOWrite32(distrAddr + GICD_ICPENDRn + offs, 0xFFFFFFFFu);
        CPU::MMIOWrite32(distrAddr + GICD_ICACTIVERn + offs, 0xFFFFFFFFu);
        CPU::MMIOWrite32(distrAddr + GICD_IGROUPRn + offs, 0u);
    }

    // set affinity and priority
    for(unsigned i = 0; i < (intCount >> 2); ++i)
    {
        CPU::MMIOWrite32(distrAddr + GICD_IPRIORITYRn + (i << 2), 0x00000000);
        CPU::MMIOWrite32(distrAddr + GICD_ITARGETSRn + (i << 2), 0x01010101);
    }

    // make all ints level triggered
    for (unsigned i = 0; i < (intCount >> 4); ++i)
        CPU::MMIOWrite32(distrAddr + GICD_ICFGRn + (i << 2), 0);

    // enable distributor
    CPU::MMIOWrite32(distrAddr + GICD_CTLR, 0x1);

    // enable CPU interface
    CPU::MMIOWrite32(cpuAddr + GICC_PMR, 0xFF);
    CPU::MMIOWrite32(cpuAddr + GICC_CTLR, 0x1);

    if(status) *status = ESUCCESS;
}


I tried different IRQBypDisGrp1 and IRQBypDisGrp0 combinations. Nothing works. After all that IRQ is never fired and ISR_EL1 register stays at value 0. Am I missing something in controller initialization, am I using it wrong. Or maybe there are some extra thing to be set in CPU special registers? I have no idea. I've been using similiar code on dual Cortex-A9 (Cyclone V SoC) and it worked.

_________________
https://github.com/pvc988/woot64
https://github.com/pvc988/wootc


Last edited by pvc on Thu Jan 09, 2020 6:03 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Can't get IRQs to work on RPI4
PostPosted: Thu Jan 09, 2020 10:55 am 
Offline
Member
Member
User avatar

Joined: Mon Jan 15, 2018 2:27 pm
Posts: 126
There is some progress.

But now I am even more confused. It seems like there is second interrupt controller in that thing. It looks like the same controller as in BCM2836. I can generate some IRQs using that, but I think I will need to use GIC as well (to handle IRQs from devices like UART, PCIe host, NIC, etc.

_________________
https://github.com/pvc988/woot64
https://github.com/pvc988/wootc


Top
 Profile  
 
 Post subject: Re: Can't get IRQs to work on RPI4
PostPosted: Thu Jan 09, 2020 5:56 pm 
Offline
Member
Member
User avatar

Joined: Mon Jan 15, 2018 2:27 pm
Posts: 126
Problem finally solved. I am posting here so it might be of use to someone else.

GIC-400 controller is present and functional but doesn't seem to be used at all. There are 3 levels of IRQ/FIQ routing.
  • I and F flags inside the CPU core itself. Just your plain old masking bits. With some extra ELx shenanigans.
  • Local interrupt controller at address 0xFF800000. This one is the same as in BCM2836 and is responsible for IPIs (called mailbox interrupts here) and routing interrupts between cores. Also has built-in timer.
  • GPU interrupt controller at address 0xFE00B000. This one seems to be compatible with its counterpart from BCM2835. It is just a simple switch/multiplexer thingy that routes all 64 GPU interrupts to interrupt 8 (GPU interrupt) of local controller.

Any more sophisticated functionallity (that is present in most interrupt controllers; like prioritization or handler selection) has to be done in software in your common IRQ/FIQ handler. Interrupt numbers for devices seem to roughly match those in .dtb file used in Linux for RPI4/BCM2711 (after masking some bits).

EDIT: Turns out that GIC is actually used and working. There is `enable_gic` property available to use in `config.txt` which, when set to 1, well… enables GIC. After doing that, everything works exactly as expected. This property doesn't seem to be documented anywhere.

_________________
https://github.com/pvc988/woot64
https://github.com/pvc988/wootc


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 3 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: clementttttttttt, ITchimp and 7 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group