OSDev.org

The Place to Start for Operating System Developers
It is currently Thu Mar 28, 2024 2:54 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 35 posts ]  Go to page 1, 2, 3  Next
Author Message
 Post subject: PCI Configuration Process
PostPosted: Fri Aug 27, 2021 8:29 am 
Offline
Member
Member
User avatar

Joined: Tue Sep 15, 2020 8:07 am
Posts: 264
Location: London, UK
Hi all,

I'm slowly adding PCI support to my OS, as I want to use USB (and be free of the antiquated PS2 ports!). Anyway, my PCI driver can enumerate the PCI busses, identifiying all the devices/functions along the way using the x86 IOPorts 0xcf8 and 0xcfc. But I am curious about what I can actually do with the information returned.

Several emulators I have tried have something with a VendorID: 0x1234, DeviceID: 0x1111, of class 0x3. I have identified via the magic of Google that this is an emulated CirusLogic 5446 GFX board. Though this would be a good starting point to figuring out what the PCI configuration registers actually mean. I assume I need to start with the BAR registers, now the emulated gfx card has BAR0 set to 0xFD000008 which is very close to the framebuffer address provided by GRUB at boot with address 0xFD000000... So what is BAR0 actually showing me (8bytes/2pixels into my frame buffer?!?!)?

_________________
CuriOS: A single address space GUI based operating system built upon a fairly pure Microkernel/Nanokernel. Download latest bootable x86 Disk Image: https://github.com/h5n1xp/CuriOS/blob/main/disk.img.zip
Discord:https://discord.gg/zn2vV2Su


Top
 Profile  
 
 Post subject: Re: PCI Configuration Process
PostPosted: Fri Aug 27, 2021 8:48 am 
Offline
Member
Member

Joined: Sun Apr 05, 2020 1:01 pm
Posts: 182
bloodline wrote:
Hi all,

I'm slowly adding PCI support to my OS, as I want to use USB (and be free of the antiquated PS2 ports!). Anyway, my PCI driver can enumerate the PCI busses, identifiying all the devices/functions along the way using the x86 IOPorts 0xcf8 and 0xcfc. But I am curious about what I can actually do with the information returned.

Several emulators I have tried have something with a VendorID: 0x1234, DeviceID: 0x1111, of class 0x3. I have identified via the magic of Google that this is an emulated CirusLogic 5446 GFX board. Though this would be a good starting point to figuring out what the PCI configuration registers actually mean. I assume I need to start with the BAR registers, now the emulated gfx card has BAR0 set to 0xFD000008 which is very close to the framebuffer address provided by GRUB at boot with address 0xFD000000... So what is BAR0 actually showing me (8bytes/2pixels into my frame buffer?!?!)?


I suggest you at least read the PCI page here, it describes everything you need to know about both PCI & registers.
Lower 4 bits of a memory BAR are reserved for information, such as whether memory is prefetchable, as well as whether its 64/32/16 bits.
Also I recommend that you use MMIO PCI (ECAM) instead of polling IO registers.


Top
 Profile  
 
 Post subject: Re: PCI Configuration Process
PostPosted: Fri Aug 27, 2021 9:03 am 
Offline
Member
Member
User avatar

Joined: Tue Sep 15, 2020 8:07 am
Posts: 264
Location: London, UK
8infy wrote:
bloodline wrote:
Hi all,

I'm slowly adding PCI support to my OS, as I want to use USB (and be free of the antiquated PS2 ports!). Anyway, my PCI driver can enumerate the PCI busses, identifiying all the devices/functions along the way using the x86 IOPorts 0xcf8 and 0xcfc. But I am curious about what I can actually do with the information returned.

Several emulators I have tried have something with a VendorID: 0x1234, DeviceID: 0x1111, of class 0x3. I have identified via the magic of Google that this is an emulated CirusLogic 5446 GFX board. Though this would be a good starting point to figuring out what the PCI configuration registers actually mean. I assume I need to start with the BAR registers, now the emulated gfx card has BAR0 set to 0xFD000008 which is very close to the framebuffer address provided by GRUB at boot with address 0xFD000000... So what is BAR0 actually showing me (8bytes/2pixels into my frame buffer?!?!)?


I suggest you at least read the PCI page here, it describes everything you need to know about both PCI & registers.
Lower 4 bits of a memory BAR are reserved for information, such as whether memory is prefetchable, as well as whether its 64/32/16 bits.


Ok, I'm glad you pointed that out! Now it all makes sense, Sometimes I just need a nudge in the right direction. Many thanks. Yes, BAR0 just points to the framebuffer and is apparently prefetchable.

Quote:
Also I recommend that you use MMIO PCI (ECAM) instead of polling IO registers.


Well, I don't have ACPI yet... So I think I'm stuck with the old x86 IO port method for now. But I would hope to move to MMIO as soon as possible, but until then I can work on getting PCI drivers working

_________________
CuriOS: A single address space GUI based operating system built upon a fairly pure Microkernel/Nanokernel. Download latest bootable x86 Disk Image: https://github.com/h5n1xp/CuriOS/blob/main/disk.img.zip
Discord:https://discord.gg/zn2vV2Su


Top
 Profile  
 
 Post subject: Re: PCI Configuration Process
PostPosted: Fri Aug 27, 2021 9:04 am 
Offline
Member
Member

Joined: Tue Feb 18, 2020 3:29 pm
Posts: 1071
As 8infy said, the low 4 bits need to be masked off. There are various other complexities when reading BARs, meaning that you should read the PCI page, or, even better, buy the book PCI systems architecture. Or pay $1000 dollars to PCI-SIG to get the specs :) .

_________________
"How did you do this?"
"It's very simple — you read the protocol and write the code." - Bill Joy
Projects: NexNix | libnex | nnpkg


Top
 Profile  
 
 Post subject: Re: PCI Configuration Process
PostPosted: Fri Aug 27, 2021 9:06 am 
Offline
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 5099
bloodline wrote:
Several emulators I have tried have something with a VendorID: 0x1234, DeviceID: 0x1111, of class 0x3. I have identified via the magic of Google that this is an emulated CirusLogic 5446 GFX board.

Your Google doesn't have enough magic. That's a Bochs graphics adapter. The Cirrus adapter is 1013:00B8.


Top
 Profile  
 
 Post subject: Re: PCI Configuration Process
PostPosted: Fri Aug 27, 2021 9:19 am 
Offline
Member
Member
User avatar

Joined: Tue Sep 15, 2020 8:07 am
Posts: 264
Location: London, UK
Octocontrabass wrote:
bloodline wrote:
Several emulators I have tried have something with a VendorID: 0x1234, DeviceID: 0x1111, of class 0x3. I have identified via the magic of Google that this is an emulated CirusLogic 5446 GFX board.

Your Google doesn't have enough magic. That's a Bochs graphics adapter. The Cirrus adapter is 1013:00B8.


:shock: As usual, I'm glad I posted here first, cheers guys! :D

_________________
CuriOS: A single address space GUI based operating system built upon a fairly pure Microkernel/Nanokernel. Download latest bootable x86 Disk Image: https://github.com/h5n1xp/CuriOS/blob/main/disk.img.zip
Discord:https://discord.gg/zn2vV2Su


Top
 Profile  
 
 Post subject: Re: PCI Configuration Process
PostPosted: Fri Aug 27, 2021 10:53 am 
Offline
Member
Member

Joined: Sun Jun 23, 2019 5:36 pm
Posts: 618
Location: North Dakota, United States
You can do a lot with the information provides you. Essentially, the PCI configuration process goes something like this:
  1. PCI device enumerator scans all PCI devices (including those behind bridges and such).
  2. A driver (say, a GFX adapter driver) loads and wants to access an intel GPU in particular. Lets assume that this intel GPU is a skylake GPU. The driver would do the following:
    1. The driver would begin scanning each discovered PCI device in the system.
    2. The driver would check the base class code (BCC), sub-class code (SCC), and program interface (PI) to determine the type of device. In this instance, for the BCC the driver would check for 03h or 04h, meaning a display controller or multimedia device, respectively. For the SCC, the driver would check for either 00h or 80h, meaning a VGA-compatible device or other multimedia device, respectively. And for the PI, the driver would check for 00h.
    3. The driver could perform other checks, such as scanning the vendor and device IDs as well.
    4. Once the driver has determined that it can handle the device in question, it would scan the BARs and map them. It might also enable bus mastering, interrupts, etc.
  3. Now that the driver has determined it can handle the device and has performed any other necessary PCI configuration steps that it needs, it can access the devices registers and they will be automatically translated into PCI configuration accesses.
So, in sum, any driver can just:
  1. Enumerate PCI devices
  2. Scan the base class code, sub-class code, and program interface, if applicable
  3. Scan other PCI device properties if applicable
  4. Perform any PCI configuration required
  5. Map the BAR ranges
  6. Ready to drive the device
HTH


Top
 Profile  
 
 Post subject: Re: PCI Configuration Process
PostPosted: Fri Aug 27, 2021 12:15 pm 
Offline
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 5099
Isn't that a bit backwards?

Usually you would enumerate devices, then use the vendor:device ID to look up and load compatible drivers, then if you don't find a driver that way use the class:subclass:IF to look up and load compatible drivers. I don't see why you would load a driver without knowing which device you want it to drive.


Top
 Profile  
 
 Post subject: Re: PCI Configuration Process
PostPosted: Fri Aug 27, 2021 2:46 pm 
Offline
Member
Member
User avatar

Joined: Tue Sep 15, 2020 8:07 am
Posts: 264
Location: London, UK
Ethin wrote:
You can do a lot with the information provides you. Essentially, the PCI configuration process goes something like this:
  1. PCI device enumerator scans all PCI devices (including those behind bridges and such).
  2. A driver (say, a GFX adapter driver) loads and wants to access an intel GPU in particular. Lets assume that this intel GPU is a skylake GPU. The driver would do the following:
    1. The driver would begin scanning each discovered PCI device in the system.
    2. The driver would check the base class code (BCC), sub-class code (SCC), and program interface (PI) to determine the type of device. In this instance, for the BCC the driver would check for 03h or 04h, meaning a display controller or multimedia device, respectively. For the SCC, the driver would check for either 00h or 80h, meaning a VGA-compatible device or other multimedia device, respectively. And for the PI, the driver would check for 00h.
    3. The driver could perform other checks, such as scanning the vendor and device IDs as well.
    4. Once the driver has determined that it can handle the device in question, it would scan the BARs and map them. It might also enable bus mastering, interrupts, etc.


Fantastic summary! I appreciate you taking the time to bullet point it for me.
I’m still fuzz as to how exactly the BARs work. As my PCI driver builds a database of what it finds on the PCI BUS, I notice that some devices have their BAR registers filled with values, some just have zeros. Am I able to just use whatever value they have set, or should I write my own values there, I assume the documentation for the devices will explain what each BAR points to!?



Quote:
  • Now that the driver has determined it can handle the device and has performed any other necessary PCI configuration steps that it needs, it can access the devices registers and they will be automatically translated into PCI configuration accesses.
  • So, in sum, any driver can just:
    1. Enumerate PCI devices
    2. Scan the base class code, sub-class code, and program interface, if applicable
    3. Scan other PCI device properties if applicable
    4. Perform any PCI configuration required
    5. Map the BAR ranges
    6. Ready to drive the device
    HTH

    _________________
    CuriOS: A single address space GUI based operating system built upon a fairly pure Microkernel/Nanokernel. Download latest bootable x86 Disk Image: https://github.com/h5n1xp/CuriOS/blob/main/disk.img.zip
    Discord:https://discord.gg/zn2vV2Su


    Top
     Profile  
     
     Post subject: Re: PCI Configuration Process
    PostPosted: Fri Aug 27, 2021 3:09 pm 
    Offline
    Member
    Member

    Joined: Tue Feb 18, 2020 3:29 pm
    Posts: 1071
    Not every PCI device has filled BARs. it normally will be up to each driver which BARs should be filled or not. The PCI driver should only hand out BARs, not use them.

    Also, device docs should contains which BARs they use (in theory).

    _________________
    "How did you do this?"
    "It's very simple — you read the protocol and write the code." - Bill Joy
    Projects: NexNix | libnex | nnpkg


    Top
     Profile  
     
     Post subject: Re: PCI Configuration Process
    PostPosted: Fri Aug 27, 2021 3:34 pm 
    Offline
    Member
    Member

    Joined: Mon Mar 25, 2013 7:01 pm
    Posts: 5099
    bloodline wrote:
    I’m still fuzz as to how exactly the BARs work. As my PCI driver builds a database of what it finds on the PCI BUS, I notice that some devices have their BAR registers filled with values, some just have zeros. Am I able to just use whatever value they have set, or should I write my own values there, I assume the documentation for the devices will explain what each BAR points to!?

    During boot, PC firmware enumerates PCI and assigns reasonable default values to the BARs. You can use those values. You don't need to write your own values unless you're working with PCI hotplug or certain non-PC hardware.

    The type (memory or I/O) and amount of the resources assigned to each BAR are fixed in hardware. For example, a device that needs 1 MiB of MMIO will have a BAR that only allows memory addresses aligned to 1 MiB boundaries. This is how the firmware (and you) can assign resources without knowing anything about the device.

    When you're writing drivers for PCI devices, the documentation for each device should explain the purpose of that device's BARs.


    Top
     Profile  
     
     Post subject: Re: PCI Configuration Process
    PostPosted: Fri Aug 27, 2021 5:10 pm 
    Offline
    Member
    Member

    Joined: Sun Jun 23, 2019 5:36 pm
    Posts: 618
    Location: North Dakota, United States
    Octocontrabass wrote:
    Isn't that a bit backwards?

    Usually you would enumerate devices, then use the vendor:device ID to look up and load compatible drivers, then if you don't find a driver that way use the class:subclass:IF to look up and load compatible drivers. I don't see why you would load a driver without knowing which device you want it to drive.

    I mean, that's another way to do it, but either way works I imagine.


    Top
     Profile  
     
     Post subject: Re: PCI Configuration Process
    PostPosted: Sat Aug 28, 2021 12:02 am 
    Offline
    Member
    Member

    Joined: Wed Aug 30, 2017 8:24 am
    Posts: 1593
    Octocontrabass wrote:
    Isn't that a bit backwards?

    Usually you would enumerate devices, then use the vendor:device ID to look up and load compatible drivers, then if you don't find a driver that way use the class:subclass:IF to look up and load compatible drivers. I don't see why you would load a driver without knowing which device you want it to drive.

    This requires loadable drivers. I think it is reasonable to not start out with those. But yes, it is reasonable to have the PCI enumerator find the correct driver, rather than having the driver find the PCI device.

    _________________
    Carpe diem!


    Top
     Profile  
     
     Post subject: Re: PCI Configuration Process
    PostPosted: Sat Aug 28, 2021 4:33 am 
    Offline
    Member
    Member

    Joined: Tue Feb 18, 2020 3:29 pm
    Posts: 1071
    I have a pretty sophisticated plan for my driver manager. Basically, it will start by loading the ACPI driver to parse the motherboard's basic hardware. The ACPI driver will report devices to the driver managers, which will then try to find a driver and load the driver for it. The ACPI driver will also report buses. Once the ACPI driver finishes enumerating, the driver manager will load and run each bus driver for every bus on the system. This will result in the PCI bus finding devices, and telling the driver manager about them, along with a hint as to how it should load up the driver (i.e, vendorID:classID, or class:subclass:progIF). It will do the same for buses on the PCI bus (i.e, USB, AHCI, SCSI). Any devices the driver manager couldn't find a driver for will be put in a list, and then the user will be told about them at logon.

    _________________
    "How did you do this?"
    "It's very simple — you read the protocol and write the code." - Bill Joy
    Projects: NexNix | libnex | nnpkg


    Top
     Profile  
     
     Post subject: Re: PCI Configuration Process
    PostPosted: Thu Sep 09, 2021 3:34 am 
    Offline
    Member
    Member
    User avatar

    Joined: Tue Sep 15, 2020 8:07 am
    Posts: 264
    Location: London, UK
    Octocontrabass wrote:
    bloodline wrote:
    I’m still fuzz as to how exactly the BARs work. As my PCI driver builds a database of what it finds on the PCI BUS, I notice that some devices have their BAR registers filled with values, some just have zeros. Am I able to just use whatever value they have set, or should I write my own values there, I assume the documentation for the devices will explain what each BAR points to!?

    During boot, PC firmware enumerates PCI and assigns reasonable default values to the BARs. You can use those values. You don't need to write your own values unless you're working with PCI hotplug or certain non-PC hardware.

    The type (memory or I/O) and amount of the resources assigned to each BAR are fixed in hardware. For example, a device that needs 1 MiB of MMIO will have a BAR that only allows memory addresses aligned to 1 MiB boundaries. This is how the firmware (and you) can assign resources without knowing anything about the device.

    When you're writing drivers for PCI devices, the documentation for each device should explain the purpose of that device's BARs.


    So yeah, I'm happy to let the firmware set the BARs to whatever works.

    During PCI Bus Enumeration I can find the gfx board which Qemu emulates, and I'm going to use that to test writing PCI based driver code. I've found the Tech Doc for the QEmu graphics chip (the Cirrus Logic GD5446), http://www.vgamuseum.info/index.php/cpu ... 42814695bc

    On page 67, this helpfully (though vaguely) lists all the available hardware regs, but I'm confused as to how to access them. The table give an I/O Port address, some of which appear to match up with the PCI Configuration register address/offsets. I'm having trouble picturing how this works as I'm used to hardware registers just sitting in the normal address space.

    -Edit- I might have jumped the gun a bit here... So BAR0 is the framebuffer address. BAR1 is 0, but BAR2 is an address 32megs above the framebuffer... Is it possible the hardware registers are mapped here?

    _________________
    CuriOS: A single address space GUI based operating system built upon a fairly pure Microkernel/Nanokernel. Download latest bootable x86 Disk Image: https://github.com/h5n1xp/CuriOS/blob/main/disk.img.zip
    Discord:https://discord.gg/zn2vV2Su


    Top
     Profile  
     
    Display posts from previous:  Sort by  
    Post new topic Reply to topic  [ 35 posts ]  Go to page 1, 2, 3  Next

    All times are UTC - 6 hours


    Who is online

    Users browsing this forum: No registered users and 58 guests


    You cannot post new topics in this forum
    You cannot reply to topics in this forum
    You cannot edit your posts in this forum
    You cannot delete your posts in this forum
    You cannot post attachments in this forum

    Search for:
    Jump to:  
    Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group