OSDev.org

Hi,

I'm currently in a process of writing some HDD/ODD drivers, the problem is they're all mixed up.
There are: ATA, SATA, IDE, ACHI, ATAPI, PATA. How many different drivers do I need to support all of them?
Is my understanding of them correct?
IDE and ACHI = Controllers, ICs that need to be told what to do
ATA (same as PATA?) and SATA = Cables (aka interfaces)
ATAPI = special controller (or not?), same as ATA but for optical drives

More confusion: IDE can talk to ATA (PATA?) and ATAPI, but ACHI can talk to SATA and ATA (PATA?) and ATAPI???
So I need like 3 different drivers?

Hi,

I have two different drivers, one for the ATA which includes ATAPI and standard SATA, and one for AHCI which is HBA enabled SATA controllers. However, they have to be somewhat aware of each other.

My ATA driver handles ATA as well as ATAPI. Once a packet device is detected, it is a simple task to set a flag and then use this flag throughout the driver to indicate whether I am using the ATA registers or a packet interface.

Some SATA drives and controllers handle just like ATA controllers until you tell them otherwise. Some are ATA interface, period, with a few extra status registers.

An (AHCI) SATA controller, that has legacy support, will act just like an ATA controller until you set a bit enabling the HBA interface. Then your AHCI driver must take control. Some (AHCI) SATA controllers are AHCI (HBA) only...no legacy.

First enumerate the PCI, which will tell you if it is a SATA controller or not, detect if it has an HBA interface, then enable it if so. If not, pass it on to the ATA driver. If your PCI enumeration detects an ATA controller, pass it on as well. Finally, if you care about ISA stuff, detect if there are ATA controllers at the default ISA addresses, though making sure that you haven't already detected that controller via PCI.

Ben
- http://www.fysnet.net/media_storage_devices.htm

Hi,



Once upon a time there was the original version of IDE; and over time this evolved through multiple versions of the specification (e.g. "IDE", "ATA-1", ""ATA-2", ..., "ATA/ATAPI-7") with each new version of the specification gaining new things (and in some cases deprecating older things). This series of specifications primarily covers the interface between a controller and a device - the interface between software and controller is only a secondary concern and you may need additional specs (e.g. for chipset or controller), especially when you start getting to bus mastering engines that got slapped on top. However, because there's a lot of similarities between versions (and between controllers) a single driver (with a lot of "if(version > " and "if( is_supported(feature)..") should be enough. Note that "PATA" is a pseudonym and is not different to IDE/ATA - when SATA was invented people created the name "PATA" ("Parallel ATA") for the older IDE/ATA to distinguish it from the newer SATA ("Serial ATA").

SATA is the hardware interface between a SATA controller and a SATA device; and AHCI is a software interface between a driver and a SATA controller. To support SATA/AHCI you need one driver; but there can be other types of SATA controllers (mainly only SATA RAID controllers) that don't use AHCI and will need a different driver.

Note that even though SATA has it's roots in (parallel) IDE/ATA/ATAPI; the software interface is so different that there's no point trying to cover them both with the same driver.

Essentially; for what you've listed, you might only want 2 drivers if you don't support any SATA RAID controllers; but if you do support SATA RAID controllers (including those operating in "just a bunch of disks" mode without any actual RAID being used) then it's probably more like 50 drivers.

For other stuff:

• There is also SCSI, which has a similar history. E.g. the original "parallel SCSI" with multiple versions of evolving specifications (where each version adds more things and deprecates some old things) for the interface between a controller and SCSI devices that leads up to modern "serial attached SCSI" (SAS). Unfortunately for SCSI there was never any kind of standard for the interface between software and the controller; so you'll probably need a different driver for each different SCSI controller.
• For USB; there's multiple separate standards for the USB controller (UHCI, OHCI, EHCI, xHCI), multiple versions of the "USB mass storage device" specification, and probably a few versions of the "USB floppy drive" specification
• For floppy drives there's a few variations of the "8272 floppy drive controller chip" which are very similar anyway (plus the newer USB floppy drives). You'd need one driver for all versions of the old "floppy drive controller chip" (if you bother at all).
• There are also tape drives for all of the above (for SCSI, IDE/ATA, SATA, USB and for the floppy drive controller); where some use standards and some don't.
• Before ATAPI existed there were 3 different proprietary CD-drive interfaces (named after the each of the companies that created them - Sony, Panasonic and Creative labs) which were mostly built into sound cards in the early 1990s. These aren't really worth bothering with now (like floppy - too obsolete), but if you bother then you'll probably need to built support into different ISA sound card drivers.
• NVMe is a very modern software interface for high-speed solid state drives (that connect directly to PCI-E or to an M.2. slot) which is a little unique in that there's no distinction between "controller" and "device" (it's like there's no controller and the device just has a software interface of its own). There were also a few high-end SSD using non-standard interfaces before NVMe was standardised (where you'd need a device driver for each non-standard interface if you want to support them).

For "all storage devices" (including all the obsolete stuff) you'd probably be looking at maybe around 300 different drivers.

Of course some of these are very common (IDE/ATA, AHCI, USB and NVMe), so (especially for desktop/laptop) you can probably cover 99% of storage devices people care about with as few as 8 drivers (and then ignore the other ~292 drivers until you've got nothing better to do).


Cheers,

Brendan

There are two common interfaces you can use to communicate with drives: IDE and AHCI.

There are two common command sets that the connected drive may understand: ATA and ATAPI.

How many separate drivers you need is up to you. For example, you might put the ATA and ATAPI stuff in libraries, so your IDE, AHCI, and other drivers can share that code.


IDE, ATA, and PATA are somewhat interchangeable terms since the communication interface, command set, and physical connection are all mostly defined by the same standard.

I think I managed to put this all together, in my head. Right now I don't really need anything fancy, so 2 drivers will do. (although I'm interested in NVMe)
Looks like ATA and AHCI are the things (software interfaces, command sets) that I need to worry about.
So IDE is just a cable signaling interface, while ATA describes all the commands and is the whole specification itself. SATA is a signaling interface just as IDE, governed by AHCI.
That looks sane, right?
Thank you all for helping me understand this mess.

Hi,



Think of it like this..

Originally Western Digital (with cooperation from a few other companies) created the IDE standard; which covered the signalling between a controller and a drive, the commands a drive accepts, the 40-pin plugs and sockets and the physical size of the drive and where the (threaded) mounting holes are. Then it was shifted over to a standardisation committee (INCITS) who renamed it to "ATA", so the second version of the IDE standard was "ATA-1". Later they added ATAPI to it and renamed it to "ATA/ATAPI", then even later they converted it from a parallel interface to a serial interface and renamed it to "SATA". Essentially, it's all a continuation of the same evolving standard from IDE to ATA to ATA/ATAPI to SATA.

None of this covers the interface between software and a controller; and (at least in theory) any type of computer (MIPS, ARM, PowerPC, ...) could have a completely different type of controller that uses the exact same (IDE, ATA, SATA) specification for the disk drives themselves.

For specifications for the communication between software and a controller; for 80x86 PCs there wasn't really a name for it (and wasn't really an official specification either) so people just called it whatever they felt like (e.g. "the PC's IDE/ATA controller software interface"), and (fortunately) the controllers from different manufacturers mostly kept backward compatibility as they evolved so there wasn't much ambiguity. Then SATA came along and instead of just keeping the same "the PC's IDE/ATA controller software interface" Intel felt like doing something better, so they created a completely different software interface, made it a standard, and gave their new software interface an official name (AHCI).

Note that it is entirely possible for a controller to use the old "IDE/ATA controller software interface" for SATA devices, and entirely possible for a SATA controller to support both the old "IDE/ATA controller software interface" and AHCI (and anything else it feels like). For backward compatibility this is exactly what did happen (typically with a setting in the BIOS config to control which software interface the firmware configured the controller to provide). In the same way (in theory) it'd be possible for a controller to provide AHCI for old parallel ATA devices, it's just that no chipset has bothered to do that.


Cheers,

Brendan

Sure there is: IDE.

You could also go with "AT-compatible" for systems that predate IDE, since it follows the interface laid out by IBM for the PC AT Fixed Disk Adapter.


The ATA-1 standard all but explicitly states how the PC should communicate with the disk, and only barely avoids mentioning specific I/O port addresses or (ISA) DMA channels. I suspect these values were explicitly mentioned in the IDE or EIDE standards, which would explain why the name "IDE" caught on, but I haven't been able to find a copy of either to check. There's also the PC AT Fixed Disk Adapter manual, if you want to look way back.

When PCI was introduced, these were then explicitly standardized in two documents titled "PCI IDE Controller Specification" and "Programming Interface for Bus Master IDE Controller" (which replaced ISA DMA with PCI DMA, for obvious reasons).


You might see an old PATA disk hooked up through an adapter, though.

Hi,



So you're saying that (because there never was an official name for the software interface equivalent to AHCI for the software interface used by most 80x86 SATA controllers) you can use "IDE" or "AT-compatible" for the name of the software interface?



No, it doesn't. What it does do is describe the registers in the disk drives, and early controllers mostly just passed information between IO port and the registers in the drive/s (because it was all "integrated in the drive's electronics"), so by pure luck/history what you see in the early IDE and ATA specifications looks like a software interface (but isn't a software interface and was never intended to be a software interface).

Please understand that I'm trying to separate software interfaces and hardware interfaces so that people (Octane) are more able to understand the relationships between different standards, and to make it easier for people to understand things like (e.g.) IDE controllers for "non-PC" computers (like Amiga, which never had IO ports at all), and ATA RAID controllers, and SCSI. Note that this same "software interface != hardware interface" applies to almost everything (e.g. USB vs. OCHI/UHCI/EHCI/xHCI, Ethernet vs. lots of non-standard software interfaces for network cards, RS232 vs. "the software interface a 65535 chip provides", PS/2 vs. "the software interface an 8042 chip provides, etc); and is why (e.g.) PCI configuration space has a "programming interface" field.


Cheers,

Brendan

Yes. Most specifications refer to the register-level interface as IDE, so calling it that is the best way to stay consistent with those documents.


It was specifically designed to be a software interface! It's no coincidence that ISA IDE adapters are nothing more than address decoding logic.

Even when it's connected to non-x86 computers, it's still unmistakably IDE. Adapting a PC-centric IDE driver to work with an Amiga IDE controller requires merely changing the addressing logic. It's address translation, and an OS developer should already be familiar with that.


Then let's separate them. Call the software interfaces IDE and AHCI, the command sets ATA and ATAPI, and the hardware interfaces PATA and SATA.

Now, from the perspective of an OS developer, you need an IDE driver and an AHCI driver, and each of those drivers needs to be able to speak both ATA and ATAPI, and you don't need to worry about PATA or SATA because the differences are hidden from software.

Hi,



No, most specifications use "random whatever the author felt like", in some cases only using "IDE" for everything and not using "ATA" at all, and in some cases mixing both. For example, the first chapter of Intel's "Programming Interface for Bus Master IDE Controller" says:

Quote:
1.0. IDE Bus Master

This document defines a register level programming interface for a bus master ATA compatible (IDE) disk controller that directly moves data between IDE devices and main memory. By performing the IDE data transfer as a bus master, the Bus Master Device offloads the CPU (no programmed IO for data transfer) and improves system performance in multitasking environments.

It's obvious from the very first sentence (e.g. "ATA compatible (IDE) disk controller") that they treating "ATA compatible" and "IDE" as synonyms.



No. It was designed to be the interface a disk drive provides, and it was designed for the controller to be mostly just address decoding; and these things combined caused it to become a crude software interface by accident. It was not the opposite (where they designed a software interface that accidentally ended up being the interface a disk drive provides).



No. The software interface originally used on 80x86 PCs (where there's IO ports at fixed bases and an ISA IRQ numbers) simply can not make any sense on anything else (that doesn't have IO ports and doesn't have ISA IRQs), and attempting to pretend the software interface remains the same in these cases (when that's obviously impossible) is extremely idiotic.



No. ATA is just the successor to IDE, and both IDE and ATA covered the communication between a controller and a drive (and both include electronic signalling and commands). Calling IDE the software interface and ATA the command set is as wrong as saying that EFI is the core of the firmware and UEFI is a set of functions, or as wrong as saying that Windows 10 is a GUI (a user interface) that runs on top of Windows 8 (the underlying OS).


Cheers,

Brendan

Right. And since they use the term "IDE" when they're referring to the register-level interface, you can easily find this specification by searching for things like "IDE controllers".


It was designed to move the controller into the drive, where the controller provides the same software interface as the IBM PC AT Fixed Disk Adapter.


I think we might disagree on what constitutes a software interface. According to you, it is not the same software interface if you change how the device in question is addressed. (Does this mean changing a PCI BAR changes the software interface?) I consider the software interface to be the series of reads and writes that are used to interact with a piece of hardware, where the locations being read or written can be mapped at any address, since the mapping has no bearing on driver design. The difference between MMIO and port IO is a single address bit. Most drivers should have no say in where their hardware is mapped.

It's much the same with IRQs, where most drivers should have no say in which IRQ is mapped to the device.


Being wrong according to the original definition does not make it wrong according to the commonly-used definitions.

Hi,



I wasn't able to find any (low level) programming information for any of the Amiga IDE controllers; but let's assume that there was one address where you write "selected register number" and another address where you read or write "data to/from the selected register"; and that DMA simply didn't work (because Amiga had 2 CPUs so there wouldn't be much point providing an equivalent to the ISA DMA controller chip that PC used back then) and polling had to be used (because the Amiga only had one remaining "int6" IRQ line and an IDE controller uses two IRQs, one for each channel). Would say that this (hypothetical but not implausible) software interface is exactly the same as the software interface that was used on 80x86 PCs at the time?



Being wrong according to the original definition makes it wrong. If it's commonly wrong then it's still wrong; but there aren't any statistics to measure how far the wrongness actually has spread and (at least for people that matter - e.g. Intel) I don't think wrong definitions are commonly used.

For example, Wikipedia (which suffices as an indicator of "common usage" until something better comes along) does not use your wrong definitions. In fact the Wikipedia page for (Parallel) ATA says (from 2nd paragraph, highlighting mine): "The ATA interface itself evolved in several stages from Western Digital's original Integrated Drive Electronics (IDE) interface. As a result, many near-synonyms for ATA/ATAPI and its previous incarnations are still in common informal use".


Cheers,

Brendan

No, but only because you've changed the register access. An IDE driver for IBM-compatible PCs in the same time period would still need to be prepared to handle IRQ sharing and a lack of DMA.

The real (not hypothetical) software interface for Amiga IDE was the same as contemporary IBM-compatible PC IDE.

Hi,



No; you're making up nonsense to fit your deluded fantasy. The original PC (and all later versions and clones) had the DMA controller chip (and the design depended on it for DRAM refresh) and there was no IRQ sharing.



No.

If there was ever a spec for the software interface for the IDE controller on 80x86 PCs, the first thing it'd say is "primary controller uses IO ports 0x01F0 to 0x01F7"; and the first thing an Amiga programmer would realise is that the exact same software interface is completely and utterly impossible on the Amiga.

The software interface on Amiga may have been vaguely similar, but "similar" does NOT mean "same".


Cheers,

Brendan

You said "contemporary" which I took to mean "hardware in wide distribution at the same time as the Amiga IDE". That includes ISA-only systems (which typically did not connect the DMA controller to the hard disk at all) as well as PCI-based systems (which are capable of sharing IRQs between devices).


The software interface is independent of how the device is mapped. A spec for the IDE controller interface would state that the primary controller is typically mapped using IO ports 0x1f0 to 0x1f7 and 0x3f6 to 0x3f7, but may be mapped elsewhere, and there may be any number of IDE controllers installed.

The first thing an Amiga programmer would do is look at the Amiga documentation, which helpfully lists both the Amiga IDE mapping and the typical PC IDE mapping side-by-side so the programmer can see it's only mapped to a different address, and not a different software interface.