OSDev.org

The Place to Start for Operating System Developers
It is currently Sun Aug 20, 2017 2:23 am

All times are UTC - 6 hours




Post new topic Reply to topic  [ 10 posts ] 
Author Message
 Post subject: Chipset and driver (firmware)
PostPosted: Fri Jan 20, 2017 4:50 pm 
Offline

Joined: Fri Jan 20, 2017 4:35 pm
Posts: 11
Hi,

i am creating an operating system and i happened at the writing of the hard disk driver, it is SATA and the processor x86-64, i am in protected mode. i saw this page http://wiki.osdev.org/AHCI and was surprised that writing hard disk driver does not involve i control how physicaly the hard disk works (head position, rotation speed, etc...), so my question is theorycal, what does in the computer control the hard disk physical working ? Is it the chipsets ? If yes, is chipset a hardware or software thing ?


Last edited by ComputerMail on Sat Jan 21, 2017 12:38 am, edited 2 times in total.

Top
 Profile  
 
 Post subject: Re: Chipset and driver
PostPosted: Fri Jan 20, 2017 6:21 pm 
Offline
Member
Member
User avatar

Joined: Fri Mar 07, 2008 5:36 pm
Posts: 2070
Location: Bucharest, Romania
The hard disk comes with some firmware on it that does the LBA-to-CHS conversion. This makes interfacing with the drive easier but also makes scheduling less efficient, as it cannot reliably take the disk's geometry into the equation. Even when you are able to send out commands in the CHS format, you cannot trust that they reflect the geometry as modern disks automatically remap bad sectors to spare ones, if available.

_________________
"Computers in the future may weigh no more than 1.5 tons.", Popular Mechanics (1949)
[ Project UDI ]


Top
 Profile  
 
 Post subject: Re: Chipset and driver
PostPosted: Fri Jan 20, 2017 9:32 pm 
Offline
Member
Member
User avatar

Joined: Sat Jan 15, 2005 12:00 am
Posts: 8054
Location: At his keyboard!
Hi,

ComputerMail wrote:
i am creating an operating system and i arrived at the writing of the hard disk driver, it is SATA and the processor x86-64, i am in protected mode. i saw this page http://wiki.osdev.org/AHCI and was surprised that writing hard disk driver does not involve i control how physicaly the hard disk works (head position, rotation speed, etc...), so my question is theorycal, what does in the computer control the hard disk physical working ? Is it the chipsets ? If yes, is chipset a hardware or software thing ?


The OS's driver tells (the PCI bus to tell) the SATA controller to send a command to a device attached to the controller, the SATA controller has some control logic to figure out how to send a command to the requested device, and the device has some logic (typically a small CPU and firmware and RAM - almost like a tiny little embedded computer) that decodes the command, determines if the command is sane, and performs the requested command; and takes care of things like:
  • Device initialisation/power up, and reset
  • Media initialisation and removal (for CD drives, etc)
  • Emulating a different sector size if necessary (e.g. hard disk with 4 KiB sectors emulating a hard disk with 512 byte sectors)
  • Wear levelling (for SSD)
  • Bad sector replacement (for hard disk, where modern hard disks have "spare sectors" that are used to replace sectors that fail and extend the drive's "mean time between failures")
  • Determining the most optimal way of performing the command/s (e.g. the order commands should be done when NCQ is being used)
  • Power management (e.g. turning the drive's motor off to save some power if it hasn't been used for a while)
  • Encoding/decoding the data if necessary (e.g. converting your raw bytes into some sort of error detecting or correcting code)
  • Tracking various statistics, mainly to predict failures before they happen (Self-Monitoring, Analysis and Reporting Technology)
  • Handling the motor speed, movement of heads, head selection, etc (including all the timing requirements).
  • Handling any "retry N times if there was an error" fault recovery (potentially including replacing a sector with a "spare sector" if the drive had too much trouble reading but succeeded eventually). Note that this may include writing, where a drive can reading the data back (using a "read head") as it's being written (by a "write head") so that the drive can verify that the data was written correctly.
  • Sending status/replies back to the controller; and handling or monitoring any data transfers between controller and drive's buffer
  • Pre-fetching data into the device's buffer in anticipation of a future read


Cheers,

Brendan

_________________
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.


Top
 Profile  
 
 Post subject: Re: Chipset and driver
PostPosted: Sat Jan 21, 2017 12:31 am 
Offline

Joined: Fri Jan 20, 2017 4:35 pm
Posts: 11
Thank you a lot for answers, by seeing all what the firmware does i assume no operating system developer pays attention to write his own one, is it right ? Also, same for any component's firmware ?


Top
 Profile  
 
 Post subject: Re: Chipset and driver
PostPosted: Sat Jan 21, 2017 4:13 am 
Offline
Member
Member
User avatar

Joined: Thu Jul 12, 2012 7:29 am
Posts: 712
Location: Tallinn, Estonia
ComputerMail wrote:
Thank you a lot for answers, by seeing all what the firmware does i assume no operating system developer pays attention to write his own one, is it right ? Also, same for any component's firmware ?


You can't. It's specific for each device and is written by Hard Disk manufacturer.

_________________
Learn to read.


Top
 Profile  
 
 Post subject: Re: Chipset and driver
PostPosted: Sat Jan 21, 2017 5:30 am 
Offline
Member
Member
User avatar

Joined: Sat Dec 27, 2014 9:11 am
Posts: 848
Location: Maadi, Cairo, Egypt
ComputerMail wrote:
Thank you a lot for answers, by seeing all what the firmware does i assume no operating system developer pays attention to write his own one, is it right ? Also, same for any component's firmware ?

You can't write your own firmware for a specific piece of hardware. This is all drive-specific, and things like AHCI and IDE are just interfaces to hide the differences between the actual drive differences. Brendan already explained how it works; you tell AHCI to send a command to the drive, and AHCI sends the command to the drive, and the drive does its drive-specific functions transparently to you and the AHCI, and when it finishes it returns its status to the AHCI and you can find out whether or not the command succeeded from there. IDE also follows similar theory to AHCI.

_________________
Byte me; OS site, source code.
You know your OS is advanced when you stop using the Intel programming guide as a reference.


Top
 Profile  
 
 Post subject: Re: Chipset and driver (firmware)
PostPosted: Sat Jan 21, 2017 6:34 am 
Offline
Member
Member
User avatar

Joined: Fri Mar 07, 2008 5:36 pm
Posts: 2070
Location: Bucharest, Romania
Well, you can do it but, in many cases, you'll need to do a bit of reverse engineering in order to figure out how to do it since the documentation is lacking. But people have done it before, they've even written malware that rests in such firmware, which needless to say, is very difficult to detect. Either way, custom firmware is not something an OS needs to concern itself with---ultimately it needs to trust the hardware and that responsibility lies outside the realm of the OS IMO.

_________________
"Computers in the future may weigh no more than 1.5 tons.", Popular Mechanics (1949)
[ Project UDI ]


Top
 Profile  
 
 Post subject: Re: Chipset and driver (firmware)
PostPosted: Sat Jan 21, 2017 7:57 am 
Offline
Member
Member
User avatar

Joined: Thu Jul 12, 2012 7:29 am
Posts: 712
Location: Tallinn, Estonia
Love4Boobies wrote:
Well, you can do it but, in many cases, you'll need to do a bit of reverse engineering in order to figure out how to do it since the documentation is lacking. But people have done it before, they've even written malware that rests in such firmware, which needless to say, is very difficult to detect. Either way, custom firmware is not something an OS needs to concern itself with---ultimately it needs to trust the hardware and that responsibility lies outside the realm of the OS IMO.


UNLESS you're writing a trusted kernel for trusted computing system, but that's usually rare in hobby OSdev.

_________________
Learn to read.


Top
 Profile  
 
 Post subject: Re: Chipset and driver (firmware)
PostPosted: Sat Jan 21, 2017 8:38 am 
Offline
Member
Member
User avatar

Joined: Fri Mar 07, 2008 5:36 pm
Posts: 2070
Location: Bucharest, Romania
If it's a trusted computer, why should the OS care about the firmware? Presumably it should trust it implicitly. If it doesn't, pick different hardware.

There is really not much overlap between firmware development and OS development like there is between OS development and application development.

_________________
"Computers in the future may weigh no more than 1.5 tons.", Popular Mechanics (1949)
[ Project UDI ]


Top
 Profile  
 
 Post subject: Re: Chipset and driver (firmware)
PostPosted: Sat Jan 21, 2017 1:15 pm 
Offline
Member
Member
User avatar

Joined: Fri Oct 27, 2006 9:42 am
Posts: 936
Location: Athens, GA, USA
dozniak wrote:
UNLESS you're writing a trusted kernel for trusted computing system, but that's usually rare in hobby OSdev.


I have a problem with the concept of Trusted Computing in general - or rather, with the narrative laid on top of it by marketing departments, which gives a highly distorted view of the idea - but that is going off topic enough that anything further said about it should be in a separate thread. Suffice it to say that actual Trusted Computing is incompatible with consumer-grade hardware and software.

Getting back to the topic, the real issue is that the CPU does not, as a rule, have access to the drive's firmware and internal hardware in the first place - the ATA model realistically requires that drive be managed by an embedded microcontroller with its own memory and firmware (which on higher-end devices are usually a flash memory, to allow for firmware updates, but could be an actual masked ROM in cheaper models), which the CPU has no direct interaction with - and a programmer would need details of the hardware which are not generally made available in order to affect the on-drive control electronics directly. This isolation is at least partially deliberate, as the primary goal of the controller design going back to the XT days was to relieve the CPU of these tasks - even with the original MFM drives, the CPU didn't need to do most of that work, and successive controller models have offloaded more and more of the work onto the controller board and the drive itself over time.

In most cases, the only interaction that the computer's CPU has with the firmware is to pass a firmware update to it, and a reputable hardware manufacturer should be using some kind of encryption on that and bundle it with with an ECC signature and a hash signature for tamper-checking (while those aren't foolproof, they do make tampering impractical for most casual attacks). I don't know if any do, but... anyway.

This isolation is a Good Thing, because these operations take a lot of computation, so off-loading it only makes sense - just as with CPU-driven video, a CPU-driven disk would eat an inordinate amount of cycles when in use. Also, it modularizes the system better, reducing the chances that a bug in the driver will trash the disk records or the disk itself - in the bad old days of minicomputers with loadable platter packs, and even in the early 8-bit home computer days with the Shugart, Apple ProFile, and Apple Sider drives, it was possible for a wonky driver to cause physical damage to the platters and/or read heads. Finally, it abstracts away the design of the drive itself, making it possible to use a single generic driver for not only many models of hard disk without regard to the drive's actual geometry or control electronics, but also with completely different storage technologies such as solid-state flash and NVMe.

_________________
Rev. First Speaker Schol-R-LEA;2 LCF ELF JAM POEE KoR KCO PPWMTF
μή εἶναι βασιλικήν ἀτραπόν ἐπί γεωμετρίαν
Lisp programmers tend to seem very odd to outsiders, just like anyone else who has had a religious experience they can't quite explain to others.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 10 posts ] 

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 5 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group