OSDev.org

The Place to Start for Operating System Developers
It is currently Fri Jul 19, 2019 8:05 pm

All times are UTC - 6 hours




Post new topic This topic is locked, you cannot edit posts or make further replies.  [ 29 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: Flash memory (SD) and File System Design Considerations
PostPosted: Fri Jun 07, 2019 6:00 pm 
Offline
Member
Member

Joined: Tue Apr 24, 2018 9:46 pm
Posts: 52
I plan on using an SD card as my "disk" (i.e. persistent memory). What should I keep in mind when designing a simple file system with regards to sending read and write requests to the SD card?

For example, given that there is no head that physically moves, do data blocks of a file still benefit from being stored contiguously?

I have heard that SD cards have a limited number of writes. Also that writes should be spread across the card so that regions wear out at a similar rate. How much of this does the file system have to take into account? Does the firmware in the SD card do some or all of this work?

As a sidenote, why are CompactFlash cards so popular among homebrew computers? Do they offer a meaningful advantage over SD cards?

Second sidenote, the Raspberry PI comes to mind as an example of using an SD as the "disk". Do the Linux distributions targeted at the Raspberry PI do anything special with this in mind?


Top
 Profile  
 
 Post subject: Re: Flash memory (SD) and File System Design Considerations
PostPosted: Fri Jun 07, 2019 7:18 pm 
Offline
Member
Member
User avatar

Joined: Thu Oct 13, 2016 4:55 pm
Posts: 326
Hi,

You have many questions :-)

1. contiguous data blocks: not really (so minimal that you won't notice)
2. probably the firmware will spread the writes for you, but it couldn't hurt if you do so as well
3. there's no advantage to CF cards, other than they preceed SD, so they are more widespread and cheaper (also slower and smaller capacity)
4. No, the Linux kernel does not do anything special.
https://github.com/raspberrypi/linux/bl ... /bcm2835.c

Cheers,
bzt


Top
 Profile  
 
 Post subject: Re: Flash memory (SD) and File System Design Considerations
PostPosted: Sat Jun 08, 2019 2:05 am 
Offline
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 1552
quadrant wrote:
As a sidenote, why are CompactFlash cards so popular among homebrew computers?

It's usually easier to build a fast CF interface than it is to build a fast SD interface.


Top
 Profile  
 
 Post subject: Re: Flash memory (SD) and File System Design Considerations
PostPosted: Sat Jun 08, 2019 6:19 am 
Offline
Member
Member
User avatar

Joined: Fri Feb 17, 2017 4:01 pm
Posts: 380
Location: Ukraine, Bachmut
first, i don't get why you decided that ancient CF cards are dominant now? literally every SBC now comes with the SD interface. most of them come with eMMC too. these two storage interfaces are the main ones, you should pay attention to in the first place. they are pretty similar, eMMC is significantly faster, but pricier as well, of course. finally, there become to appear UFS modules - insanely fast things, but so far, I am aware of only a couple of SBCs sporting them (HiKey960/970 and some qualcomm SoC based board, forgot the name).
NONE of these require wear leveling on the host side, they are exposed to the host as ordinary block devices. of course, the host "knows", that these are flash based block storages, so it handles them a little bit differently in some cases, but everything related to protecting NAND cells is done internally by the card controller (FTL). host controller and thus the OS are freed of this duty. Just like with SATA and NVMe SSDs. So you are strongly discouraged from trying to do wear leveling at the host side - it is a pure waste of time, resources and efforts and it will make just one effect - slows down your storage stack. the same applies to trying to keep data locality (in the LBA space), defragmentation includingly. all of that is waste of time for flash based block storages. of course, avoiding fragmenration may be benefitial for other reasons, on the FS level (less metadata to describe the file), but this is heavily dependant on the FS and absolutely has nothing to do with the underlying storage device.
But there still occur, however rarely, bare NAND devices. they do need wear leveling on the host side. it's vital for their longevity. it's all complex and to make it worse - often poorly documented. for example accompanying chip for doing error corrections, it could be poorly documented, that would result in hard times for the developer to get this all work as needed. happily, as I've said all modern SBCs come with SD/eMMC, so you don't need to worry about whimsy NAND at the beginning. in short, for such devices, you need special FS, or some underlying layer, taking care of wear leveling. YAFFS2 is the example of the former variant, you could take a look at. it is a log based FS, that writes everytime in a different place, in a circular fashion, across the whole volume, making thus wear leveling happen naturally.
are you gonna work with ARM SBCs?
and btw, about linux distributions "doing something special" with this regard. they, more, than often, DON'T DO anything even on the machines with bare NAND, where it's absolutely needed. just hell of ext4 bashing that poor NAND device. and given UBI/UBIFS (a NAND oriented FS/layer devised for linux) has dropped support for MLC (they said MLC is "unreliable"! probably they aren't aware of TLCs used in most SSDs... oh well, at least they were honest in that they managed to protect only the most durable and expensive SLC NANDs with their "tecnhology"), the situation even worsened. basically, now every linux on SBCs with NAND uses ordinary ext4 and this kills the device quickly.

_________________
future big goal: ANT - NT-like OS for mips, arm and x86.
current smaller goal: efify - UEFI for a couple of boards (mips and arm).


Top
 Profile  
 
 Post subject: Re: Flash memory (SD) and File System Design Considerations
PostPosted: Sat Jun 08, 2019 12:06 pm 
Offline
Member
Member

Joined: Tue Apr 24, 2018 9:46 pm
Posts: 52
Thank you all for the insight! I'm very relieved to learn that my naive file system will not unduly wear out the SD card. Even more so that I don't have to worry about locality! :D


Top
 Profile  
 
 Post subject: Re: Flash memory (SD) and File System Design Considerations
PostPosted: Mon Jun 17, 2019 3:26 am 
Offline
Member
Member

Joined: Tue May 13, 2014 3:02 am
Posts: 263
Location: Private, UK
CF cards are popular among hobbyists because they "speak" ATA and thus can be connected to an IDE interface with no/minimal additional circuitry. They can also usually be used on an 8-bit interface (aka XTA), while many/most hard drives cannot. These characteristics make them more-or-less ideal for upgrading/replacing dead hard drives in very old PCs (and many other devices that were designed for use with IDE hard drives).

As for an FS for SD cards; I assume you know that the SD standard dictates the use of FAT(32) and that many SD cards have specific optimisations (such as having faster/more resilient storage for specific structures, on-card caching, etc.) in their control chips for this filesystem...? While, obviously, other filesystems almost always work just fine, you may well find that the speed and/or lifetime of the cards suffers when not using FAT.

_________________
Image


Top
 Profile  
 
 Post subject: Re: Flash memory (SD) and File System Design Considerations
PostPosted: Mon Jun 17, 2019 7:50 am 
Offline
Member
Member
User avatar

Joined: Thu Oct 13, 2016 4:55 pm
Posts: 326
Hi,

mallard wrote:
I assume you know that the SD standard dictates the use of FAT(32)
Since when? Would you care to link a specification for us? I'm using SD cards without FAT just fine. Also in Raspbian's SD card image it is only the first small boot partition that uses FAT, the rest, the bigger part of the card is formatted to ext4. I doubt that the engineers at Element 14 or Raspberry Inc are mistaken.
mallard wrote:
and that many SD cards have specific optimisations (such as having faster/more resilient storage for specific structures, on-card caching, etc.) in their control chips for this filesystem...?
Again, would you care to link where this is written? Physical sector address randomization optimizations exists to extend card's write-cycles, but they work for any other filesystems as well. The SD card doesn't care if that often modified area at the beginning of the card is an allocation table with cluster addresses or an inode table with inode numbers. (And frankly, why would it care? What would it do differently for those two?)
mallard wrote:
While, obviously, other filesystems almost always work just fine, you may well find that the speed and/or lifetime of the cards suffers when not using FAT.
It's not obvious, would you mind to tell us the source where you get this? Maybe I'm wrong, but I'd like to see the spec if I am. No offense, but without a proof this smells like bullshit, and it's against my every day experience. I did performance tests on a Raspbian SD card, and there was no difference reading files from the boot (FAT) partition and the root (ext4) partition (I've used a big, contiguous (defragmented) file for the test to eliminate the filesystem's overhead).

Cheers,
bzt


Top
 Profile  
 
 Post subject: Re: Flash memory (SD) and File System Design Considerations
PostPosted: Mon Jun 17, 2019 8:30 am 
Offline
Member
Member

Joined: Thu Jul 05, 2007 8:58 am
Posts: 219
This can be found in the SD card specification. A "simplified" version of part 1 can be downloaded from the SD card foundations website here. It references FAT itself in defining speed categories (making it very likely cards will optimise for fat to get faster speed ratings), as well as specifying FAT itself as the storage format to be used on SD cards in part 2. Unfortunately, that last part doesn't seem to be publicly available, but someone familiar with the internet will probably be able to find old versions floating around.


Top
 Profile  
 
 Post subject: Re: Flash memory (SD) and File System Design Considerations
PostPosted: Mon Jun 17, 2019 9:41 am 
Offline
Member
Member
User avatar

Joined: Thu Oct 13, 2016 4:55 pm
Posts: 326
Hi,

davidv1992 wrote:
This can be found in the SD card specification. A "simplified" version of part 1 can be downloaded from the SD card foundations website here. It references FAT itself in defining speed categories (making it very likely cards will optimise for fat to get faster speed ratings), as well as specifying FAT itself as the storage format to be used on SD cards in part 2. Unfortunately, that last part doesn't seem to be publicly available, but someone familiar with the internet will probably be able to find old versions floating around.
Thank you! That specification tells exactly nothing about mandatory, suggested or preferred FS type on the cards (or about the data stored on the card for that matter, except it has to splitted in 4M RU blocks).

It also doesn't mention anything about degraded SD card lifetime regarding a specific FS type.

For read speed, it lists no FS specific constraints at all (section 4.13.2.3). For FS writes (section 4.13.2.4), it only states that:
SD spec wrote:
In the case of SDXC, the FAT update cycle consists of three write operations, FAT, Bitmap, and directory entry. The FAT is written starting at any 512-byte boundary address and with any size up to 16Kbytes. The Bitmap is written starting at any 512-byte boundary address and with any size from 512-byte to 16Kbytes. A directory entry should be created before starting recording
So in short,
a) It only says *IF* you use FAT on an SD card, then consider speed issues (section 4.13.1.7 and section 4.13.2.4)
b) "FAT Update" is not an SD card protocol command, rather a sequence of three specific CMD24 or CMD25 commands typical for FAT file systems
c) to take advantage of the optimization implemented for that, it's quite simple to fulfill those requirements with any file system, and you can just as easily write an inode table and inode allocation bitmap in a write sequence which looks exactly like a "FAT Update" cycle. ;-)

Because there's nothing FS specific in the SD Card Physical Layer Specification and command protocol, it is safe to assume using FAT on SD cards (as recommended by Part2) is just a marketing bullshit probably sponsored by M$. (Don't get me wrong, using FAT on SD cards has many advantages, interoperability for one, so it's not bad at all, just not dictated or handled differently at Physical Layer Level.)

Cheers,
bzt


Top
 Profile  
 
 Post subject: Re: Flash memory (SD) and File System Design Considerations
PostPosted: Mon Jun 17, 2019 3:20 pm 
Offline
Member
Member
User avatar

Joined: Thu Oct 13, 2016 4:55 pm
Posts: 326
zaval wrote:
here is a recipe for ya, what you need to do to get it:
And how exactly that prove that SD cards supposed to be used with FAT only? Or that using ext4 on an SD card for example would shorten the card's life time?
zaval wrote:
Physical Layer part has nothing to do with how the controller should deal with NAND cells, it specifies interface between card and host, - bus, protocol, commands, responses, modes etc,
That's what I'm saying. It has nothing to do with file systems, that's a higher abstraction (btw NAND cells are on a lower level).

Please don't be a troll. Thanks.
bzt


Top
 Profile  
 
 Post subject: Re: Flash memory (SD) and File System Design Considerations
PostPosted: Mon Jun 17, 2019 3:51 pm 
Offline
Member
Member
User avatar

Joined: Fri Feb 17, 2017 4:01 pm
Posts: 380
Location: Ukraine, Bachmut
bzt wrote:
And how exactly that prove that SD cards supposed to be used with FAT only? Or that using ext4 on an SD card for example would shorten the card's life time?

man, if you don't want to search leaked versions of it on the web, you may pay for it and get it legally. there, it clearly says - the FS is FAT. for both general area and secure area. even though special requirements differ for two. that's how you can get it. maybe. You may use it with something else, but since FAT is mandatory and that certainly would mean relying by controllers on its presence, when doing their wear leveling and optimizing jobs, lack of it could at least make it run slower and of course - speed up wear out, since the controller wouldn't be able to do its best for wear leveling without information it expected and relied on. Sometimes though, as I've said, card just rejects to work at all, spitting sh1tloads of errors at linux with its ext4.

Quote:
zaval wrote:
Physical Layer part has nothing to do with how the controller should deal with NAND cells, it specifies interface between card and host, - bus, protocol, commands, responses, modes etc,

That's what I'm saying. It has nothing to do with file systems, that's a higher abstraction (btw NAND cells are on a lower level).

When you format it as FAT, internal controller inside of a card is happy because it expects exactly this. It takes FAT information for its algorithms, it uses it read only, but with it, it understands how card is used by the host and thus makes better managing NAND blocks - basically it sees the whole map of used/unused sectors which of course should help it with making wear leveling. with strange FS, it resorts to the worst algorithms, where it doesn't know anything about usage of the blocks except what it gets from the host on the command level (read that, write that, that's all). This behavior is not specified by the SD specification, since internal NAND handling is out of the scope, but FAT as the file system is specified. Not hard to guess, that controllers make use this knowledge.

_________________
future big goal: ANT - NT-like OS for mips, arm and x86.
current smaller goal: efify - UEFI for a couple of boards (mips and arm).


Last edited by zaval on Mon Jun 17, 2019 4:51 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Flash memory (SD) and File System Design Considerations
PostPosted: Mon Jun 17, 2019 3:55 pm 
Offline
Member
Member

Joined: Mon Mar 25, 2013 7:01 pm
Posts: 1552
bzt wrote:
Because there's nothing FS specific in the SD Card Physical Layer Specification and command protocol, it is safe to assume using FAT on SD cards (as recommended by Part2) is just a marketing bullshit probably sponsored by M$.

If you only look at Part 1. However, Part E7 (the Wireless LAN Addendum) makes several mentions to the file system used on the SD card.

Additionally, Part 2 does not "recommend" the filesystem, it requires the filesystem.
Quote:
That is, all Standard Capacity SD Memory Cards whose capacity is 2048MB or less shall use FAT12 / FAT16 file system, and never use the other file system. Similarly, all High Capacity SD Memory Cards shall use FAT32 file system, and never use the other file system. And all Extended Capacity SD Memory Cards shall use exFAT file system, and never use the other file system.

It goes on to require exactly one partition, formatted to use the full capacity of the card.

bzt wrote:
(Don't get me wrong, using FAT on SD cards has many advantages, interoperability for one, so it's not bad at all, just not dictated or handled differently at Physical Layer Level.)

We're talking about how the card internally handles things, not how the card presents itself to the host. The access timings are dictated based on the filesystem, and that means the card may examine filesystem structures in order to optimize its performance to meet its timing requirements. For example, the card may detect directory entry updates and write only the updated part of the sector in its internal log-structured filesystem, to minimize write/erase cycles. (More extreme optimizations could also be possible, if the specification allows it, such as storing only one FAT but giving the appearance of having two. I wasn't able to find any part of the spec that explicitly allows or forbids such behavior.)


Top
 Profile  
 
 Post subject: Re: Flash memory (SD) and File System Design Considerations
PostPosted: Tue Jun 18, 2019 10:04 am 
Offline
Member
Member
User avatar

Joined: Thu Oct 13, 2016 4:55 pm
Posts: 326
Octocontrabass wrote:
Additionally, Part 2 does not "recommend" the filesystem, it requires the filesystem.
Despite of this "requirement", I can read and write an SD card with raw data. I had absolutely no problems using my own, totally custom file system either. Works like a charm, and I'm still using the same 4 or 5 years old cards without any signs of shortened life time.
Octocontrabass wrote:
It goes on to require exactly one partition, formatted to use the full capacity of the card.
The same, I never had problems using more partitions on an SD card. Raspbian also has more partitions (some of which are not FAT at all) without any problems, which questions this so called "requirement". To paraphrase Euclid, never believe something just because someone said so (and if I may add, specially don't believe when that someone is financially interested to say that something).

Octocontrabass wrote:
that means the card may examine filesystem structures
Okay, now I see what confused you.

The card does not examine file system structures. It only checks the written data size, and expects the higher level (FS and block) driver to tell whether the next sector to be written is a directory entry, allocation data, bitmap etc. That's what the CMD20 is for, see section 4.13.2.8 Speed Class Control:
Quote:
Speed Class Control (SCC) in argument controls several functions which assist the card supporting and meeting Class performance.
And also section 4.13.2.5 CI (Continuous Information) Update
Quote:
CI (Continuous Information) is a new structure used to manage exFAT file fragments. It is newly defined in the Part2 File System Specification Version 3.00. CI may be updated during Speed Class recording. Creating a CI is optional for the host.
...
CI is always written by a 512-byte single block write (either CMD24 or CMD25) and preceded by CMD20 Update CI command.
In other words, the card doesn't care if the data written is a FAT directory entry for example, it only cares about if it's flagged as one. Surely you can flag a directory entry write for ext4 too.

But let's take a step back. What would happen if you don't tell the card what kind of sector are you about to write? The answer is in section 4.13.2.6 Distinction of Data Type:
Quote:
To satisfy Class performance, an SDXC card needs to distinguish between each type of data in order to treat them properly. For example, a directory entry and CI can be distinguished by their data size (always written as 512B), so the card can store them in separate areas from the stream data.
Simply put, if no CMD20 issued, then single sector writes will be assigned a real sector address from a different pool than multiple writes. That's all.

But if despite all of this using exclusively the limited FAT file system makes you happy, then do so. I'll keep using partitions and better file systems on the SD card without probs in the meantime ;-) Let's close this discussion with this conclusion.

Have a nice day,
bzt


Top
 Profile  
 
 Post subject: Re: Flash memory (SD) and File System Design Considerations
PostPosted: Tue Jun 18, 2019 2:17 pm 
Offline
Member
Member

Joined: Wed Aug 17, 2016 4:55 am
Posts: 234
*reads thread* Oh boy, what a mess…

My understanding from having looked up about SD cards earlier (as I was looking into making a cartridge with SD card slot) is that: 1) while not necessarily required, FAT is pretty much the only thing guaranteed to work everywhere and 2) the microcontroller may implement wear levelling optimizations based on common access patterns, and that the preformatted filesystem has the correct settings to make these assumptions work (e.g. cluster size matching the Flash block size, etc.).

Note that an implication of #2 is that even formatting FAT again is a bad idea, because you're breaking whatever assumptions the wear levelling was doing and risk slow down in the long term (even with the same settings I wouldn't suggest it, as formatting touches quite a large bunch of blocks). Again, take into account this depends entirely on the particular card (as every vendor will do wear levelling in whatever way they like), so it's perfectly possible to run into a card where this isn't a problem, while it could completely trash the usability of a different card.

So huh, ideally use whatever the card came with from factory already. And if for some reason it came unformatted, it's likely there weren't any filesystem-specific optimizations to worry about and whether you pick FAT or anything else is up to you.


Top
 Profile  
 
 Post subject: Re: Flash memory (SD) and File System Design Considerations
PostPosted: Tue Jun 18, 2019 3:29 pm 
Offline
Member
Member
User avatar

Joined: Mon Jan 15, 2018 2:27 pm
Posts: 63
IMO. It's just like with those 'IBM formatted' floppy disks. Meaning that, this just is the way they come from the factory. And that's it. Also that they are performance tested using some sort of FAT FS.

_________________
https://github.com/pvc988/woot64


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic This topic is locked, you cannot edit posts or make further replies.  [ 29 posts ]  Go to page 1, 2  Next

All times are UTC - 6 hours


Who is online

Users browsing this forum: Bing [Bot], Google [Bot], klange and 8 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group