Filesystem design

[Sean]

I'm creating an operating system in Asm/C/C++ and would like to create a new filesystem to be used.

I've reviewed the alreeady existing filesystems and I'd like to create a new one with the following capabilities:

-Support for filenames greater than 256 bytes long.
-Support for compression and encryption
-Abiliity to hide an entire volume from the user and applications (for security)
-Fast searching.

I've looked into several file systems, and I still don't really understand how everything works in a B-tree fs like HPFS. Could someone please help? I would like to write this fs myself, but if someone could explain things like fnods and B-trees it would be really helpful. If you can help, please keep in mind that I'm a highschool student, and don't have much experience with data sorting.

Thanks

[Tim]

ext2 should do all this. See http://www.nongnu.org/ext2-doc/ext2.html. I'm of the opinion that there are enough file systems in the world which all do pretty much the same thing, so why bother designing another one? I also believe that designing and implementing a good file system is just as much work as designing and implementing a good operating system, so you should do one or the other (and not both at the same time).

[DarylD]

I have to agree with Tim, designing a file system is a "huge" task never mind actually coding it. And by using a filesystem already created you can use current tools to format/check etc. and move files to it.

I would probably stick to something like ext2 or even FAT32. The one advantage of FAT32 is it is very well documented and there is a huge wealth of resources. If any point in the future you need a more complex system then you can go ahead and code that as well, which will still leave you with a driver for the VERY valuable FAT32 system.

Also with FAT32, its possible to use a windows 9x boot disk to get at it if you need to.

Ext2 is a nice performing system, but if I was going to use a unix life FS i would probably try my hand at XFS/ReiserFs (very advanced admittedly).

[Pype.Clicker]

sorry guys, i have to disagree with you on that point. Filesystems are one of the most interresting part of operating systems. It's usually a huge task to deal with, yes, but how exciting !
It's clear that some cross-compatibility must be provided (so that the user can easily move files from/to your OS), but new functionnalities can be brought such as indexing, consistent files versionning, object-oriented file access, etc.

I also agree that there are a lot of existing FS (and xFS from SGI is probably a very interresting one, but still ... 50 000 code lines ... understanding and porting that seems a huger task than designing a new FS, imho)

[Perica]

..

[Pype.Clicker]

Da*n Straight!

Besides, it isn't that-a big deal as you'se make it out to be -- The Actual hard part is getting your OS to read from the Floppy/Hard-Drive (The Hard-Drive is a Much bigger problem than the Floppy, there are a lot of things to consider and do!);

I think you underestimate the problem of a Good FS, perica...

off-topic: This is becomming a bigger and bigger problem, a lot of people (Absoloute Beginners) are stealing other people's code and calling it their own!)

re-off-topic: I don't see this as a problem! they are lamerz! that's all. They suck. You cannot prevent them to do so and they will only cheat other lamerz as people that have a few knowledge will easily recognize the original code

[DarylD]

Yes, filesystems are *hard*. It may be relatively easy to throw together a SuperDuperFS which actually works the same as FAT! (I have seen this a million times)

To actually design and code a reliable FS is quite a challenge. But the possibilities are endless.

Just not something I fancy chewing on!

[Tim]

Me neither. Designing a decent file system amounts to designing a large relational database which will work 100% reliably given huge amounts of data, yet will still be efficient for very small amounts.

On top of that, any (good) file system design for a desktop OS you come up with will be 90% similar to all the other designs; there are only so many 'correct' ways of storing structured data reliably on a storage device that satisfy the constraints imposed on you.

[Warmaster199]

Apparently, the 4 BEST performing filesystems are the following:

#1) SGI's XFS (IRIX, circa 1994 - still the best)
#2) ReiserFS (For LINUX)
#3) EXT3 (For LINUX)
#4) IBM's JFS (New OS/2 Server)

XFS and ReiserFS are closely matched in performance (benchmarks) - with XFS in a slight performance edge. They both are better than JFS and EXT3 (LINUX's 4th generation filesystem[MINIXFS, EXT, EXT2, EXT3]). All are journalling filesystems, meaning they record all disk transactions to a file (the journal) so that error recovery is extremely quick and painless.

XFS and ReiserFS include LARGE file support (a file can be several terabytes/petabytes... whatever... huge files and volumes... 2^64 is how many bytes a file or volume can be). I don't really know much about Reiser, but I know the basic info about XFS:

Splits the disk into 8 sub-sections... like 8 volumes, but on a single partition. Uses a techique called "lazy writes", meaning that it will write to the disk all at once, and only when either the buffer in memory is full OR the machine is properly shutdown. This minimizes the amount of noise the disk makes too (less chirping). The problem is that if you lose power or the OS crashes (something that IRIX never does), you may lose newly created small files (Last I checked, the buffer is 32MBytes). Uses a B-Tree structure as well... According to SGI, the XFS code is 50,000 lines of C source code. You CAN download the entire XFS code and binaries for use on LINUX under the GPL. XFS keeps a 2MByte file of journal entries incase something goes wrong... 2MBytes is alot of entries (If I remember, 4000 entries)