OSDev.org

Hi,

I've been a long-time reader of OSDev forums and wiki, which helped me a lot in the past. Today I've stumbled upon an interesting project that seems to be unknown for OSDev dwellers, and, IMHO, it is really revolutionary and a real game changer.

The project I'm talking about is a new open source domain-specific language Kaitai Struct and its sister project, Kaitai FS. Kaitai Struct allows one to specify multitude of complex data structures using declarative language, which can be then examined in an browser-based IDE:



or console IDE:



In this respect, it is like a free clone of 010 Editor or Hexinator, which is already cool. But there's a much, much cooler feature around the corner: it can be compiled into a parser library in many programming languages (C/C++, C#, Java, JavaScript, Python, Ruby, Perl, PHP, Go, etc).

The idea of specifying filesystems as data structures is just brilliant. Instead of re-writing yet another (probably buggy) implementation of FAT, ISO9660, NTFS, or whatever filesystem you plan to support in your OS, you can just take ready-made vfat.ksy, iso9660.ksy, etc, and you get a working implementation library in language of your choice right away! Kaitai FS is a project that does exactly that: it provides a thin wrapper that converts .ksy filesystem implementations (and also containers, archive files, etc) into FUSE-based implementations in Python for Linux and OS X.

There's even a guy who reverse engineering newest Apple's APFS filesystem using Kaitai: https://github.com/cugu/apfs.ksy — and, I guess, it means that APFS can be supported as well

What do you think of it? Anyone planning to use filesystems declared in .ksy syntax for your projects? I do

I wonder if we should add ksy links to all the formats specified informally in Wiki? Kaitai project already seems to have ELF, MZ, PE, Mach-O executables described in .ksy, and quite a few filesystems. That would be likely helpful for generations to come

The language seems good. I need to check if this little one can replace lexx/yacc. From what i see, you can create context and branch according to magic items so it may work.
Nice finding !

Not really — it is for parsing binary, which is usually designed to be machine-readable, i.e. non-ambiguous. Generally, it's a bad idea to parse, for example, FAT filesystem with lexx/yacc (that would be way too verbose and you'll end up writing gazillions of states and lexems instead of focusing on the format), but it is a good idea to parse it with KS.