Discussions on more advanced topics such as monolithic vs micro-kernels, transactional memory models, and paging vs segmentation should go here. Use this forum to expand and improve the wiki!
frank wrote:It wouldn't be that hard to make the database engine understand the contents of most files and for the other files they could be stored with meta data just like on a real filesystem.
You could even have a module-type system so each application could have a module that would allow the database system to understand its own file type. (Blender, for instance, would load a module that would allow the system to preview, read, etc. 3D type files.) Then other applications could use each other's module's to read and write those types of files.
I think this is what WinFS was going for. Reading about it just makes me want to use it...
Yeah I was really hoping for WinFS. When they decided not to put it in Vista I lost just a little bit more of my respect for Microsoft.
WinFS is not the only approach. "Database" does not necessarily imply "relational database".
<anti-MS-rant>
IMO WinFS is overkill. You can get the same benefits without the relational DBMS back-end. In fact, it's already been done. IMO it makes more sense to study a working implementation that's already been providing tangible benefits to users for 3 years than to study a half-baked piece of mostly-vaporware.
</anti-MS-rant>
As I said before, you don't need the relational model in order to have good search capabilities and extensible metadata. I have a hard time imagining what value a relational DBMS adds to such a scheme in the first place. Did WinFS have any support for transactions? Triggers? Referential integrity? I'd be surprised if these features make much sense on unstructured file data... For example, why bother with RDBMS-style transactions when journalling filesystems already exist?
Top three reasons why my OS project died:
Too much overtime at work
Got married
My brain got stuck in an infinite loop while trying to design the memory manager
a DB as an FS is a great idea (id love to do an open source version of something like OS400)...
a normal FS is a DB..
when most people go 'oh a DB would make a great FS' they commonly instantly forget you need on-disk-structures and such to manage the DB.
some DB systems used raw disk (enterprisy stuff), bypassing the filesystem, but still you need the whole free space/bitmaps/inodes type deal to manage all the data.
the idea is great, but it comes down to the implementation which suddenly becomes hairy.
You are forgetting that bare hardware will never be able to support markup structures/complex data. They will ALWAYS only support numbers, and as such all OS's will ALWAYS need to transmit information in words (byte length standard could change). How much marshalling/unmarshalling actually takes place in kernel-space can be variable, but it's just another layer of abstraction.
(by always I mean in the forseeable future, not a millennium from now).
I've had an idea of standardising common file types (raw files, images, sounds, contacts, e-mails). My idea is about representing files as having properties based on their type rather than just as raw binary. For example, imaging the following is how a file-system stores data: (I'm only using XML to show my idea - in a real-world scenario you wouldn't store your FS in an ASCII-readable format)
A conventional file system has, around 3 properties for a file: name, permissions, stream of data.
What if, instead of just having a stream of data, the entry can have more properties depending on it's type. If it were a contact's details stored on the FS, it could be represented as:
<file type="contact" name="Joe Bloggs" e-mail="jblogs" picture="PNG)(*#@)$*#@)$*"></file>
An offline copy of a HTML file could be represented as:
<file type="offline-html" name="mysite.html" origional-url="http://www.mysite.com/mysite.html" last-updated=200700724>raw-html-data-here</file>
There could be a limitless number of fields (for associating attachments with a page). A binary could be represented as:
<file type="binary" developer="Joe Bloggs" homepage="http://www.mysite.com" version=1.4.2 icon="#@$*@)($*@" minimum-memory=4096>raw-binary-data-here</file>
A file could have an owner, and the database could look for the contact entry, and then list all other files also associated with that owner.
The operating system should have a 'plug-in' interface so it can understand different types of files, what to do with them, and how to associate them with other types.
MessiahAndrw wrote:The operating system should have a 'plug-in' interface so it can understand different types of files, what to do with them, and how to associate them with other types.
MessiahAndrw wrote:The operating system should have a 'plug-in' interface so it can understand different types of files, what to do with them, and how to associate them with other types.
Spotlight is getting close to what I mean, but not exactly. Spotlight stores the file in a conventional way but with extra metadata.
I'm thinking of a completely new way to store files. A file will not have 'data' in a traditional way, with meta-data wrapping around it. Instead, a file will just be an 'entry' in the FS, with various fields each with their own attributes (e.g. each file has a Name attribute an a type attribute as a bare minimum, an image will also have a width attribute, a height attribute, a bpp attribute, what format it is attribute (raw bitmap, compressed raster, compressed vector) and an image-data attribute). There will be no data this entry wraps around (i.e. it's not a file 'header'), rather all data is stored in the entry as an attribute.
MessiahAndrw wrote:Spotlight is getting close to what I mean, but not exactly. Spotlight stores the file in a conventional way but with extra metadata.
I'm thinking of a completely new way to store files.
What advantage is there to such a scheme? Wouldn't it make it harder to trade files with other file systems without stripping off the extra metadata (like Mac resource forks in the days of yore)? A lot of important file formats already include their own metadata (mp3, jpeg, most video formats I can think of, etc.)
Top three reasons why my OS project died:
Too much overtime at work
Got married
My brain got stuck in an infinite loop while trying to design the memory manager
MessiahAndrw wrote:Spotlight is getting close to what I mean, but not exactly. Spotlight stores the file in a conventional way but with extra metadata.
I'm thinking of a completely new way to store files.
What advantage is there to such a scheme? Wouldn't it make it harder to trade files with other file systems without stripping off the extra metadata (like Mac resource forks in the days of yore)? A lot of important file formats already include their own metadata (mp3, jpeg, most video formats I can think of, etc.)
I'll have conversion tools.. Anyway, in another post I said I wanted to try out new ways of doing things in my OS, and not write yet another DOS or POSIX clone just for the sake of compatibility.
MessiahAndrw wrote:I'll have conversion tools.. Anyway, in another post I said I wanted to try out new ways of doing things in my OS, and not write yet another DOS or POSIX clone just for the sake of compatibility.
IMO, it's better to be better than to just be different.
Top three reasons why my OS project died:
Too much overtime at work
Got married
My brain got stuck in an infinite loop while trying to design the memory manager
MessiahAndrw wrote:I'll have conversion tools.. Anyway, in another post I said I wanted to try out new ways of doing things in my OS, and not write yet another DOS or POSIX clone just for the sake of compatibility.
IMO, it's better to be better than to just be different.
I've already planned how it would work. My file open function would be:
Where the field would be the type, size, the data, etc. Depending on what you want to access within the file.
And I'll have a legacy fopen which defaults the field to "data". When a conventional file is copied the to file system, it just has all it's data placed in the "data" field until a program converts it to the new file-system's format. Live conversion tools will convert the format back when it is being copied to another file system.
There are a old OS called Pick, the filesystem are a DB, very practical and functional, I work with this in a 128kb computer with 2 terminals in 1990 in the army of my country (la colimba).
I think this is the beter aproach posible, the infinite levels of folders not help for speed
