OK, let's go over how Xanadu is intended to work, and how I intend to apply the ideas, if not always the methods, thereof.
As I already said, the point of Xanadu is to replace files. Much of what Nelson & Co. are doing
can be done with existing file systems, and in fact several of the iterations of the project ran on top of existing file systems or RDBMS systems, but doing them with existing files involves a lot of ad-hoc work. Xanadu, at least in the file-system based forms, is essentially a library for doing just that, though part of how it does that is to create what amounts to a separate database system on top of the existing ones - which is why the real eventual goal was to implement it in a more stand-alone form, working on the media directly rather than through the file system.
In the 1988 and 1993 designs of Xanadu (and presumably in OpenXanadu, though despite the name not all the code is exposed yet AFAICT), there is supposed a Back End and a Front End, with the Front-End/Back End (FEBE) protocol between them. The BE handles managing the storage and retrieval of data fragments, both locally and remotely, keeps track of what is where and which things are stored or mirrored locally, and maintaining coherence across distributed storage and caching. It is a hairy piece of work and while its operations are secondary to the
goals of Xanadu, it is at the heart of the
implementation of those goals.
My understanding is that the FE handles the decisions about which fragments to request at a given time. Note that the FE is a front end to the
applications, not some equivalent of a browser - it is primarily an API for the BE, though it does do some management of the data views as well.
Presentation to the user is up to the applications themselves, and to the display manager, which is supposed to permit various ways of display connections between applications to the user. At this point, you can probably see part of what most people get confused by in all of this, as there is no single 'browser' anywhere in all of this.
Basically, when a new datum is created - whether it is a 'text document', a 'spreadsheet document', a 'saved game record', an 'image', or what have you - the FE passes it to the BE, which it encrypts in some way and then writes to some storage media - possibly as part of a journal that contains data from several other applications and users.
Along with the data, the FE passes the BE information about the datum and it source. If it is part of a larger 'document' - which is usually going to be the case - it includes information about the document, including a link to the address of any related data, and how they are related. For example, for a 'word processing' application, it might pass a link to the datum which was, at the time of editing, the immediate predecessor of the datum being stored, and that the datum was (again, when created) the successive item in a larger document.
The BE catalogs each of the recently written data according to the format the datum is in, its size, the user who created it, local date and time of creation, the application it originated in, the encryption type - all things that a conventional file system may or may not record - but also the publication status (and later, publication history), the current (and later, previous) owners/maintainers of the datum, the current location it is stored in, and whether to mirror it elsewhere (which is the default for most things).
Up to this point, it looks normal. Here is where it changes course a bit.
The BE generates a permanent address link for the datum, one which is independent of its current location in storage. This is a key point, because the storage location itself is only an ephemeris to the system - while the datum is meant to be treated as immutable, and the parent copy should never be overwritten, the actual physical image of the datum in the storage medium isn't the datum itself. This is also why, for networked systems, automatic mirroring is the default (and why it being encrypted - and the fact that the encryption methods can vary from datum to datum or even copy to copy of the same datum - is important).
A large part of this is to abstract away, from the perspective of the FE, the applications, and the user, the process of storing, transmitting, mirroring, and caching the data. As far as everything outside of the BE is concerned, the datum is (or should be) immutable and eternal, approximating a Platonic essence of the idea it encodes. The reality is obviously more complicated, but the system is meant to bend over backwards to maintain that illusion, across the entire 'docuverse' straddling the network.
(So far, it hasn't quite managed this, and perhaps never will, but in terms of its goals, it goes further than any other system that I know of.)
Now, you may have noted that I haven't talked about links, hyper or otherwise, yet. This is where things go even further out of the norm, because the Xanadu idea of a 'hyperlink' has nothing much at all to do with the hyperlinks of things like the WWW.
In Xanadu, there are several types of links, most of which are not directly related to how the datum is presented to the user. The particular kind of links in consideration right now might be called 'resolution links', which describe the physical location(s) of the data; and 'association links', which store how two or more data relate to each other (these aren't the terms used by the Project, but explaining their terms would take hundreds of pages, and I only know a fraction of the terminology myself). The former are ephemeral, relating to the specific physical storage, and are stored as the equivalent of a FAT or an i-node structure, while the latter are permanent, and have their own resolution links when they are themselves stored.
Some of the types of association links are:
- 'span links', which refers to a slice or section out of the datum, allowing just the relevant sections to be referenced in documents or transferred across a network, without having to serve the whole document - the 'whole document' is itself just a series of different kinds of association links.
- 'permutation-of-order links', which are used to manipulate the structure of the document, creating a view - or collection of views - which can themselves be stored and manipulated. This relates to the immutability of data - rather than changing the data when updating, the FE permutes the order of the links that make up the 'document' or view, and pass that permutation to the BE to record it. This, among other things, serves as both persistent undo/redo, and as version control.
- 'structuring links', which describe the layout of the view independent of the data and the ordering thereof. This acts as out-of-band markup, among other things - the markup is not part of the datum itself.
- 'citation links', which represent a place where a user wanted to record a connection between two ideas. This link associates bi-directionally, and has its own separate publication and visibility which is partially dependent of that of the data - an application, and hence a user, can view any citation link IIF they have permission to view both the citation and all of the data it refers to. There many also be 'meta-citations' which aggregate several related citations, but I don't know if that was something actually planned or just something discussed - since citations are themselves data, and all data are first-class citizens, such a meta-citation would just be a specific case of a view.
It is important to recall that the 'views' in question are to the
applications and the display manager. They can then organize the actual user display based on those views into the data as needed. The same data - or even the same views - may be shown as part of a 'text document' by one application, as set of spreadsheet cells by another, or composed with some image in yet another. This is why markup is out-of-band, and why structuring links applied to a given set of data are stored for later use by the applications.
There are still other links for recording the history of the datum's ownership and publication status, connecting a data format to one or more means of interpreting the format or transposing it into another format, indirection (to allow for updating of views - since most links available to the FE are immutable, these allow for the equivalent of a VCS repo's 'HEAD' branch, allowing the applications to fetch whatever the latest version of a document is and separating 'currently published' from 'previously published'), tracking where copies of a given datum can be found for the purposes of caching and Torrent-like network distribution, and so forth, but most of those are only for use internally by the BE.
When the new datum is created as part of some new document, a new association link is created to connect it to that document, which is then passed back to the FE for use by the application. The FE then creates a permutation link for the document, incorporating the datum into the document link traces, which is then passed back to the BE for storage.
Moving on...