replace big-tech and host basic infrastructure yourself! ordner.fyi
ordner docs developers concepts.md
7.0 kB · 129 lines
Markdown
at main

Concepts #

1. Intelligence lives in the application #

An ordner.fyi install keeps the person's files and pictures. It never looks inside them for meaning: no part of the install runs a model, classifies, recognises or embeds anything. What a picture shows, what a document says, who a face is, which pictures belong together: that is found by applications the person chooses to connect, on their own hardware or a server of their choosing, under the person's control.

The install keeps what applications found, indexes it at once and answers queries over it: full-text search over descriptions, labels and text, filters and sorts by fields an application declared, nearest-neighbour search over vectors an application wrote. Similarity is arithmetic over vectors applications wrote (cosine between vectors of one model), never a model of the install's. A text search "by meaning" therefore needs an application that embeds the query text with the same model it embedded the pictures with.

Whatever an application writes is used by the install's own web apps (Files and Photos) in the same moment, with no change to the install: searched by their search box, offered as their filters and sorts, shown in their details panels, listed as views, and pushed to every open page. An application that defines a new kind of metadata (a receipt's total, a document's due date, a scene) gets it searched, filtered, sorted and shown without anyone touching the suite: the server stores and indexes by type, never by meaning.

2. Two kinds of application, often one #

  • An annotator reads content and writes what it found: a description, labels with a confidence, the file's text, values of fields it declared, faces and who they are, vectors for similarity search, albums worth making. It works through a queue the server keeps for it: the files whose current content it has not annotated yet (Keeping up §1).
  • A front-end is a full client of Files and Photos: it reads every node, every picture's photo value, every application's annotations, faces, people, vectors and suggestions, keeps a local copy in step (Keeping up §2), and, when the person grants it, changes the drive as the web apps do: moves, renames, edits, albums, sharing. It may run models of its own over that copy, on the person's device.

There is no second, narrower API for outsiders: the install's web apps are clients of this same surface, with the same methods.

3. Sources: who wrote what #

Every annotation, face, vector, field schema, view and album suggestion is kept under its source, which the server derives from the credential that wrote it, never from anything the request claims:

  • the actor, always the drive's owner: an application acts as its person, and a delegate acts as the owner, so what Emma writes into Pip's drive as Pip's delegate is Pip's, and
  • the application: a registered relying party (by its client_id), an app password (by its id), or none for the person's own web session.

The source id is opaque and has the form <actorId>.<kind>.<id>, with kind app (a relying party), pw (an app password) or web (a web session, <actorId>.web.session). Pip's "Photo Sorter" writes as pip.app.rp-7Kq2mWv9XbTa, and so does Emma using the same application on Pip's drive: she acts as Pip, and only the application marks where a write came from. Her own app password writes there as pip.pw.<its id>, and her web session as pip.web.session, as if Pip had typed it. Wherever this API takes "the application" as an argument or a filter, it takes a source id.

A source is shown as { id, actor: { id, name }, app: { id, kind, name } | null }; app.name is the name the application registered (client_name), which nobody has checked, or an app password's label. The web apps say "unverified" beside it.

An application learns its own source id by writing (every write answers or records it, and AnnotationSchema/get lists it), or from its parts: in the person's own drive the actor is the account's owner, so the id is <accountId>.app.<client_id>.

4. What an application may write, and what it never changes #

An application with drive:annotate writes only what is its own:

  • annotations on any file (FileAnnotation/set): a description, labels, the file's text, values of its own fields;
  • faces on a picture (through FileAnnotation/set): regions, a name if it knows one, and an embedding when the person allowed face data;
  • vectors of a file's content (FileVector/set);
  • a field schema and views (AnnotationSchema/set, AnnotationView/set);
  • album suggestions (AlbumSuggestion/set).

An annotation never changes a file's content, name, place or sharing, and drive:annotate alone can change none of them. Each write names the content it was made from (blobId): when the file has changed since the application read it, the write is refused (contentChanged), and when it changes later, what the application wrote of the old content lapses: it is no longer answered, and the file is back in the application's queue.

What an application wrote stays apart from the person's own metadata. The person may keep a suggestion as their own (a label kept becomes a keyword of the picture, a description kept becomes the picture's description), or remove it; a face region the person named, moved or drew is the person's, and no application changes it again.

5. The person is in control #

  • The person chooses which applications to connect, and what each may do, line by line, on a consent page (Signing in §2).
  • Face embeddings identify people: they are biometric data and need their own consent line (drive:faces), which starts unticked.
  • Settings › Connected apps lists each application with what it may reach and do, when it last wrote and how much; each line can be withdrawn, the whole consent revoked.
  • Remove what it added destroys an application's annotations, the faces it found that the person never named, every embedding and vector it wrote, its views and its pending suggestions.
  • The drive's owner sees every source that wrote into their drive, their own and their delegates', and can remove what any of them added.

Privacy duties says what follows for an application.

6. What the server never does #

  • Run a model, recognise a face, classify or embed anything.
  • Trust a claim about who wrote something: the source is the credential's.
  • Let an application write into another source's annotation: it may only remove from it (Reference §4.2).
  • Put a face embedding into a file, a sidecar, a ZIP, a share or WebDAV.
  • Keep what an application wrote of content that is no longer the file's.
  • Send an application's text to the person as markup: titles, units, values, descriptions and labels are shown as plain text (Privacy duties §4).