# Updraft Updraft is a Bluesky feed generator framework. You define feeds as [Quamina](https://github.com/timbray/quamina) patterns. Updraft reads the [Jetstream](https://github.com/bluesky-social/jetstream) firehose, stores matching posts in SQLite, and serves `app.bsky.feed.getFeedSkeleton`. It is a single binary, meant to run behind a reverse proxy such as Caddy. ## Build ```sh go build -o updraft ./cmd/updraft ``` Updraft needs Go 1.26 or later. It does not use cgo, so it cross-compiles with `GOOS`/`GOARCH`. ## Configuration ### Environment variables | Variable | Default | Purpose | |---|---|---| | `UPDRAFT_HOSTNAME` | required | Public host name, e.g. `feeds.example.com` | | `UPDRAFT_PUBLISHER_DID` | required | DID of the account whose repo holds the feed records | | `UPDRAFT_SERVICE_DID` | `did:web:$UPDRAFT_HOSTNAME` | The feed generator's service DID | | `UPDRAFT_LISTEN_ADDR` | `127.0.0.1:3000` | HTTP listen address | | `UPDRAFT_DB_PATH` | `updraft.db` | SQLite database path | | `UPDRAFT_FEEDS_FILE` | `feeds.toml` | Feed definitions file | | `UPDRAFT_JETSTREAM_HOST` | `jetstream.us-east.bsky.network` | Jetstream instance to read from | ### feeds.toml See [examples/feeds.toml](examples/feeds.toml). Settings in `[default]` apply to every feed, and any feed can override them. If neither `[default]` nor the feed sets a value, Updraft uses its built-in default. | Key | Built-in default | Meaning | |---|---|---| | `pattern` | (none) | Path to the feed's Quamina pattern file, relative to `feeds.toml`. Set this or `rules` | | `page_size` | `30` | Posts per page when the request omits `limit` (1 to 100) | | `max_age` | `"48h"` | Posts older than this are purged (Go duration syntax) | | `algorithm` | `"chronological"` | `"chronological"` (newest first) or `"hot"` | | `gravity` | `0.1` | Used by `hot` only. The hourly cooling rate; heat halves every `0.693 / gravity` hours | | `track_likes` | `false` | Count likes on the feed's posts. `hot` requires this | Each feed is a `[feeds.]` table. `` is the feed's record key, so the feed's URI is `at:///app.bsky.feed.generator/`. ### Patterns Updraft runs a pattern only against post-create events, so a pattern does not need to filter on collection or operation. Each event is matched as JSON in the Jetstream Go SDK's `Event` shape: ```json { "did": "did:plc:...", "cursor": 27093221882, "time_us": 1794466298861793, "kind": "commit", "commit": { "operation": "create", "collection": "app.bsky.feed.post", "rkey": "3n23uz67ors2a", "rev": "...", "cid": "bafy...", "record": { "$type": "app.bsky.feed.post", "text": "...", "langs": ["en"], "createdAt": "..." } } } ``` For example, this pattern matches English posts that mention golang: ```json {"commit": {"record": {"text": [{"shellstyle": "*golang*"}], "langs": ["en"]}}} ``` Values in a pattern array are alternatives, and every field in the pattern must match. Quamina matching is case-sensitive, so list case variants separately. See Quamina's [PATTERNS.md](https://github.com/timbray/quamina/blob/main/PATTERNS.md) for the full syntax (`prefix`, `shellstyle`, `wildcard`, `anything-but`, `equals-ignore-case`, `exists`, `regexp`). ### Rules When a single pattern can't describe a feed, list `rules` instead of `pattern`. An event that satisfies any rule adds that rule's target to the feed. A rule is satisfied when all of its patterns match. ```toml [feeds.dudes] max_age = "24h" [[feeds.dudes.rules]] patterns = ["patterns/dudes-reply.json"] target = "commit.record.reply.parent" [[feeds.dudes.rules]] patterns = ["patterns/dudes-quote.json"] target = "commit.record.embed.record" [[feeds.standard.rules]] patterns = ["patterns/standard-publication.json", "patterns/standard-document.json"] ``` | Key | Meaning | |---|---| | `patterns` | Paths to Quamina pattern files, relative to `feeds.toml`. All of them must match | | `target` | Dot path, from the event root, to a `{uri, cid}` reference naming the post to add. Without it, the rule adds the matched post | `pattern = "p.json"` is shorthand for one rule with `patterns = ["p.json"]` and no `target`. A target is skipped unless it is an `app.bsky.feed.post` URI, so a rule targeting quotes ignores quoted lists, feeds and starter packs. A quote with images or video nests the quoted post one level deeper, at `commit.record.embed.record.record`, so it needs its own rule. If an event satisfies several rules, every target is added. A post that is already in the feed keeps the time it was first added. ## Ranking - **chronological**: newest first, using the Jetstream event time. - **hot**: Evan Miller's [Newton's law of cooling](https://www.evanmiller.org/rank-hotness-with-newtons-law-of-cooling.html). A post starts with a heat of 1. Each like cools the stored heat to the time of the like (`heat * exp(-gravity * hours)`) and then adds 1. Posts are ranked by heat cooled to the time of the request. A recent like counts for more than an old one. Likes are counted for posts in feeds with `track_likes = true`. Updraft only subscribes to `app.bsky.feed.like` when at least one feed tracks likes. ## Firehose behavior - After each batch, Updraft writes the posts and the Jetstream cursor in one SQLite transaction. On restart it resumes from the saved cursor, so nothing is missed while it is down, up to the Jetstream instance's lookback window (36 hours on Bluesky-hosted instances). - When a post is deleted on the firehose, Updraft removes it from every feed. - Expired posts (older than `max_age`) are purged every 10 minutes, and once right after startup. ## HTTP endpoints - `GET /xrpc/app.bsky.feed.getFeedSkeleton?feed=&limit=&cursor=` - `GET /xrpc/app.bsky.feed.describeFeedGenerator` - `GET /.well-known/did.json`, served only when the service DID is `did:web:$UPDRAFT_HOSTNAME` Example Caddy configuration: ``` feeds.example.com { reverse_proxy 127.0.0.1:3000 } ``` ## Publishing a feed Updraft does not publish feed records. For each feed, create an `app.bsky.feed.generator` record in the publisher account's repo. Use the feed name as the record key and set `did` to the service DID. For example, use the `publishFeedGen` script from [bluesky-social/feed-generator](https://github.com/bluesky-social/feed-generator).