a bluesky feed generator framework
updraft README.md
6.4 kB · 167 lines
Markdown
at main

Updraft #

Updraft is a Bluesky feed generator framework. You define feeds as Quamina patterns. Updraft reads the Jetstream firehose, stores matching posts in SQLite, and serves app.bsky.feed.getFeedSkeleton. It is a single binary, meant to run behind a reverse proxy such as Caddy.

Build #

go build -o updraft ./cmd/updraft

Updraft needs Go 1.26 or later. It does not use cgo, so it cross-compiles with GOOS/GOARCH.

Configuration #

Environment variables #

Variable Default Purpose
UPDRAFT_HOSTNAME required Public host name, e.g. feeds.example.com
UPDRAFT_PUBLISHER_DID required DID of the account whose repo holds the feed records
UPDRAFT_SERVICE_DID did:web:$UPDRAFT_HOSTNAME The feed generator's service DID
UPDRAFT_LISTEN_ADDR 127.0.0.1:3000 HTTP listen address
UPDRAFT_DB_PATH updraft.db SQLite database path
UPDRAFT_FEEDS_FILE feeds.toml Feed definitions file
UPDRAFT_JETSTREAM_HOST jetstream.us-east.bsky.network Jetstream instance to read from

feeds.toml #

See examples/feeds.toml. Settings in [default] apply to every feed, and any feed can override them. If neither [default] nor the feed sets a value, Updraft uses its built-in default.

Key Built-in default Meaning
pattern (none) Path to the feed's Quamina pattern file, relative to feeds.toml. Set this or rules
page_size 30 Posts per page when the request omits limit (1 to 100)
max_age "48h" Posts older than this are purged (Go duration syntax)
algorithm "chronological" "chronological" (newest first) or "hot"
gravity 0.1 Used by hot only. The hourly cooling rate; heat halves every 0.693 / gravity hours
track_likes false Count likes on the feed's posts. hot requires this

Each feed is a [feeds.<name>] table. <name> is the feed's record key, so the feed's URI is at://<UPDRAFT_PUBLISHER_DID>/app.bsky.feed.generator/<name>.

Patterns #

Updraft runs a pattern only against post-create events, so a pattern does not need to filter on collection or operation. Each event is matched as JSON in the Jetstream Go SDK's Event shape:

{
  "did": "did:plc:...",
  "cursor": 27093221882,
  "time_us": 1794466298861793,
  "kind": "commit",
  "commit": {
    "operation": "create",
    "collection": "app.bsky.feed.post",
    "rkey": "3n23uz67ors2a",
    "rev": "...",
    "cid": "bafy...",
    "record": { "$type": "app.bsky.feed.post", "text": "...", "langs": ["en"], "createdAt": "..." }
  }
}

For example, this pattern matches English posts that mention golang:

{"commit": {"record": {"text": [{"shellstyle": "*golang*"}], "langs": ["en"]}}}

Values in a pattern array are alternatives, and every field in the pattern must match. Quamina matching is case-sensitive, so list case variants separately. See Quamina's PATTERNS.md for the full syntax (prefix, shellstyle, wildcard, anything-but, equals-ignore-case, exists, regexp).

Rules #

When a single pattern can't describe a feed, list rules instead of pattern. An event that satisfies any rule adds that rule's target to the feed. A rule is satisfied when all of its patterns match.

[feeds.dudes]
max_age = "24h"

[[feeds.dudes.rules]]
patterns = ["patterns/dudes-reply.json"]
target   = "commit.record.reply.parent"

[[feeds.dudes.rules]]
patterns = ["patterns/dudes-quote.json"]
target   = "commit.record.embed.record"

[[feeds.standard.rules]]
patterns = ["patterns/standard-publication.json", "patterns/standard-document.json"]
Key Meaning
patterns Paths to Quamina pattern files, relative to feeds.toml. All of them must match
target Dot path, from the event root, to a {uri, cid} reference naming the post to add. Without it, the rule adds the matched post

pattern = "p.json" is shorthand for one rule with patterns = ["p.json"] and no target.

A target is skipped unless it is an app.bsky.feed.post URI, so a rule targeting quotes ignores quoted lists, feeds and starter packs. A quote with images or video nests the quoted post one level deeper, at commit.record.embed.record.record, so it needs its own rule. If an event satisfies several rules, every target is added. A post that is already in the feed keeps the time it was first added.

Ranking #

  • chronological: newest first, using the Jetstream event time.
  • hot: Evan Miller's Newton's law of cooling. A post starts with a heat of 1. Each like cools the stored heat to the time of the like (heat * exp(-gravity * hours)) and then adds 1. Posts are ranked by heat cooled to the time of the request. A recent like counts for more than an old one.

Likes are counted for posts in feeds with track_likes = true. Updraft only subscribes to app.bsky.feed.like when at least one feed tracks likes.

Firehose behavior #

  • After each batch, Updraft writes the posts and the Jetstream cursor in one SQLite transaction. On restart it resumes from the saved cursor, so nothing is missed while it is down, up to the Jetstream instance's lookback window (36 hours on Bluesky-hosted instances).
  • When a post is deleted on the firehose, Updraft removes it from every feed.
  • Expired posts (older than max_age) are purged every 10 minutes, and once right after startup.

HTTP endpoints #

  • GET /xrpc/app.bsky.feed.getFeedSkeleton?feed=&limit=&cursor=
  • GET /xrpc/app.bsky.feed.describeFeedGenerator
  • GET /.well-known/did.json, served only when the service DID is did:web:$UPDRAFT_HOSTNAME

Example Caddy configuration:

feeds.example.com {
	reverse_proxy 127.0.0.1:3000
}

Publishing a feed #

Updraft does not publish feed records. For each feed, create an app.bsky.feed.generator record in the publisher account's repo. Use the feed name as the record key and set did to the service DID. For example, use the publishFeedGen script from bluesky-social/feed-generator.