oppi.li
1
2`rainbow`: atproto Firehose Fanout Service
3==========================================
4
5This is an atproto service which consumes from a firehose (eg, from a relay or PDS) and fans out events to many subscribers.
6
7Features and design points:
8
9- retains "backfill window" on local disk (using [pebble](https://github.com/cockroachdb/pebble))
10- serves the `com.atproto.sync.subscribeRepos` endpoint (WebSocket)
11- proxies through public and administrative API requests to the backing host
12- retains upstream firehose "sequence numbers"
13- does not validate events (signatures, repo tree, hashes, etc), just passes through
14- does not archive or mirror individual records or entire repositories (or implement related API endpoints)
15- somewhat disk I/O intensive: fast NVMe disks are recommended, and RAM is helpful for caching
16- single golang binary for easy deployment
17- observability: logging, prometheus metrics, and OTEL traces
18
19## Running
20
21This is a simple, single-binary Go program. You can also build and run it as a docker container (see `./Dockerfile`).
22
23From the top level of this repo, you can build:
24
25```shell
26go build ./cmd/rainbow -o rainbow
27```
28
29or just run it, and see configuration options:
30
31```shell
32go run ./cmd/rainbow --help
33```