Architecture #

Guide for contributors working on HappyView itself. For a user-facing overview, see the Introduction.

System overview #

Reads flow top-down through the query handler to the database (SQLite by default, or Postgres). Writes flow through the procedure handler to the user's PDS, then HappyView indexes the record locally. All record data enters the system through Tap, which handles both real-time firehose events and historical backfill. HappyView syncs collection filters to Tap and discovers repos via the relay for backfill, but Tap performs all record fetching.

Module overview #

src/
  main.rs           Startup: config, DB, migrations, build OAuth client, spawn Tap worker, start server
  lib.rs            AppState struct (incl. OAuth client + cookie key), module declarations
  config.rs         Environment variable loading
  dns.rs            DNS TXT resolver for atrium handle resolution
  error.rs          AppError enum (Auth, BadRequest, Forbidden, Internal, NotFound, PdsError)
  server.rs         Axum router: fixed routes + admin nest + auth routes + XRPC catch-all + static files
  lexicon.rs        ParsedLexicon, LexiconRegistry (Arc<RwLock<HashMap>>)
  profile.rs        DID document resolution, PDS discovery, profile fetching
  tap.rs            Tap WebSocket listener, collection filter sync, backfill delegation
  resolve.rs        NSID authority resolution (DNS TXT → DID → PDS)
  auth/
    mod.rs          Re-exports, COOKIE_NAME constant
    middleware.rs   Claims extractor (cookie auth, API key, or service auth JWT)
    routes.rs       OAuth endpoints (/auth/login, /auth/callback, /auth/logout, /auth/me)
    oauth_store.rs  Database-backed session and state stores for atrium-oauth
    service_auth.rs XRPC service-to-service JWT validation (ES256/ES256K)
  admin/
    mod.rs          Admin route definitions
    auth.rs         UserAuth extractor (Claims + DID lookup + permission check + auto-bootstrap)
    users.rs        User CRUD handlers (create, list, get, delete, update permissions, transfer super)
    permissions.rs  Permission enum (20 permissions), templates (Viewer, Operator, Manager, FullAccess)
    api_keys.rs     API key CRUD handlers (create, list, revoke) with scoped permissions
    events.rs       Event log query handler
    settings.rs     Instance settings CRUD handlers (list, upsert, delete, logo upload/serve)
    script_variables.rs  Script variable CRUD handlers (list, upsert, delete)
    lexicons.rs     Lexicon CRUD handlers
    network_lexicons.rs  Network lexicon tracking (add, list, remove)
    records.rs      Record listing handler
    stats.rs        Record count stats
    backfill.rs     Backfill job creation (relay discovery + Tap delegation)
    types.rs        Request/response structs for admin endpoints
  lua/
    mod.rs          Re-exports
    context.rs      Lua context globals (method, params, input, caller_did, collection)
    db_api.rs       Lua database API (db.query, db.get, db.count)
    execute.rs      Script execution and sandbox setup
    record.rs       Lua Record API (constructor, save, delete, load)
    sandbox.rs      Restricted Lua environment (removed modules, instruction limit)
    tid.rs          TID generation for Lua scripts
  repo/
    mod.rs          Re-exports
    pds.rs          PDS proxy helpers (JSON POST, blob POST, response forwarding via OAuth session)
    session.rs      OAuth session restoration from atrium store
    upload_blob.rs  Blob upload handler
  xrpc/
    mod.rs          Re-exports
    query.rs        Dynamic GET handler (Lua script or default: single record + list)
    procedure.rs    Dynamic POST handler (Lua script or default: create vs put)

Request flow #

Reads (queries) #

Client GET /xrpc/{method}?params
  -> xrpc::xrpc_get()
  -> LexiconRegistry lookup (must be Query type)
  -> If Lua script attached: execute script (has access to db API)
  -> Else: default SQL query on records table (collection from target_collection)
  -> JSON response

Writes (procedures) #

Client POST /xrpc/{method} + session cookie or Bearer token
  -> Claims extractor (cookie, API key, or service auth JWT)
  -> xrpc::xrpc_post()
  -> LexiconRegistry lookup (must be Procedure type)
  -> If Lua script attached: execute script (has access to Record API)
  -> Else: default create/update (auto-detect based on uri field)
  -> Restore OAuth session from atrium store (by DID)
  -> atrium handles DPoP proof generation and token refresh
  -> Proxy to user's PDS (createRecord or putRecord)
  -> Upsert record locally
  -> Forward PDS response

Admin endpoints #

Client request + session cookie or Bearer token
  -> AdminAuth extractor:
     1. Claims validation (cookie, API key, or service auth JWT)
     2. DID lookup in users table (auto-bootstrap super user if empty)
     3. Permission check (403 if missing required permission)
  -> Admin handler
  -> JSON response

Data flow #

Real-time indexing #

Tap WebSocket connection (tap::spawn)
  -> Collection filters synced to Tap on startup and lexicon changes
  -> Record events:
     create/update -> UPSERT into records table
     delete        -> DELETE from records table
  -> Lexicon schema events (com.atproto.lexicon.schema):
     -> Update tracked network lexicons in DB and registry
  -> Reconnects automatically on errors or collection filter changes

Backfill #

POST /admin/backfill
  -> Create backfill_jobs record (status = running)
  -> Relay listReposByCollection -> list of DIDs
  -> Send DIDs to Tap in batches of 1000 (POST /repos/add)
  -> Mark job as completed
  -> Tap fetches records asynchronously and delivers via WebSocket

Database schema #

records #

Column	Type	Description
uri	text (PK)	AT URI (at://did/collection/rkey)
did	text	Author DID
collection	text	Lexicon NSID
rkey	text	Record key
record	jsonb	Record value
cid	text	Content identifier
indexed_at	timestamptz	When HappyView indexed this record

lexicons #

Column	Type	Description
id	text (PK)	Lexicon NSID
revision	integer	Incremented on upsert
lexicon_json	jsonb	Raw lexicon definition
lexicon_type	text	record, query, procedure, definitions
backfill	boolean	Whether to backfill on upload
target_collection	text	For queries/procedures: which record collection
created_at	timestamptz
updated_at	timestamptz

users #

Column	Type	Description
id	uuid (PK)
did	text (unique)	User's AT Protocol DID
is_super	boolean	Whether this is the super user (only one allowed)
created_at	timestamptz
last_used_at	timestamptz	Updated on each authenticated request

user_permissions #

Column	Type	Description
user_id	uuid (FK)	References users.id
permission	text	Permission string (e.g. lexicons:create)
(PK)		Composite primary key: (user_id, permission)

api_keys #

Column	Type	Description
id	uuid (PK)
user_id	uuid (FK)	References users.id
name	text	Descriptive label
key_hash	text	SHA-256 hash of the full key
key_prefix	text	First 11 characters for display
permissions	text[]	Permissions granted to this key
created_at	timestamptz
last_used_at	timestamptz
revoked_at	timestamptz	Set when revoked (soft delete)

oauth_sessions #

Column	Type	Description
did	text (PK)	User's AT Protocol DID
session_data	text	Serialized OAuth session (managed by atrium)
created_at	timestamptz
updated_at	timestamptz

oauth_state #

Column	Type	Description
state_key	text (PK)	OAuth state parameter
state_data	text	Serialized state (managed by atrium)
created_at	timestamptz

instance_settings #

Column	Type	Description
key	text (PK)	Setting name (e.g. app_name)
value	text	Setting value
updated_at	timestamptz	Last modified

event_logs #

Column	Type	Description
id	uuid (PK)
event_type	text	Category.action format (e.g. user.created)
severity	text	info, warn, or error
actor_did	text	DID of the user who triggered the event
subject	text	What was affected (DID, NSID, URI, etc.)
detail	jsonb	Event-specific data
created_at	timestamptz

script_variables #

Column	Type	Description
key	text (PK)	Variable name
value	text	Variable value (encrypted at rest)
created_at	timestamptz
updated_at	timestamptz

backfill_jobs #

Column	Type	Description
id	uuid (PK)
collection	text	Target collection (null = all)
did	text	Target DID (null = all)
status	text	pending, running, completed, failed
total_repos	integer
processed_repos	integer
total_records	integer
error	text	Error message if failed
started_at	timestamptz
completed_at	timestamptz
created_at	timestamptz

Testing #

# Unit tests (no database needed)
cargo test --lib

# All tests including end-to-end (SQLite by default)
cargo test

# Or run against Postgres
docker compose -f docker-compose.test.yml up -d
TEST_DATABASE_URL=postgres://happyview:happyview@localhost:5433/happyview_test cargo test
docker compose -f docker-compose.test.yml down

End-to-end tests use wiremock to mock external services (PLC directory, PDSes) and a real database for full integration coverage. By default tests use SQLite; set TEST_DATABASE_URL to a Postgres connection string to test against Postgres.