app / ARCHITECTURE.md

# Architecture

## Project Structure

Monorepo using Bun workspaces. Each module (feeds, polls, forms, etc.) is an independent package with its own backend routes and frontend components. Modules share common infrastructure but can be developed and deployed independently.

```
exosphere/
├── packages/
│   ├── core/              # Shared server-side infrastructure
│   │   ├── auth/          # AT Protocol OAuth, session management
│   │   ├── db/            # SQLite setup, base migrations
│   │   └── types/         # Common types and Zod schemas
│   ├── client/            # Shared client-side utilities
│   │   └── src/           # API helpers, auth, router, hooks, theme, styles, types
│   ├── indexer/            # Jetstream consumer & shared module registry
│   │   ├── src/
│   │   │   ├── modules.ts  # Shared module list (used by app + standalone)
│   │   │   ├── start.ts    # startJetstream() — WebSocket consumer
│   │   │   ├── cursor.ts   # Cursor persistence for resume-after-crash
│   │   │   └── main.ts     # Standalone entry point
│   ├── <module>/             # Feed & discussions module
│   │   ├── api/           # Hono routes
│   │   ├── ui/            # Preact page components
│   │   └── schemas/       # Module-specific types and schemas
│   └── app/               # Host app — assembles selected modules
├── bunfig.toml
└── package.json
```

The `app` package is the host: it imports the desired modules, mounts their routes and UI, and produces the final deployable artifact. A Sphere operator picks which modules to enable.

## Stack

### Backend

- **Runtime**: Bun
- **Framework**: Hono.js
- **Database**: SQLite (via `bun:sqlite`)
- **Auth**: AT Protocol OAuth via `@atproto/*` packages

### Frontend

- **Framework**: Preact
- **State management**: Preact Signals
- **Styling**: vanilla-extract

### Shared (core)

- **Language**: TypeScript with strict types
- **Validation**: Zod schemas at system boundaries

## Module Design

Each module:

- Exposes a Hono sub-app for its API routes
- Exposes Preact components for its UI
- Manages its own SQLite tables (migrations scoped per module)
- Can depend on `core` and `client` but not on other modules

### Server / Client Separation

Backend and frontend code must stay in separate entry points to avoid bundling server-only dependencies (e.g. `bun:sqlite`) into the browser bundle.

Each module exposes two entry points in its `package.json` exports:

| Export       | File            | Purpose                                                                            |
| ------------ | --------------- | ---------------------------------------------------------------------------------- |
| `"."`        | `src/index.ts`  | Backend — `ExosphereModule` with Hono API routes. Imported by `app/src/server.ts`. |
| `"./client"` | `src/client.ts` | Frontend — `ClientModule` with page routes. Imported by `app/src/app.tsx`.         |

`ExosphereModule` (backend) and `ClientModule` (frontend) are independent types — `ClientModule` does not extend `ExosphereModule` to prevent any transitive import of server code.

### Client Module and Route Registration

The `@exosphere/client` package (`packages/client`) provides shared client-side utilities (API helpers, auth state, router, hooks, theme, styles) and the `ClientModule` type:

```typescript
interface ModuleRoute {
  path: string; // e.g. "/feeds" or "/feature-requests/:number"
  component: ComponentType;
}

interface ClientModule {
  name: string;
  routes?: ModuleRoute[];
}
```

Each module's `src/client.ts` exports a `ClientModule` that declares its routes and page components. The host app collects all client modules into an array and maps them to `<Route>` elements inside a preact-iso `<Router>` — adding a new module's pages requires only importing its `ClientModule` and appending it to the array.

### Server-Side Rendering (SSR)

The app server-renders pages so the browser receives ready-to-display HTML. SSR runs in both dev (via a Vite plugin) and production (Bun serves pre-built assets).

#### How it works

1. **Template** — the built `index.html` is loaded once at startup. In production, all CSS from the Vite manifest is injected as render-blocking `<link>` tags to prevent FOUC.
2. **Data prefetch** — for each request, the server resolves auth (from the `sid` cookie), loads the current Sphere, and calls its own API routes internally (`app.request()`) to prefetch page data. Dependent prefetches (e.g. comments after a feature request) run sequentially.
3. **Render** — `entry-server.tsx` sets Preact signals (`auth`, `sphereState`, `ssrPageData`) from the prefetched data, then calls `preact-iso/prerender` to produce HTML.
4. **Hydration** — the HTML and serialized `__SSR_DATA__` are injected into the template. The client reads `__SSR_DATA__` to populate signals before hydration, so components skip their initial fetch via `useQuery`'s `initialData` option.

#### Eager vs lazy imports

Module routes use `lazy()` (from `preact-iso`) on the client for code splitting. During SSR, lazy components throw promises that the router can't resolve. Each module therefore exposes a `./client-ssr` entry with direct (eager) imports of the same page components. `entry-server.tsx` imports these eager modules; the client entry uses the default lazy ones.

| Entry point            | Module imports  | Code splitting |
| ---------------------- | --------------- | -------------- |
| `src/client.tsx`       | Lazy (`lazy()`) | Yes            |
| `src/entry-server.tsx` | Eager (direct)  | N/A            |

#### Dev vs production

- **Dev** — a Vite plugin (`vite-ssr-plugin.ts`) intercepts page requests before the SPA fallback. It fetches auth/sphere/page data from the running API server (`localhost:3001`) via HTTP, SSR-renders via `server.ssrLoadModule`, and inlines CSS collected from the module graph.
- **Production** — the Hono catch-all route in `server.ts` handles SSR. The template and CSS links are prepared once at startup. Data prefetch uses `app.request()` to call API routes in-process (no HTTP round-trip).
- Both environments share the same prefetch logic (`ssr-prefetch.ts`) — only the transport differs (HTTP in dev, in-process in prod).

## AT Protocol Integration

- Authentication via AT Protocol OAuth (handled in `core/auth`)
- User data (posts, votes, etc.) is written to each user's PDS
- The backend indexes relevant data from PDS for fast querying
- Sphere configuration and membership are published on PDS for interoperability (see below)
- The local SQLite database caches/indexes all PDS data for fast querying
- Every record type written to a user's PDS has a formal Lexicon schema definition hosted at `landing/.well-known/`

### Lexicon schemas

All AT Protocol record types are defined as Lexicon JSON files:

| Lexicon ID                                 | Published by | Purpose                                                                  |
| ------------------------------------------ | ------------ | ------------------------------------------------------------------------ |
| `site.exosphere.sphere`                    | Sphere owner | Sphere declaration (name, slug, visibility, modules) — enables discovery |
| `site.exosphere.sphereMember`              | Member       | "I am a member of this Sphere" — member-side of bilateral membership     |
| `site.exosphere.sphereMemberApproval`      | Owner/admin  | "This user is an approved member" — admin-side of bilateral membership   |
| `site.exosphere.featureRequest`            | Author       | Feature request content                                                  |
| `site.exosphere.featureRequestVote`        | Voter        | Upvote on a feature request                                              |
| `site.exosphere.featureRequestComment`     | Commenter    | Comment on a feature request                                             |
| `site.exosphere.featureRequestCommentVote` | Voter        | Upvote on a comment                                                      |
| `site.exosphere.featureRequestStatus`      | Admin/owner  | Status change on a feature request                                       |
| `site.exosphere.moderation`                | Admin/owner  | Moderation action on any content (e.g. comment removal)                  |

## Data Ownership

- Logged-in users own their content via their PDS
- The local SQLite database acts as a cache/index, not the source of truth for user content
- Sphere configuration lives on the owner's PDS, membership is bilateral (both parties publish records)
- A third party can reconstruct any public Sphere entirely from PDS data — no dependency on a specific Exosphere instance

### PDS as source of truth

For public Spheres, **no meaningful action should exist only in the local database**. Every user action (creating content, voting, commenting) and every admin action (status changes, moderation, role updates) must be represented as an AT Protocol record on someone's PDS. The local SQLite database indexes these records for fast querying, but it is always rebuildable from PDS data.

This means:

- **User actions** are published on the **user's PDS** (the user owns their data).
- **Admin actions** that affect other users' content (e.g. removing a comment, changing a feature request's status) are published on the **admin's PDS** — not by modifying or deleting the original author's record. The admin has no authority over another user's PDS repo.
- **Moderation** follows this pattern: when an admin removes content, they publish a `site.exosphere.moderation` record on their own PDS referencing the content's AT URI. The original content stays on the author's PDS. The local DB marks the content as hidden for fast filtering, but the moderation decision is reconstructable from protocol data.

If a piece of state exists only in SQLite with no corresponding PDS record, it cannot survive a database rebuild and is invisible to third-party indexers. The exception is **private Spheres**, where content intentionally stays off-protocol (AT Protocol repos are public by design).

## Deployment

- Docker container for self-hosting
- Single image containing both backend and frontend (backend serves static frontend assets)

## Sphere Access Models

A Sphere can be configured as private or public, each with different data storage and access patterns.

### Sphere declaration on PDS

When a Sphere is created, the owner publishes a `site.exosphere.sphere` record on their PDS. This makes the Sphere discoverable by anyone crawling the AT Protocol network and serves as the canonical reference that other records (membership, content) point to via AT URI.

### Private Spheres

Private Spheres are invitation-only. All content is hidden from the public.

- **Authentication**: Required. Users must authenticate with their AT Protocol DID.
- **Data storage**: All content (posts, reactions, comments, polls, etc.) is stored in the Sphere's local SQLite database — not on users' PDS. AT Protocol repos are public by design, so private content must stay off-protocol.
- **Sphere record**: The Sphere declaration is still published on the owner's PDS (with `visibility: "private"`), making it discoverable but not its content.
- **Identity**: Users are still identified by their AT Protocol DID, which provides portable identity and verification without exposing content to the public network.

```
User (authenticated via AT Proto OAuth)
  → Sphere API (checks membership)
    → SQLite (read/write private data)
```

### Public Spheres

Public Spheres content is readable by anyone. Write access depends on the Sphere's configuration.

#### Authentication

All interactions (posts, reactions, comments) require AT Protocol authentication. Users log in via AT Protocol OAuth and their content is written to their PDS as records using Exosphere's Lexicon schemas. This makes user data portable and user-owned.

Public Spheres are publicly **readable** without authentication — anyone with the URL can view the content. But all **write** actions require a DID.

#### Future: unauthenticated write access

Unauthenticated write access (e.g. anonymous poll votes, guest comments) is deferred to a later phase. It would allow users without an AT Protocol account to interact with specific modules, potentially helping adoption by lowering the entry barrier.

However, it introduces a hybrid data architecture: the AppView would need to merge data from two sources (user PDS records and local SQLite records) into a single unified view. This adds complexity to every query path and leaks into every module's implementation.

When revisited, unauthenticated write should be:

- Opt-in per module (not a core concern)
- Limited to modules where it clearly adds value (polls, forms) rather than applied broadly
- Designed with account linking in mind (what happens when an anonymous user later signs up)

#### Write access modes

- **Open**: Any authenticated user can participate (react, comment, post).
- **Members only**: Public read, permissioned write. Anyone can view the content, but only invited members can interact (react, comment, post).

Important: AT Protocol cannot prevent a user from writing records to their own PDS that reference a Sphere's content. Write restrictions are enforced at the AppView level — only interactions from approved members are indexed and displayed. Non-member interactions are silently ignored.

## AppView and Data Consumption

The AppView is the service that consumes, indexes, and serves AT Protocol data for a Sphere. It is the layer where access control and data aggregation happen.

### Jetstream

Jetstream is a lightweight alternative to the full AT Protocol firehose. Instead of decoding binary CBOR/CAR data, it provides a WebSocket stream of JSON events.

- The AppView connects to a Jetstream instance and subscribes to events matching Exosphere's Lexicon record types.
- Jetstream can be consumed from Bluesky's public instances, or self-hosted for full independence.
- Self-hosting Jetstream also requires running a relay. For most deployments, using a public Jetstream instance is sufficient — the Sphere still self-hosts all indexing, storage, and access control.

### Indexing pipeline

The indexer lives in its own package (`packages/indexer`) and can run either **in-process** with the API server or as a **standalone service**. Both modes use the same `startJetstream()` function and share the same module registry (`packages/indexer/src/modules.ts`).

#### Deployment modes

| Mode                   | Command                           | Description                                                                                     |
| ---------------------- | --------------------------------- | ----------------------------------------------------------------------------------------------- |
| **Combined** (default) | `bun run start`                   | API server starts the indexer in-process. Simplest deployment — one process, one start command. |
| **API only**           | `DISABLE_INDEXER=1 bun run start` | API server without the Jetstream consumer. Use when the indexer runs separately.                |
| **Indexer only**       | `bun run start:indexer`           | Standalone Jetstream consumer. Useful for scaling or isolating the indexer from the API.        |

The shared module registry (`@exosphere/indexer/modules`) is the single source of truth for which modules are loaded. Both the API server and the standalone indexer import from it, so adding a new module requires updating only one file.

#### Event flow

For Public Spheres with authenticated users, the AppView indexes data from the AT Protocol network:

```
Jetstream (WebSocket, filtered by Exosphere Lexicons)
  → Event received (e.g. a reaction record created on a user's PDS)
    → Is this record referencing a known Sphere? → No → discard
    → Is this Sphere open or is this user a member? → No → discard
    → Index the record into SQLite
    → Available via Sphere API
```

For Private Spheres and unauthenticated data, there is no Jetstream consumption. Data is written directly to the Sphere database through the API:

```
User → Sphere API (auth + membership check) → SQLite
```

### Data flow summary

| Sphere type | Data written to | Data read from         | Jetstream needed |
| ----------- | --------------- | ---------------------- | ---------------- |
| Private     | Sphere SQLite   | Sphere SQLite          | No               |
| Public      | User's PDS      | AppView index (SQLite) | Yes              |

## Membership Model

Membership uses a **bilateral model** inspired by AT Protocol follows. Both the Sphere admin and the member publish records on their respective PDS, creating a verifiable two-sided proof of membership. The local SQLite database indexes these records for fast access control checks.

### Bilateral membership records

Membership is represented by two complementary AT Protocol records:

1. **`site.exosphere.sphereMemberApproval`** — published on the **owner/admin's PDS** when they invite or approve a member. Contains the Sphere AT URI, the member's DID, and their role.
2. **`site.exosphere.sphereMember`** — published on the **member's PDS** when they accept the invitation. Contains the Sphere AT URI.

Both records must exist for a membership to be considered fully established. This mirrors how AT Protocol handles social relationships (e.g. follows) and enables third-party indexers to reconstruct the full membership graph from PDS data alone.

### Local index (SQLite)

The `sphere_members` table caches membership state for fast access control:

- **DID**: The user's AT Protocol decentralized identifier
- **Role**: Owner, admin, member (extensible per Sphere needs)
- **Status**: Active, invited, revoked
- **Invited by**: DID of the user who sent the invitation
- **Joined at**: Timestamp

This table is the authoritative source for real-time access control decisions (API checks, indexer filtering). It is kept in sync with PDS records but allows the server to make fast membership lookups without querying the network.

### Invitation flow

1. A Sphere admin invites a user by their AT Protocol handle or DID.
2. The handle is resolved to a DID (if needed).
3. The admin publishes a `sphereMemberApproval` record on their PDS.
4. An invitation record is created in the local membership table (status: invited).
5. The invited user accepts via the Sphere UI.
6. On acceptance, the user publishes a `sphereMember` record on their PDS.
7. Local status is set to active and the user can participate according to the Sphere's write access mode.

### Private Spheres and membership privacy

For **private Spheres**, the Sphere record itself (`site.exosphere.sphere`) has `visibility: "private"`. The bilateral membership records are still published on PDS (since AT Protocol repos are public), but the Sphere's content remains off-protocol. A third party could see that a user is a member of a private Sphere, but cannot access the Sphere's content. This is an acceptable trade-off — membership is public, content is private — similar to how a private GitHub repository's collaborator list can be partially visible.

If full membership privacy is required, operators can skip PDS writes for membership and rely solely on the local SQLite table. This sacrifices interoperability for privacy.

### Enforcement points

| Layer                 | Role                                                                                                           |
| --------------------- | -------------------------------------------------------------------------------------------------------------- |
| **AppView / Indexer** | Filters incoming Jetstream events. Only indexes interactions from active members (for members-only Spheres).   |
| **Sphere API**        | Checks membership on write requests. Rejects unauthorized actions before they reach the database.              |
| **Client UI**         | Hides write controls (comment box, reaction buttons) for non-members. UX convenience, not a security boundary. |

Membership checks happen at both the API level (for direct writes) and the indexer level (for PDS records arriving via Jetstream). Both paths must enforce the same rules.