accidental.cc / feedline
offline-first, p2p synced, atproto enabled, feed reader
fork atom
feedline / CLAUDE.md
at main 12 kB view raw view code

CLAUDE.md#

This file provides guidance to Claude Code when working with code in this repository.

Project Overview#

Feedline (feedline.at) is a local-first, peer-to-peer RSS reader and podcast client with ATProto integration. Think Google Reader meets BitTorrent - your data lives on your devices, syncs peer-to-peer via the Realm protocol, with optional ATProto for discovery and public sharing.

Key Technologies: TypeScript, Solid.js, Vite, Dexie.js (IndexedDB), better-sqlite3, Express, WebRTC (simple-peer), WebSockets, Zod

Current Status: Early development - porting realm protocol from prototype (skypod-node) and establishing architecture patterns. Realm sync works; feed/podcast features are in progress.

Development Commands#

Starting Development#

npm run dev

Starts all development services concurrently using wireit:

  • Vite dev server (frontend)
  • Backend server (WebSocket + API)
  • Type-checking in watch mode

Running Tests#

npm run test              # Run all tests once
npm run start:tests       # Run tests in watch mode

Tests use Jest with ts-jest. Test files are named *.spec.ts or *.spec.tsx.

To run a single test file:

npx jest src/path/to/file.spec.ts

Linting and Type-Checking#

npm run lint              # Run ESLint
npm run types             # Run TypeScript type-checking
npm run build             # Build production frontend

Production#

npm run start:prod        # Build and run production server

Version Control#

This project uses Jujutsu (jj) for version control, not Git. Common operations:

jj status                 # Check working copy status
jj diff                   # View changes
jj commit -m "message"    # Create commit
jj log                    # View history

Architecture#

High-Level Vision#

Feedline is built on three layers:

  1. Lib Layer (#lib/*) - Pure utilities (async primitives, crypto, schema helpers)
  2. Realm Layer (#realm/*) - Generic P2P action sync protocol (frontend-agnostic)
  3. Feedline Layer (#feedline/*) - RSS/podcast application logic

The Realm protocol enables devices to sync "actions" (state changes) in a causally-ordered, offline-first manner. Feedline builds podcast/RSS features on top by defining domain-specific actions like feed:add, feed:patch, entry:analyzed.

Module Structure#

The codebase uses path aliases configured in both package.json (imports) and tsconfig.json (paths):

  • #lib/* (src/lib/) - Reusable utilities that could be extracted to separate packages

    • async/ - Async primitives (semaphore, blocking queue, aborts, sleep)
    • crypto/ - JWT, JWK, cipher utilities
    • schema/ - Zod schema helpers (brands, JSON validation)
    • errors.ts, types.ts, utils.ts - General utilities

  • #realm/* (src/realm/) - P2P action sync protocol (frontend-agnostic)

    • logical-clock.ts - Hybrid Logical Clock (HLC) for causal ordering
    • protocol/ - Message schemas and protocol definitions
    • schema/ - Realm-specific schemas (actions, identities, timestamps)
    • client/ - Client-side realm connection management
    • server/ - Server-side signaling and relay (server acts as peer)

  • #feedline/* (src/feedline/) - Application layer (RSS/podcast domain)

    • client/ - Frontend application
    • server/ - Backend services
    • worker/ - Background job processing (planned)
    • scheduler/ - Feed refresh scheduling (planned)
    • cmd/ - CLI tools

  • #spec/* (src/spec/) - Test helpers and integration test utilities

Always use these path aliases for imports across module boundaries. Never use relative paths like ../../../realm/.

Data Storage#

Client (Browser):

  • IndexedDB via Dexie.js
    • Realm actions database (managed by #realm/*)
    • Feedline materialized views (managed by #feedline/*, separate DB)
    • Private keys storage (IndexedDB provides secure, per-origin isolation)

Server (Node.js):

  • better-sqlite3
    • Realm databases (one per realm, stores actions)
    • Job queue database (planned)
    • Note: Would prefer node:sqlite, but Nix doesn't have good support yet

These are completely separate - realm maintains the action log, feedline materializes a queryable schema from those actions.

Realm Protocol (The Complex Part)#

The realm protocol is the most intricate piece of the codebase. It uses a Hybrid Logical Clock (HLC) for causal ordering of events across distributed peers.

Key Concepts:

  1. Actions - Immutable state changes (like Redux actions, but distributed)
  2. HLC Timestamps - Format: lc:seconds:counter:identid - provides total ordering even with clock skew
  3. PULL for Catch-up - New clients pull complete history from one peer
  4. PUSH for Updates - All clients push their new actions to all connected peers (with deduplication via knowledge vectors)

When working on realm code:

  • Ordering is critical - never break HLC timestamp comparison logic
  • Actions are append-only, never mutate
  • Server is just another peer, no special privileges
  • WebSocket is for signaling; WebRTC data channels do the heavy lifting

Important Files:

  • src/realm/logical-clock.ts - HLC implementation
  • src/realm/protocol/ - Message schemas (Zod)
  • src/realm/client/realm.ts - Client-side connection management
  • src/realm/server/driver-realm.ts - Server-side realm driver

Tests exist and should be reviewed - the server peer implementation works but needs code review for correctness.

Action-Driven Architecture#

Everything in Feedline flows through actions:

// Example: User adds a feed
{
  type: 'feed:add',
  payload: {
    url: 'https://example.com/feed.rss',
    private: false
  },
  clock: '0:1735689234:0:alice123',
  actor: 'alice123'
}

Actions sync across all devices via realm. Each device materializes its own queryable database from the action log.

Server as Peer: The server doesn't have special APIs - it just creates actions like any other peer (e.g., feed:patch with new episodes after refresh).

Design Decisions & Open Questions#

Decided#

  • Realm protocol is frontend-agnostic - No Solid/React dependencies in #realm/*
  • Action-driven architecture - All state changes flow through realm actions
  • Server as peer - No special server APIs, server creates actions like any client
  • Offline-first - Clients work independently, sync when connected
  • IndexedDB for client - Needed for secure private key storage
  • Path aliases - Always use #lib/*, #realm/*, #feedline/*
  • Wireit for task orchestration - Handles build dependencies and watch mode
  • Jujutsu for VCS - Using jj instead of git
  • Solid.js for UI - Migrated from Preact to Solid for better performance and modern reactive primitives

Still Figuring Out#

  • Job queue architecture - Designed (Piscina workers, shared job DB) but not implemented yet
  • WebTorrent integration - Planned for P2P audio distribution, not started
  • ATProto integration - Will be used for discovery and optional public sharing, design in progress
  • TypeDoc - Experimenting with API docs generation, unclear if worth it
  • Server/client split in #feedline/* - May organize feedline layer into server/client subdirs, not decided

Code Style & Patterns#

  • Strict TypeScript - strict: true, noImplicitAny: true
  • Zod for validation - All schemas defined with Zod, used for both types and runtime validation
  • Branded types - Use Zod brands for semantic types (IdentID, RealmID, etc.)
  • ESLint + Prettier - Auto-formatting and linting enforced
  • Solid Signals - Using Solid's built-in reactive primitives (createSignal, createEffect, etc.)

Working with Claude#

What Claude should know:

Mainly: You're a Great Rubber Duck - Claude should ask questions, suggest patterns, and help think through design decisions; rarely should we jump to implementation of anything, and Claude should assume the user would prefer to do implementation themselves.

  1. This is early-stage - Architecture is being established, patterns are still evolving
  2. Realm protocol is sacred - Don't break HLC ordering, action immutability, or sync logic
  3. Layer boundaries matter - Keep #lib/* pure, #realm/* frontend-agnostic, #feedline/* application-specific
  4. Tests are concrete - Written after structure is stable to "lock in" decisions

When suggesting code:

  • Respect the layer boundaries
  • Use path aliases
  • Follow the action-driven pattern
  • Ask if you're unsure whether something belongs in lib/realm/feedline
  • Point out potential issues with distributed sync or causal ordering

Current focus areas:

  • Porting realm protocol from prototype (needs review)
  • Establishing feedline action schemas
  • Designing job queue for background work (not implemented yet)
  • Building out Solid.js UI components

Testing Strategy#

Tests are written after the structure is stable - think of them as "pouring concrete" to lock in architectural decisions.

Test organization:

  • Unit tests alongside source: foo.ts → foo.spec.ts
  • Integration tests in #spec/*
  • Use @jest/globals for Jest functions
  • Path aliases work in tests (configured in jest.config.js)

What needs testing:

  • Realm protocol correctness (HLC ordering, sync protocol)
  • Action materialization (replaying action logs correctly)
  • Server peer behavior (relay, storage, deduplication)

Roadmap#

Based on tmp-analizer/FEEDLINE_ARCHITECTURE.md, the planned implementation phases are:

Phase 1: Foundation (In Progress)#

  • ✅ Basic realm protocol ported (needs review)
  • 🔄 Realm client/server architecture
  • ⏳ Action sync verification
  • ⏳ Feed schemas

Phase 2: Feed Tracking (Not Started)#

  • feed:track / feed:untrack actions
  • Server-side scheduler
  • RSS fetcher
  • feed:patch action creator

Phase 3: Analysis Integration (Planned)#

  • Analysis worker service (Piscina-based job queue)
  • entry:analyzed action
  • Static file server
  • Audio processing

Phase 4: WebTorrent P2P (Planned)#

  • RealmTorrentManager
  • Separate WebRTC connections for torrents
  • Stream source switching

Phase 5: ATProto Integration (Planned)#

  • Publisher service (opt-in)
  • Magnet discovery
  • Jetstream subscriber for aggregation

Phase 6: Self-Hosted Stack (Planned)#

  • Container images
  • macOS menu bar app
  • Linux Flatpak

Important Files#

Realm Protocol:

  • src/realm/logical-clock.ts - HLC implementation
  • src/realm/schema/timestamp.ts - Timestamp schemas
  • src/realm/client/realm.ts - Client realm connection
  • src/realm/server/driver-realm.ts - Server realm driver

Utilities:

  • src/lib/async/ - Async primitives (critical for coordination)
  • src/lib/crypto/ - JWT/JWK for authentication
  • src/lib/schema/ - Zod helpers

Tests:

  • src/realm/logical-clock.spec.ts - HLC tests (important!)
  • src/realm/server/ - Server peer tests (need review)

ATProto Integration Notes#

Feedline will use ATProto (feedline.at domain) for:

  • Discovery - Find popular feeds, episode magnets
  • Sharing - Optionally publish your subscriptions/magnets
  • Privacy-first - Everything private by default, ATProto is opt-in

ATProto is not used for:

  • Primary data storage (all local-first)
  • Sync protocol (realm handles that)
  • Authentication between your devices (realm uses JWTs)

Think of ATProto as a public discovery layer on top of a fully private P2P foundation.