P2PDS#

Peer-to-peer replication infrastructure for AT Protocol. Backs up and serves atproto account data over IPFS, acting on behalf of authenticated users.

• Syncs and stores repos and blobs for configured accounts
• Provides data on P2P networks (IPFS/libp2p) for other nodes to replicate
• Fetches and stores data from P2P networks for serviced accounts
• Mutual replication agreements between peers via on-protocol consent records

P2PDS is infrastructure — like a torrent client for atproto data. It does not have its own identity. Users authenticate with their own atproto accounts, and records (org.p2pds.peer, org.p2pds.replication.offer) are published to the user's own repo via their PDS.

Stack#

• Runtime: Node.js, TypeScript (ES2022, strict)
• Base: Generalized from Cirrus
• HTTP: Hono
• Database: better-sqlite3 (sync API)
• IPFS: Helia (libp2p + DHT + bitswap + gossipsub)
• Identity: AT Protocol DIDs via PLC directory
• Desktop: Tauri v2 (optional, apps/desktop/)
• Content addressing: DASL CIDs (CIDv1, SHA-256, dag-cbor/raw, base32lower)

Architecture#

User's atproto account (any PDS: Bluesky, Cirrus, self-hosted)
        │
        ▼
   ┌─────────┐
   │  p2pds   │  ← replication infrastructure (local, cloud, or co-located)
   │          │
   │  SQLite  │  block/blob tracking, sync state, challenge history
   │  Helia   │  IPFS storage, DHT announcements, bitswap, gossipsub
   │  Hono    │  XRPC endpoints, admin dashboard, RASL
   └─────────┘
        │
        ▼
   Other p2pds nodes (mutual replication via offer records)

Configured with a list of DIDs to replicate:

1. Resolves DIDs via PLC directory to find source PDSes
2. Fetches repos as CAR files from each DID's PDS (incremental via since)
3. Stores blocks in IPFS (Helia) and announces via DHT
4. Serves blocks via content-addressed RASL endpoint
5. Real-time sync via firehose (com.atproto.sync.subscribeRepos)
6. Gossipsub notifications for low-latency cross-node sync
7. Verifies block availability on remote peers via challenge-response protocol

Design choices#

• DHT only for discovery/routing — no IPNI or centralized indexers
• Slow data is fine as a tradeoff for resilience and decentralization
• Transport-agnostic verification — RASL works over any HTTP transport
• DASL-compliant content addressing — all CIDs are CIDv1 + SHA-256 with either dag-cbor (0x71) or raw (0x55) codec, encoded as base32lower (b prefix). This is enforced at the library level by @atcute/cid and matches atproto's CID conventions. See DASL CID spec.
• No node identity — p2pds acts on behalf of users, not as its own entity. Records publish to the user's own atproto repo.

Deployment flexibility#

P2PDS works with any combination of:

• PDS: Bluesky, Cirrus, self-hosted, any atproto-compatible PDS
• Location: local desktop (via Tauri app), cloud (Railway, etc.), co-located server

Lexicons#

P2PDS defines two record types for the on-protocol interop surface:

Schemas are in lexicons/ and validated by src/lexicons.ts.

Offer negotiation: Peers publish offer records declaring willingness to replicate specific DIDs. When two peers have mutual offers (A offers to replicate B, B offers to replicate A), a replication agreement is automatically formed. Parameters are merged: max(minCopies), min(intervalSec), max(priority). Revoking an offer = deleting the record.

Verification#

Content-addressed retrieval is unforgeable: if a peer returns the correct bytes for a CID, they have the data. The verification stack exploits this property at multiple layers:

Challenge-response protocol: Three message types (StorageChallenge → StorageChallengeResponse → StorageChallengeResult). Deterministic generation from epoch + DIDs + nonce. Transport-agnostic with libp2p primary and HTTP fallback (FailoverChallengeTransport). Challenge history and peer reliability tracked in SQLite.

L0 and L1 run on a configurable timer (default 30 min). L1 samples are tuneable via VerificationConfig.raslSampleSize (default 50 blocks).

L2 (libp2p+HTTP Gateway) — future#

Reuses RASL verification logic over libp2p transports for NAT traversal and encryption without public IP. Requires libp2p+HTTP Gateway spec in Helia.

• Kubo (Go): ipfs/kubo#10049 (shipped)
• Helia (JS): ipfs/helia#348 (not yet)

Replication#

Sync loop (per DID, periodic with policy-driven intervals):

1. Resolve DID → PDS endpoint (via PLC directory)
2. Fetch repo (com.atproto.sync.getRepo, incremental via since)
3. Parse CAR, store blocks in IPFS, fetch and store blobs
4. Track block/blob CIDs, populate record paths via MST walk
5. Announce to DHT
6. Verify local block availability
7. If source PDS fails, fall back to peer endpoints

Real-time sync: Firehose subscription (com.atproto.sync.subscribeRepos) with cursor persistence, DID filtering, and incremental block application. Gossipsub commit notifications for low-latency cross-node coordination.

GC and tombstones: Deferred GC via needs_gc flag on delete/update ops. Full block/blob reconciliation during sync via MST walk. Cross-DID block sharing safety. Tombstone detection via firehose #account events with 24hr grace period.

Policy Engine#

Declarative, deterministic, transport-agnostic policy system operating on atproto accounts:

• Mutual aid: N-of-M redundancy between cooperating peers
• SaaS: SLA compliance with minimum copy counts and sync intervals
• Group governance: Multi-party replication agreements

Policies drive sync intervals, priority ordering, and shouldReplicate filtering in the replication manager. P2P policies are auto-generated from mutual offer records with p2p: prefixed IDs.

App#

• Dashboard: Server-rendered HTML at / (auto-refresh)
• API: Authenticated XRPC endpoints for overview, per-DID status, network status, policies, sync history
• DID management: Add/remove DIDs at runtime via addDid/removeDid endpoints
• Rate limiting: Per-IP and per-DID limits across HTTP, gossipsub, and libp2p

Desktop App#

Optional Tauri v2 wrapper at apps/desktop/. Spawns p2pds as a sidecar process and loads the dashboard in a webview.

cd apps/desktop
npm run build:sidecar   # compile p2pds to standalone binary via pkg
cargo tauri dev          # run in development
cargo tauri build        # build distributable

Development#

npm install
npm test
npm run dev

Project structure#

src/
  index.ts              Hono app with all routes
  server.ts             HTTP server entry point
  config.ts             Config interface + loadConfig()
  validation.ts         Record validator (atproto + p2pds lexicons)
  lexicons.ts           p2pds lexicon loader + validator
  ipfs.ts               IpfsService (Helia wrapper)
  repo-manager.ts       Local repo management
  storage.ts            SQLite block storage
  blobs.ts              Blob storage
  middleware/auth.ts     Auth middleware
  replication/           Sync, verification, challenges, offers, gossipsub
  policy/                Policy engine types, engine, presets
  xrpc/                  XRPC endpoint handlers
lexicons/                Lexicon JSON schemas
apps/desktop/            Tauri desktop app

Configuration#

Environment variables (or .env file):

Status#

1. Single-user PDS — done
2. Record replication with IPFS storage — done
3. Real-time firehose sync — done
4. Layered verification (L0-L3) — done
5. Challenge-response proof-of-storage — done
6. Policy engine — done
7. P2P offer negotiation — done
8. Admin dashboard + DID management — done
9. Rate limiting — done
10. Architecture refactor (user-DID model) — done
11. Lexicon definitions — done
12. Desktop app skeleton — done