atproto / README.md

# AT Protocol OCaml Libraries

A comprehensive implementation of the [AT Protocol](https://atproto.com/) (Authenticated Transfer Protocol) in OCaml. This library suite enables developers to build decentralized social networking applications using OCaml's strong type system and functional programming paradigm.

## Features

- **Complete AT Protocol support** - Syntax validation, cryptography, repositories, identity resolution, and more
- **Runtime-agnostic I/O** - Uses OCaml 5.4 algebraic effects for pluggable async runtimes (eio, lwt, etc.)
- **Fully tested** - Passes all 272 tests from the official [atproto-interop-tests](https://github.com/bluesky-social/atproto-interop-tests)
- **Modular design** - 11 independent packages, use only what you need
- **Pure OCaml** - No external processes or services required for core functionality

## Installation

### From opam (when published)

```bash
opam install atproto
```

### From source

```bash
git clone https://github.com/gdiazlo/atproto.git
cd atproto
opam install . --deps-only
dune build
```

## Package Overview

The library is organized into layered packages following the AT Protocol architecture:

### Foundation Layer

| Package | Description |
|---------|-------------|
| `atproto-multibase` | Base encoding (base32-sortable, base58btc) |
| `atproto-syntax` | Identifier parsing: handles, DIDs, NSIDs, TIDs, AT-URIs |
| `atproto-crypto` | P-256/K-256 elliptic curves, did:key, JWT signing |

### Data Layer

| Package | Description |
|---------|-------------|
| `atproto-ipld` | DAG-CBOR encoding, CIDs, CAR files, blobs |
| `atproto-mst` | Merkle Search Tree for content-addressed storage |
| `atproto-repo` | Repository operations, commits, record management |

### Identity Layer

| Package | Description |
|---------|-------------|
| `atproto-identity` | DID resolution (did:plc, did:web), handle resolution |

### Network Layer

| Package | Description |
|---------|-------------|
| `atproto-effects` | I/O effect types for runtime abstraction |
| `atproto-xrpc` | XRPC HTTP API client and server |
| `atproto-sync` | Firehose event streams, repository sync |

### Application Layer

| Package | Description |
|---------|-------------|
| `atproto-lexicon` | Lexicon schema parsing and validation |
| `atproto-api` | High-level client API, rich text facets |

## Quick Start

### Parsing AT Protocol Identifiers

```ocaml
open Atproto_syntax

(* Parse a handle *)
let handle = Handle.of_string "alice.bsky.social" |> Result.get_ok
let () = assert (Handle.to_string handle = "alice.bsky.social")

(* Parse a DID *)
let did = Did.of_string "did:plc:z72i7hdynmk6r22z27h6tvur" |> Result.get_ok
let () = assert (Did.method_ did = "plc")

(* Parse an AT-URI *)
let uri = At_uri.of_string "at://did:plc:xyz/app.bsky.feed.post/abc123" |> Result.get_ok
let () = assert (At_uri.collection uri = Some "app.bsky.feed.post")

(* Generate a TID (timestamp-based identifier) *)
let tid = Tid.make ()
let () = Printf.printf "New TID: %s\n" (Tid.to_string tid)
```

### Working with Cryptography

```ocaml
open Atproto_crypto

(* Generate a P-256 key pair *)
let keypair = P256.generate ()

(* Sign data *)
let data = Bytes.of_string "Hello, AT Protocol!"
let signature = P256.sign keypair.private_key data

(* Verify signature *)
let valid = P256.verify keypair.public_key data signature
let () = assert valid

(* Encode public key as did:key *)
let did_key = Did_key.encode_p256 keypair.public_key
let () = Printf.printf "DID: %s\n" did_key
```

### DAG-CBOR and CIDs

```ocaml
open Atproto_ipld

(* Create a record *)
let record = Dag_cbor.Map [
  ("$type", Dag_cbor.String "app.bsky.feed.post");
  ("text", Dag_cbor.String "Hello from OCaml!");
  ("createdAt", Dag_cbor.String "2024-01-01T00:00:00.000Z");
]

(* Encode to DAG-CBOR *)
let bytes = Dag_cbor.encode record

(* Compute CID *)
let cid = Cid.of_dag_cbor bytes
let () = Printf.printf "CID: %s\n" (Cid.to_string cid)
```

### Firehose Event Stream

```ocaml
open Atproto_sync
open Atproto_effects

(* Define an effect handler for real I/O *)
let run_with_eio f =
  (* Implementation depends on your async runtime *)
  ...

(* Subscribe to the firehose *)
let () = run_with_eio (fun () ->
  let config = Firehose.{
    endpoint = "wss://bsky.network";
    cursor = None;
  } in
  Firehose.subscribe config (fun event ->
    match event with
    | Firehose.Commit { repo; ops; _ } ->
      Printf.printf "Commit from %s with %d ops\n" repo (List.length ops)
    | Firehose.Handle { did; handle } ->
      Printf.printf "Handle update: %s -> %s\n" did handle
    | _ -> ()
  )
)
```

### Identity Resolution

```ocaml
open Atproto_identity
open Atproto_effects

(* Resolve a DID to get the DID document *)
let () = run_with_effects (fun () ->
  match Did_resolver.resolve "did:plc:z72i7hdynmk6r22z27h6tvur" with
  | Ok doc ->
    Printf.printf "Handle: %s\n" (Option.value ~default:"none" doc.also_known_as)
  | Error e ->
    Printf.printf "Resolution failed: %s\n" e
)

(* Resolve a handle to get the DID *)
let () = run_with_effects (fun () ->
  match Handle_resolver.resolve "bsky.app" with
  | Ok did -> Printf.printf "DID: %s\n" did
  | Error e -> Printf.printf "Resolution failed: %s\n" e
)
```

## Effects System

The libraries use OCaml 5.4 algebraic effects for I/O operations, making them runtime-agnostic. You provide effect handlers for:

- `Http_request` / `Http_get` - HTTP requests
- `Dns_txt` / `Dns_a` - DNS lookups
- `Ws_connect` / `Ws_recv` / `Ws_close` - WebSocket operations
- `Now` / `Sleep` - Time operations
- `Random_bytes` - Cryptographic randomness

Example handler with eio:

```ocaml
open Atproto_effects.Effects

let run_with_eio ~env f =
  let open Effect.Deep in
  try_with f () {
    effc = fun (type a) (eff : a Effect.t) ->
      match eff with
      | Http_get { url } -> Some (fun (k : (a, _) continuation) ->
          let response = Eio_client.get ~env url in
          continue k (Ok response))
      | Now -> Some (fun k ->
          continue k (Ptime_clock.now ()))
      | _ -> None
  }
```

## Running Tests

```bash
# Run all tests
dune runtest

# Run tests for a specific package
dune runtest test/syntax

# Run with verbose output
dune runtest --force --verbose
```

## Examples

See the `examples/` directory for complete examples:

- **firehose_demo** - Connect to the Bluesky firehose and print events

```bash
dune exec examples/firehose_demo/firehose_demo.exe
```

## Requirements

- OCaml >= 5.1 (5.4 recommended for full effects support)
- dune >= 3.20

### Dependencies

- `mirage-crypto-ec` - Elliptic curve cryptography
- `digestif` - SHA-256 hashing
- `zarith` - Big integers for low-S normalization
- `cbor` - CBOR encoding (wrapped for DAG-CBOR)
- `yojson` - JSON parsing
- `uri` - URI handling
- `ptime` - Time handling

## Architecture

```
                    +------------------+
                    |   atproto-api    |  High-level client
                    +--------+---------+
                             |
        +--------------------+--------------------+
        |                    |                    |
+-------v-------+    +-------v-------+    +-------v-------+
| atproto-xrpc  |    |atproto-identity|   | atproto-sync |
+-------+-------+    +-------+-------+    +-------+-------+
        |                    |                    |
        +--------------------+--------------------+
                             |
                    +--------v---------+
                    | atproto-effects  |  I/O abstraction
                    +------------------+
                             |
        +--------------------+--------------------+
        |                    |                    |
+-------v-------+    +-------v-------+    +-------v-------+
| atproto-repo  |    |atproto-lexicon|    |               |
+-------+-------+    +---------------+    |               |
        |                                 |               |
+-------v-------+                         |               |
|  atproto-mst  |                         |               |
+-------+-------+                         |               |
        |                                 |               |
+-------v-------+                         |               |
| atproto-ipld  |                         |               |
+-------+-------+                         |               |
        |            +---------------+    |               |
        +----------->| atproto-crypto|<---+               |
        |            +-------+-------+                    |
        |                    |                            |
+-------v-------+    +-------v-------+                    |
|atproto-syntax |<---+atproto-multibase|<-----------------+
+---------------+    +---------------+
```

## License

ISC License - see [LICENSE](LICENSE) file.

## Contributing

Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.

## Resources

- [AT Protocol Specification](https://atproto.com/specs)
- [Bluesky Documentation](https://docs.bsky.app/)
- [AT Protocol Interop Tests](https://github.com/bluesky-social/atproto-interop-tests)
- [Reference Implementation (TypeScript)](https://github.com/bluesky-social/atproto)

## Status

This implementation is feature-complete for core AT Protocol functionality. It passes all official interoperability tests. However, it has not yet been extensively tested against production AT Protocol infrastructure.