vblt.org
forked from
gazagnaire.org/irmin
Persistent store with Git semantics: lazy reads, delayed writes, content-addressing
1# Irmin 4.0
2
3Content-addressable storage for OCaml.
4
5## Architecture
6
7```
8┌─────────────────────────────────────────┐
9│ Link API (typed, schema-driven) │ ← User-facing, type-safe
10│ - Typed paths with encode/decode │
11│ - get/set with 'a leaf accessors │
12└────────────────┬────────────────────────┘
13 │ compiles to
14┌────────────────▼────────────────────────┐
15│ Tree API (lazy reads, delayed writes) │ ← Staging area
16│ - Nodes loaded on-demand │
17│ - Writes accumulate until flush │
18│ - Like Git's index/staging │
19└────────────────┬────────────────────────┘
20 │ compiles to
21┌────────────────▼────────────────────────┐
22│ KV/Backend (raw content-addressed) │ ← Storage
23│ - read/write by hash │
24│ - refs for mutable pointers │
25│ - Memory, Git, Pack implementations │
26└─────────────────────────────────────────┘
27```
28
29## Design Principles
30
311. **One functor, used once.** The `Make` functor takes a format. Pre-instantiated as `Git` and `Mst`.
322. **One module, one concern.** Hash in `Hash`. Node encoding in `Codec`. Storage in `Backend`.
333. **Explicit is better than implicit.** No magic, no hidden state.
344. **Consistent error handling.** All fallible operations return `result`.
35
36## Features
37
38- **Phantom-typed hashes**: SHA-1 and SHA-256 can't be mixed
39- **Lazy reads**: Nodes loaded on-demand from backend
40- **Delayed writes**: Changes accumulate until flush
41- **Multiple formats**: Git trees and ATProto MST
42- **Subtree operations**: For monorepo workflows (replaces git subtree shelling)
43
44## Usage
45
46```ocaml
47(* Create a memory backend *)
48let backend = Irmin.Backend.Memory.create_sha1 () in
49let store = Irmin.Store.Git.create ~backend in
50
51(* Get an empty tree *)
52let tree = Irmin.Store.Git.tree store () in
53
54(* Add content - writes are delayed *)
55let tree = Irmin.Tree.Git.add tree ["src"; "main.ml"] "let () = ()" in
56
57(* Commit - this flushes the tree to backend *)
58let commit = Irmin.Store.Git.commit store ~tree ~parents:[]
59 ~author:"me" ~message:"init" in
60
61(* Set branch head *)
62Irmin.Store.Git.set_head store ~branch:"main" commit
63```
64
65## Tree Formats
66
67| Module | Hash | Format |
68|--------|------|--------|
69| `Irmin.*.Git` | SHA-1 | Git object format |
70| `Irmin.*.Mst` | SHA-256 | ATProto DAG-CBOR MST |
71
72## Module Structure
73
74```
75Irmin
76├── Hash # Phantom-typed SHA-1/SHA-256
77├── Codec # Format.S signature + Git/Mst implementations
78├── Backend # KV storage (Memory, Git, layered, cached)
79├── Tree # Lazy tree with delayed writes
80├── Commit # Commit operations
81├── Store # High-level API (tree + commits + branches)
82├── Subtree # Monorepo subtree operations
83└── Git_interop # Git repository I/O
84```
85
86## References
87
88- [Git Internals](https://git-scm.com/book/en/v2/Git-Internals-Git-Objects)
89- [AT Protocol Repository Spec](https://atproto.com/specs/repository)
90- [Irmin Architecture](https://irmin.org/tutorial/architecture/)