vblt.org
forked from
gazagnaire.org/irmin
Persistent store with Git semantics: lazy reads, delayed writes, content-addressing
Add optimization comparison chart and restore per-optim data
Recover benchmark data from previous commits (baseline, +inline, +cache,
+inode, +all) into irmini_optims.json. Add "optims" chart category to
gen_chart_all.py. Update README with optimization breakdown showing
inodes as the biggest single optimization (220x commits, 21x reads).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
+176
-8
+49
-3
bench/README.md
+49
-3
bench/README.md
···
82
82
83
83
### Disk backends (fs, pack, lavyek)
84
84
85
-
85
+
86
86
87
87
```
88
88
Name Scenario ops/s total(s) RSS(MiB)
···
125
125
126
126
### Memory backends
127
127
128
-
128
+
129
129
130
130
```
131
131
Name Scenario ops/s total(s) RSS(MiB)
···
154
154
155
155
### Git backends
156
156
157
-
157
+
158
158
159
159
```
160
160
Name Scenario ops/s total(s) RSS(MiB)
···
180
180
Git object graph in memory; irmini reads directly from the Git store.
181
181
- **Incremental**: All three are comparable (~160–178 ops/s) — dominated by
182
182
Git I/O overhead.
183
+
184
+
### Irmini optimizations (memory)
185
+
186
+
187
+
188
+
Impact of each optimization measured independently on the memory backend
189
+
(50 commits × 500 adds, depth 10, 5000 reads, 100-byte values):
190
+
191
+
```
192
+
Name Scenario ops/s vs baseline
193
+
----------------------------------------------------------------------------------
194
+
Irmini baseline commits 519 1.0×
195
+
Irmini+inline commits 126963 244.6×
196
+
Irmini+cache commits 512 1.0×
197
+
Irmini+inode commits 114429 220.5×
198
+
Irmini+all commits 82255 158.5×
199
+
200
+
Irmini baseline reads 9564 1.0×
201
+
Irmini+inline reads 19501 2.0×
202
+
Irmini+cache reads 13399 1.4×
203
+
Irmini+inode reads 205271 21.5×
204
+
Irmini+all reads 1481040 154.9×
205
+
206
+
Irmini baseline incremental 1965 1.0×
207
+
Irmini+inline incremental 2120 1.1×
208
+
Irmini+cache incremental 2270 1.2×
209
+
Irmini+inode incremental 9439 4.8×
210
+
Irmini+all incremental 12228 6.2×
211
+
212
+
Irmini baseline large-values 1527 1.0×
213
+
Irmini+inline large-values 1569 1.0×
214
+
Irmini+cache large-values 1470 1.0×
215
+
Irmini+inode large-values 18381 12.0×
216
+
Irmini+all large-values 18007 11.8×
217
+
```
218
+
219
+
- **Inodes** are the biggest single optimization: **220× commits**, **21× reads**,
220
+
**4.8× incremental**, **12× large-values**. They avoid re-serializing the
221
+
entire tree on each commit (32-way HAMT trie, O(log n) updates).
222
+
- **Inlining** gives a massive **245× boost on commits** (small values stored
223
+
directly in tree nodes, avoiding content-addressable store lookups), but
224
+
only helps reads modestly (2×).
225
+
- **LRU cache** improves reads by **1.4×** (avoids repeated deserialization)
226
+
but has no effect on writes.
227
+
- **All combined** gives the best reads (**1.48M ops/s**, 155×) thanks to the
228
+
synergy of inodes + resolved-child cache + LRU cache.
183
229
184
230
### Key observations
185
231
+21
-5
bench/gen_chart_all.py
+21
-5
bench/gen_chart_all.py
···
35
35
36
36
37
37
def classify_backend(name):
38
-
"""Classify a result name into memory/disk/git category."""
38
+
"""Classify a result name into memory/disk/git/optims category."""
39
39
n = name.lower()
40
-
if "memory" in n or "mem" in n:
40
+
# Optimization variants go to their own chart
41
+
if "baseline" in n or "+inline" in n or "+cache" in n or "+inode" in n or "+all" in n:
42
+
return "optims"
43
+
elif "memory" in n or "mem" in n:
41
44
return "memory"
42
45
elif "git" in n:
43
46
return "git"
···
60
63
"Irmini (disk)": "#6d9dc5",
61
64
"Irmini (lavyek)": "#59a14f",
62
65
"Irmini (git)": "#8bc584",
66
+
# Optimization variants
67
+
"Irmini baseline": "#bbb",
68
+
"Irmini+inline": "#f28e2b",
69
+
"Irmini+cache": "#76b7b2",
70
+
"Irmini+inode": "#e15759",
71
+
"Irmini+all": "#4e79a7",
63
72
}
64
73
65
74
75
+
OPTIM_ORDER = ["Irmini baseline", "Irmini+inline", "Irmini+cache", "Irmini+inode", "Irmini+all"]
76
+
77
+
66
78
def family_sort_key(name):
67
79
"""Sort backends: Irmin-Lwt first, then Irmin-Eio, then Irmini."""
68
80
n = name.lower()
69
-
if "irmin-lwt" in n:
81
+
if name in OPTIM_ORDER:
82
+
return (0, OPTIM_ORDER.index(name))
83
+
elif "irmin-lwt" in n:
70
84
return (0, name)
71
85
elif "irmin-eio" in n or "irmin-pack" in n or "irmin-fs" in n or "irmin-git" in n:
72
86
return (1, name)
···
242
256
print(f"Total: {len(all_results)} results")
243
257
244
258
# Group by backend type
245
-
groups = {"memory": [], "disk": [], "git": []}
259
+
groups = {"memory": [], "disk": [], "git": [], "optims": []}
246
260
for r in all_results:
247
261
cat = classify_backend(r["name"])
248
262
groups[cat].append(r)
···
259
273
260
274
chart_dir = results_dir
261
275
262
-
# Charts in order: disk, memory, git (as requested by user)
276
+
# Charts in order: disk, memory, git, optims
263
277
chart_specs = [
264
278
("disk", "Disk backends (fs, pack, lavyek) — ops/s comparison",
265
279
f"chart_disk_{timestamp}.svg"),
···
267
281
f"chart_memory_{timestamp}.svg"),
268
282
("git", "Git backends — ops/s comparison",
269
283
f"chart_git_{timestamp}.svg"),
284
+
("optims", "Irmini optimizations (memory) — ops/s comparison",
285
+
f"chart_optims_{timestamp}.svg"),
270
286
]
271
287
272
288
for cat, title, filename in chart_specs:
bench/results/chart_disk_1773074715.svg
bench/results/chart_disk_1773134885.svg
bench/results/chart_disk_1773074715.svg
bench/results/chart_disk_1773134885.svg
bench/results/chart_git_1773074715.svg
bench/results/chart_git_1773134885.svg
bench/results/chart_git_1773074715.svg
bench/results/chart_git_1773134885.svg
bench/results/chart_memory_1773074715.svg
bench/results/chart_memory_1773134885.svg
bench/results/chart_memory_1773074715.svg
bench/results/chart_memory_1773134885.svg
+84
bench/results/chart_optims_1773134885.svg
+84
bench/results/chart_optims_1773134885.svg
···
1
+
<svg xmlns="http://www.w3.org/2000/svg" width="840" height="550" font-family="system-ui, sans-serif" font-size="11">
2
+
<rect width="840" height="550" fill="white"/>
3
+
<text x="420.0" y="28" text-anchor="middle" font-size="16" font-weight="bold">Irmini optimizations (memory) — ops/s comparison</text>
4
+
<g transform="translate(120,60)">
5
+
<text x="-85" y="200.0" text-anchor="middle" font-size="12" fill="#333" transform="rotate(-90,-85,200.0)">ops/s</text>
6
+
<line x1="0" y1="300.0" x2="118" y2="300.0" stroke="#e0e0e0" stroke-width="0.5"/>
7
+
<line x1="0" y1="200.0" x2="118" y2="200.0" stroke="#e0e0e0" stroke-width="0.5"/>
8
+
<line x1="0" y1="100.0" x2="118" y2="100.0" stroke="#e0e0e0" stroke-width="0.5"/>
9
+
<line x1="0" y1="0.0" x2="118" y2="0.0" stroke="#e0e0e0" stroke-width="0.5"/>
10
+
<text x="-4.0" y="4.0" text-anchor="end" font-size="8" fill="#999">146k</text>
11
+
<rect x="0.0" y="398.6" width="22" height="1.4" fill="#bbb" rx="1"/>
12
+
<text x="11.0" y="395.6" text-anchor="middle" font-size="7" fill="#333">519</text>
13
+
<rect x="24.0" y="52.2" width="22" height="347.8" fill="#f28e2b" rx="1"/>
14
+
<text x="35.0" y="49.2" text-anchor="middle" font-size="7" fill="#333">127k</text>
15
+
<rect x="48.0" y="398.6" width="22" height="1.4" fill="#76b7b2" rx="1"/>
16
+
<text x="59.0" y="395.6" text-anchor="middle" font-size="7" fill="#333">512</text>
17
+
<rect x="72.0" y="86.5" width="22" height="313.5" fill="#e15759" rx="1"/>
18
+
<text x="83.0" y="83.5" text-anchor="middle" font-size="7" fill="#333">114k</text>
19
+
<rect x="96.0" y="174.7" width="22" height="225.3" fill="#4e79a7" rx="1"/>
20
+
<text x="107.0" y="171.7" text-anchor="middle" font-size="7" fill="#333">82k</text>
21
+
<text x="59.0" y="416" text-anchor="middle" font-size="11" font-weight="bold" fill="#333">commits</text>
22
+
<line x1="168" y1="300.0" x2="286" y2="300.0" stroke="#e0e0e0" stroke-width="0.5"/>
23
+
<line x1="168" y1="200.0" x2="286" y2="200.0" stroke="#e0e0e0" stroke-width="0.5"/>
24
+
<line x1="168" y1="100.0" x2="286" y2="100.0" stroke="#e0e0e0" stroke-width="0.5"/>
25
+
<line x1="168" y1="0.0" x2="286" y2="0.0" stroke="#e0e0e0" stroke-width="0.5"/>
26
+
<text x="164.0" y="4.0" text-anchor="end" font-size="8" fill="#999">1.7M</text>
27
+
<rect x="168.0" y="397.8" width="22" height="2.2" fill="#bbb" rx="1"/>
28
+
<text x="179.0" y="394.8" text-anchor="middle" font-size="7" fill="#333">10k</text>
29
+
<rect x="192.0" y="395.4" width="22" height="4.6" fill="#f28e2b" rx="1"/>
30
+
<text x="203.0" y="392.4" text-anchor="middle" font-size="7" fill="#333">20k</text>
31
+
<rect x="216.0" y="396.9" width="22" height="3.1" fill="#76b7b2" rx="1"/>
32
+
<text x="227.0" y="393.9" text-anchor="middle" font-size="7" fill="#333">13k</text>
33
+
<rect x="240.0" y="351.8" width="22" height="48.2" fill="#e15759" rx="1"/>
34
+
<text x="251.0" y="348.8" text-anchor="middle" font-size="7" fill="#333">205k</text>
35
+
<rect x="264.0" y="52.2" width="22" height="347.8" fill="#4e79a7" rx="1"/>
36
+
<text x="275.0" y="49.2" text-anchor="middle" font-size="7" fill="#333">1.5M</text>
37
+
<text x="227.0" y="416" text-anchor="middle" font-size="11" font-weight="bold" fill="#333">reads</text>
38
+
<line x1="336" y1="300.0" x2="454" y2="300.0" stroke="#e0e0e0" stroke-width="0.5"/>
39
+
<line x1="336" y1="200.0" x2="454" y2="200.0" stroke="#e0e0e0" stroke-width="0.5"/>
40
+
<line x1="336" y1="100.0" x2="454" y2="100.0" stroke="#e0e0e0" stroke-width="0.5"/>
41
+
<line x1="336" y1="0.0" x2="454" y2="0.0" stroke="#e0e0e0" stroke-width="0.5"/>
42
+
<text x="332.0" y="4.0" text-anchor="end" font-size="8" fill="#999">14k</text>
43
+
<rect x="336.0" y="344.1" width="22" height="55.9" fill="#bbb" rx="1"/>
44
+
<text x="347.0" y="341.1" text-anchor="middle" font-size="7" fill="#333">2k</text>
45
+
<rect x="360.0" y="339.7" width="22" height="60.3" fill="#f28e2b" rx="1"/>
46
+
<text x="371.0" y="336.7" text-anchor="middle" font-size="7" fill="#333">2k</text>
47
+
<rect x="384.0" y="335.4" width="22" height="64.6" fill="#76b7b2" rx="1"/>
48
+
<text x="395.0" y="332.4" text-anchor="middle" font-size="7" fill="#333">2k</text>
49
+
<rect x="408.0" y="131.5" width="22" height="268.5" fill="#e15759" rx="1"/>
50
+
<text x="419.0" y="128.5" text-anchor="middle" font-size="7" fill="#333">9k</text>
51
+
<rect x="432.0" y="52.2" width="22" height="347.8" fill="#4e79a7" rx="1"/>
52
+
<text x="443.0" y="49.2" text-anchor="middle" font-size="7" fill="#333">12k</text>
53
+
<text x="395.0" y="416" text-anchor="middle" font-size="11" font-weight="bold" fill="#333">incremental</text>
54
+
<line x1="504" y1="300.0" x2="622" y2="300.0" stroke="#e0e0e0" stroke-width="0.5"/>
55
+
<line x1="504" y1="200.0" x2="622" y2="200.0" stroke="#e0e0e0" stroke-width="0.5"/>
56
+
<line x1="504" y1="100.0" x2="622" y2="100.0" stroke="#e0e0e0" stroke-width="0.5"/>
57
+
<line x1="504" y1="0.0" x2="622" y2="0.0" stroke="#e0e0e0" stroke-width="0.5"/>
58
+
<text x="500.0" y="4.0" text-anchor="end" font-size="8" fill="#999">21k</text>
59
+
<rect x="504.0" y="371.1" width="22" height="28.9" fill="#bbb" rx="1"/>
60
+
<text x="515.0" y="368.1" text-anchor="middle" font-size="7" fill="#333">2k</text>
61
+
<rect x="528.0" y="370.3" width="22" height="29.7" fill="#f28e2b" rx="1"/>
62
+
<text x="539.0" y="367.3" text-anchor="middle" font-size="7" fill="#333">2k</text>
63
+
<rect x="552.0" y="372.2" width="22" height="27.8" fill="#76b7b2" rx="1"/>
64
+
<text x="563.0" y="369.2" text-anchor="middle" font-size="7" fill="#333">1k</text>
65
+
<rect x="576.0" y="52.2" width="22" height="347.8" fill="#e15759" rx="1"/>
66
+
<text x="587.0" y="49.2" text-anchor="middle" font-size="7" fill="#333">18k</text>
67
+
<rect x="600.0" y="59.3" width="22" height="340.7" fill="#4e79a7" rx="1"/>
68
+
<text x="611.0" y="56.3" text-anchor="middle" font-size="7" fill="#333">18k</text>
69
+
<text x="563.0" y="416" text-anchor="middle" font-size="11" font-weight="bold" fill="#333">large-values</text>
70
+
<line x1="0" y1="400" x2="622" y2="400" stroke="#333" stroke-width="1"/>
71
+
</g>
72
+
<g transform="translate(120,500)">
73
+
<rect x="0" y="0" width="12" height="12" fill="#bbb" rx="2"/>
74
+
<text x="16" y="10" font-size="10" fill="#333">Irmini baseline</text>
75
+
<rect x="180" y="0" width="12" height="12" fill="#f28e2b" rx="2"/>
76
+
<text x="196" y="10" font-size="10" fill="#333">Irmini+inline</text>
77
+
<rect x="360" y="0" width="12" height="12" fill="#76b7b2" rx="2"/>
78
+
<text x="376" y="10" font-size="10" fill="#333">Irmini+cache</text>
79
+
<rect x="540" y="0" width="12" height="12" fill="#e15759" rx="2"/>
80
+
<text x="556" y="10" font-size="10" fill="#333">Irmini+inode</text>
81
+
<rect x="0" y="20" width="12" height="12" fill="#4e79a7" rx="2"/>
82
+
<text x="16" y="30" font-size="10" fill="#333">Irmini+all</text>
83
+
</g>
84
+
</svg>
+22
bench/results/irmini_optims.json
+22
bench/results/irmini_optims.json
···
1
+
[
2
+
{"name": "Irmini baseline", "scenario": "commits", "ops_per_sec": 519},
3
+
{"name": "Irmini baseline", "scenario": "reads", "ops_per_sec": 9564},
4
+
{"name": "Irmini baseline", "scenario": "incremental", "ops_per_sec": 1965},
5
+
{"name": "Irmini baseline", "scenario": "large-values", "ops_per_sec": 1527},
6
+
{"name": "Irmini+inline", "scenario": "commits", "ops_per_sec": 126963},
7
+
{"name": "Irmini+inline", "scenario": "reads", "ops_per_sec": 19501},
8
+
{"name": "Irmini+inline", "scenario": "incremental", "ops_per_sec": 2120},
9
+
{"name": "Irmini+inline", "scenario": "large-values", "ops_per_sec": 1569},
10
+
{"name": "Irmini+cache", "scenario": "commits", "ops_per_sec": 512},
11
+
{"name": "Irmini+cache", "scenario": "reads", "ops_per_sec": 13399},
12
+
{"name": "Irmini+cache", "scenario": "incremental", "ops_per_sec": 2270},
13
+
{"name": "Irmini+cache", "scenario": "large-values", "ops_per_sec": 1470},
14
+
{"name": "Irmini+inode", "scenario": "commits", "ops_per_sec": 114429},
15
+
{"name": "Irmini+inode", "scenario": "reads", "ops_per_sec": 205271},
16
+
{"name": "Irmini+inode", "scenario": "incremental", "ops_per_sec": 9439},
17
+
{"name": "Irmini+inode", "scenario": "large-values", "ops_per_sec": 18381},
18
+
{"name": "Irmini+all", "scenario": "commits", "ops_per_sec": 82255},
19
+
{"name": "Irmini+all", "scenario": "reads", "ops_per_sec": 1481040},
20
+
{"name": "Irmini+all", "scenario": "incremental", "ops_per_sec": 12228},
21
+
{"name": "Irmini+all", "scenario": "large-values", "ops_per_sec": 18007}
22
+
]