burrito.space
testing local-first datastores
burrito.tngl.sh
1# localstress
2
3Benchmark suite for comparing local datastore performance in Node.js.
4
5## Quick Start
6
7```bash
8npm install
9npm run generate # Generate ~1GB test data (takes a few minutes)
10npm run bench # Run benchmarks on all stores
11```
12
13## Datastores Tested
14
15### Node.js Datastores
16- **TinyBase** - Reactive data store for local-first apps
17- **LevelGraph** - Graph database built on LevelDB
18- **SQLite** - Embedded SQL database (via better-sqlite3)
19
20### Browser Datastores
21- **PGLite** - PostgreSQL compiled to WebAssembly
22- **PouchDB** - Sync-enabled NoSQL database for browsers
23- **IndexedDB** - Browser's native structured storage (key-value + indexes)
24- **LocalStorage** - Browser's simple key-value storage (limited to ~5-10MB)
25
26## Methodology
27
28Each benchmark is run **10 times** per datastore. The highest and lowest scores are discarded, and the **median** of the remaining 8 runs is reported. This reduces the impact of outliers from system variability.
29
30## Test Dataset
31
32Target ~1GB total:
33
34- 10,000 URLs with metadata
35- 1,000 PNG images (~900MB)
36- 1,000 text documents (~50MB)
37- 100,000 metadata rows
38
39## Benchmarks
40
41**Init**
42- Store initialization time
43
44**Writes**
45- Add all URLs
46- Add all metadata rows
47- Add all images
48- Add all documents
49
50**Reads**
51- Get 100 most recent URLs
52- Get 10 random images
53- Get 1000 random documents
54
55**Disk**
56- Total storage space used
57
58## Commands
59
60```bash
61# Node.js benchmarks
62npm run generate # Generate test data to test-data/
63npm run bench # Benchmark all Node.js stores (10 iterations each)
64npm run bench:store tinybase # Benchmark single store
65npm run bench:store levelgraph
66npm run bench:store sqlite
67
68# Browser benchmarks
69npm run bench:browser # Benchmark browser stores (PGLite, PouchDB) in multiple browsers
70
71# Results
72npm run charts # Regenerate charts from latest results
73```
74
75## Browser Benchmarking
76
77Browser benchmarks test in-browser datastores across multiple browsers (Chromium, Firefox, WebKit):
78
79```bash
80npm run bench:browser
81```
82
83This launches browser instances (Chromium, Firefox, WebKit), runs benchmarks for each datastore, and measures storage usage via the Storage Estimation API.
84
85**Benchmarks:**
86- IndexedDB - Transactional object store with indexes ✓
87- LocalStorage - Key-value storage (limited capacity) ✓
88- PouchDB - Document database (interactive UI only)
89- PGLite - PostgreSQL in WASM (interactive UI only)
90
91### Interactive Browser Testing
92
93The HTML harness (`src/browser/harness.html`) provides an interactive UI for manual testing:
94
95```bash
96# In one terminal, start a local server:
97python3 -m http.server 8000 --directory src/browser
98
99# Open in browser:
100# http://localhost:8000/harness.html
101```
102
103Then select a datastore and click "Start Benchmark".
104
105## Adding a New Datastore
106
107### Node.js Datastores
1081. Create `src/stores/yourstore.ts` implementing `DatastoreAdapter` from `src/harness/types.ts`
1092. Register in `src/stores/index.ts`
110
111### Browser Datastores
1121. Create `src/browser/adapters/yourstore.ts` implementing `BrowserDatastoreAdapter` from `src/harness/browser-types.ts`
1132. Add to the `stores` array in `src/browser/runner.ts`
114
115## Project Structure
116
117```
118src/
119 generator/ # Test data generation
120 harness/ # Benchmark infrastructure (types, runner, reporter)
121 stores/ # Node.js datastore adapters
122 browser/ # Browser benchmark infrastructure
123 adapters/ # In-browser datastore adapters (PGLite, PouchDB)
124 harness.html # Browser-based benchmark UI
125 harness.ts # Browser benchmark runner
126 runner.ts # Playwright automation for browser testing
127 runner.ts # CLI entry point for Node.js benchmarks
128test-data/ # Generated test data (gitignored)
129results/ # Benchmark results JSON (gitignored)
130```