burrito.space / localstress
testing local-first datastores
fork atom
localstress / README.md
at main 3.8 kB view raw view code

localstress#

Benchmark suite for comparing local datastore performance in Node.js.

Quick Start#

npm install
npm run generate    # Generate ~1GB test data (takes a few minutes)
npm run bench       # Run benchmarks on all stores

Datastores Tested#

Node.js Datastores#

  • TinyBase - Reactive data store for local-first apps
  • LevelGraph - Graph database built on LevelDB
  • SQLite - Embedded SQL database (via better-sqlite3)

Browser Datastores#

  • PGLite - PostgreSQL compiled to WebAssembly
  • PouchDB - Sync-enabled NoSQL database for browsers
  • IndexedDB - Browser's native structured storage (key-value + indexes)
  • LocalStorage - Browser's simple key-value storage (limited to ~5-10MB)

Methodology#

Each benchmark is run 10 times per datastore. The highest and lowest scores are discarded, and the median of the remaining 8 runs is reported. This reduces the impact of outliers from system variability.

Test Dataset#

Target ~1GB total:

  • 10,000 URLs with metadata
  • 1,000 PNG images (~900MB)
  • 1,000 text documents (~50MB)
  • 100,000 metadata rows

Benchmarks#

Init

  • Store initialization time

Writes

  • Add all URLs
  • Add all metadata rows
  • Add all images
  • Add all documents

Reads

  • Get 100 most recent URLs
  • Get 10 random images
  • Get 1000 random documents

Disk

  • Total storage space used

Commands#

# Node.js benchmarks
npm run generate              # Generate test data to test-data/
npm run bench                 # Benchmark all Node.js stores (10 iterations each)
npm run bench:store tinybase  # Benchmark single store
npm run bench:store levelgraph
npm run bench:store sqlite

# Browser benchmarks
npm run bench:browser         # Benchmark browser stores (PGLite, PouchDB) in multiple browsers

# Results
npm run charts                # Regenerate charts from latest results

Browser Benchmarking#

Browser benchmarks test in-browser datastores across multiple browsers (Chromium, Firefox, WebKit):

npm run bench:browser

This launches browser instances (Chromium, Firefox, WebKit), runs benchmarks for each datastore, and measures storage usage via the Storage Estimation API.

Benchmarks:

  • IndexedDB - Transactional object store with indexes ✓
  • LocalStorage - Key-value storage (limited capacity) ✓
  • PouchDB - Document database (interactive UI only)
  • PGLite - PostgreSQL in WASM (interactive UI only)

Interactive Browser Testing#

The HTML harness (src/browser/harness.html) provides an interactive UI for manual testing:

# In one terminal, start a local server:
python3 -m http.server 8000 --directory src/browser

# Open in browser:
# http://localhost:8000/harness.html

Then select a datastore and click "Start Benchmark".

Adding a New Datastore#

Node.js Datastores#

  1. Create src/stores/yourstore.ts implementing DatastoreAdapter from src/harness/types.ts
  2. Register in src/stores/index.ts

Browser Datastores#

  1. Create src/browser/adapters/yourstore.ts implementing BrowserDatastoreAdapter from src/harness/browser-types.ts
  2. Add to the stores array in src/browser/runner.ts

Project Structure#

src/
  generator/        # Test data generation
  harness/          # Benchmark infrastructure (types, runner, reporter)
  stores/           # Node.js datastore adapters
  browser/          # Browser benchmark infrastructure
    adapters/       # In-browser datastore adapters (PGLite, PouchDB)
    harness.html    # Browser-based benchmark UI
    harness.ts      # Browser benchmark runner
    runner.ts       # Playwright automation for browser testing
  runner.ts         # CLI entry point for Node.js benchmarks
test-data/          # Generated test data (gitignored)
results/            # Benchmark results JSON (gitignored)