angrydutchman.peedee.es / plcbundle
forked from atscan.net/plcbundle
A Transparent and Verifiable Way to Sync the AT Protocol's PLC Directory
fork atom
plcbundle / docs / cli.md
at test-validate 333 lines 7.0 kB view raw view rendered
tree.fail update cli guide
b5a5052f
1# CLI Guide 2 3A concise guide to using the `plcbundle` command-line tool. 4 5## Installation 6 7```bash 8go install tangled.org/atscan.net/plcbundle/cmd/plcbundle@latest 9plcbundle version # Verify installation 10``` 11 12## Quick Start 13 14```bash 15mkdir plc_archive && cd plc_archive 16plcbundle fetch -count 1 # Fetch one bundle 17plcbundle info # Check what you have 18``` 19 20--- 21 22## Commands 23 24### `fetch` - Download from PLC Directory 25 26Fetches operations from PLC directory and creates bundles. 27 28```bash 29plcbundle fetch -count 1 # Fetch exactly 1 bundle 30plcbundle fetch -count 10 # Fetch 10 bundles 31plcbundle fetch # Fetch continuously until caught up 32``` 33 34**Important:** Without `-count`, fetch runs indefinitely. Always use `-count N` for controlled fetching. 35 36**Options:** 37- `-count N` - Number of bundles to fetch (0 = all available) 38- `-plc URL` - Custom PLC directory URL (default: `https://plc.directory`) 39 40--- 41 42### `clone` - Download from Remote Server 43 44Downloads pre-made bundles from another plcbundle server (much faster than fetch). 45 46```bash 47plcbundle clone https://plc.example.com 48plcbundle clone https://plc.example.com -workers 16 # Faster with more workers 49``` 50 51**Resumable:** Press Ctrl+C to stop, run again to resume. 52 53**Options:** 54- `-workers N` - Concurrent downloads (default: 4) 55- `-v` - Verbose output 56- `-skip-existing` - Skip existing bundles (default: true) 57 58--- 59 60### `info` - View Archive Status 61 62Shows bundle count, storage size, time ranges, and chain hashes. 63 64```bash 65plcbundle info # General overview 66plcbundle info -bundle 42 # Specific bundle details 67plcbundle info --bundles # List all bundles 68plcbundle info --verify # Info + chain verification 69``` 70 71--- 72 73### `verify` - Check Integrity 74 75Verifies file hashes and chain links. 76 77```bash 78plcbundle verify # Verify entire chain 79plcbundle verify -bundle 42 # Verify one bundle 80plcbundle verify -v # Verbose output 81``` 82 83--- 84 85### `rebuild` - Recreate Index 86 87Scans bundle files and rebuilds `index.json`. 88 89```bash 90plcbundle rebuild # Auto-detect CPU cores 91plcbundle rebuild -workers 8 # Use 8 workers 92``` 93 94**When to use:** 95- Lost/corrupted `index.json` 96- Added bundle files manually 97- Moved files from another location 98 99--- 100 101### `export` - Extract Operations 102 103Exports operations as JSONL to stdout. 104 105```bash 106plcbundle export -count 1000 > ops.jsonl 107plcbundle export -after "2024-01-01T00:00:00Z" -count 5000 > jan.jsonl 108``` 109 110--- 111 112### `backfill` - Stream All Operations 113 114Streams operations from all bundles, fetching missing ones on-demand. 115 116```bash 117plcbundle backfill > all.jsonl 118plcbundle backfill -start 100 -end 200 > range.jsonl 119``` 120 121--- 122 123### `mempool` - Inspect Staging Area 124 125Shows operations waiting to form a bundle (need 10,000 to create bundle). 126 127```bash 128plcbundle mempool # Show status 129plcbundle mempool -export > mem.jsonl # Export mempool ops 130plcbundle mempool -validate # Verify chronological order 131plcbundle mempool -clear # Clear (destructive) 132``` 133 134--- 135 136### `serve` - Run HTTP Server 137 138Starts an HTTP server to share bundles with others. 139 140```bash 141plcbundle serve # Start on :8080 142plcbundle serve -port 9000 -host 0.0.0.0 # Custom port/host 143plcbundle serve -sync -sync-interval 5m # Auto-fetch new bundles 144plcbundle serve -websocket # Enable WebSocket streaming 145``` 146 147**Endpoints:** 148- `GET /` - Info page 149- `GET /index.json` - Bundle index 150- `GET /data/:number` - Download bundle 151- `WS /ws` - WebSocket stream (if enabled) 152 153--- 154 155### `compare` - Compare with Remote 156 157Shows differences between local and remote archives. 158 159```bash 160plcbundle compare https://plc.example.com 161plcbundle compare https://plc.example.com --fetch-missing # Auto-fix 162``` 163 164--- 165 166### `version` - Show Version 167 168```bash 169plcbundle version 170``` 171 172--- 173 174## Important Concepts 175 176### Working Directory 177 178plcbundle operates in your **current directory**. Always `cd` to your archive first: 179 180```bash 181cd /path/to/plc_archive 182plcbundle info 183``` 184 185### Files Created 186 187``` 188plc_archive/ 189├── 000001.jsonl.zst # Bundle files (10k ops each) 190├── 000002.jsonl.zst 191├── index.json # Index (metadata + hashes) 192└── plc_mempool_*.jsonl # Mempool (auto-managed, temporary) 193``` 194 195### Fetch vs Clone 196 197**Use `fetch`** when: 198- No mirror available 199- Want data directly from PLC 200- Building from scratch 201 202**Use `clone`** when: 203- A mirror exists 204- Want faster setup 205- Syncing with known good source 206 207--- 208 209## Common Tasks 210 211**Initial setup from mirror:** 212```bash 213mkdir plc_archive && cd plc_archive 214plcbundle clone https://plc.example.com -workers 16 215plcbundle verify 216``` 217 218**Initial setup from PLC:** 219```bash 220mkdir plc_archive && cd plc_archive 221plcbundle fetch -count 0 # Fetch all (can take hours) 222``` 223 224**Daily sync (cron):** 225```bash 226#!/bin/bash 227cd /path/to/plc_archive 228plcbundle fetch -count 5 # Fetch up to 5 new bundles 229``` 230 231**Share your archive:** 232```bash 233plcbundle serve -host 0.0.0.0 -sync 234``` 235 236**Export recent data:** 237```bash 238plcbundle export -count 10000 > recent.jsonl 239cat recent.jsonl | jq . # Process with jq 240``` 241 242**Fix corrupted index:** 243```bash 244plcbundle rebuild 245plcbundle verify 246``` 247 248--- 249 250## Troubleshooting 251 252**Command not found:** 253```bash 254export PATH=$PATH:$(go env GOPATH)/bin 255``` 256 257**Wrong directory:** 258```bash 259pwd # Check where you are 260cd /path/to/plc_archive 261``` 262 263**Fetch doesn't create bundle:** 264```bash 265plcbundle mempool # Check if waiting for more ops 266# Need 10,000 operations to create a bundle 267``` 268 269**Port already in use:** 270```bash 271plcbundle serve -port 9000 272``` 273 274**Hash verification failed:** 275```bash 276rm 000042.jsonl.zst # Delete corrupted bundle 277plcbundle rebuild # Mark as missing 278plcbundle fetch -count 1 # Re-fetch 279``` 280 281**Out of disk space:** 282```bash 283df -h . # Check space 284# Move to larger disk or delete old bundles 285``` 286 287--- 288 289## Quick Reference 290 291```bash 292# Fetch 293plcbundle fetch -count 1 # One bundle 294plcbundle fetch # All available 295 296# Clone 297plcbundle clone <url> # From mirror 298plcbundle clone <url> -workers 16 # Faster 299 300# Info 301plcbundle info # Overview 302plcbundle info -bundle 42 # Specific bundle 303 304# Verify 305plcbundle verify # Check chain 306plcbundle verify -bundle 42 # Check one 307 308# Rebuild 309plcbundle rebuild # Recreate index 310 311# Export 312plcbundle export -count 1000 > ops.jsonl 313 314# Serve 315plcbundle serve # Share bundles 316plcbundle serve -sync -websocket # Full-featured 317 318# Utilities 319plcbundle mempool # Check staging 320plcbundle compare <url> # Compare with remote 321plcbundle backfill > all.jsonl # Export all 322``` 323 324--- 325 326## Getting Help 327 328```bash 329plcbundle <command> -h # Command-specific help 330``` 331 332**Report issues:** https://tangled.org/@atscan.net/plcbundle/issues 333