@civility/cloud #
The client for a Civility Cloud server. Change-based synchronization for Civility stores — syncing @civility/store data with conflict resolution, blob syncing, and offline support
Install #
deno add @civility/cloud
Quick Start #
import { IDBStorage } from '@civility/store/idb'
import { CloudStore } from '@civility/cloud'
const cloud = new CloudStore({
id: 'my-app',
storage: { db: new IDBStorage({ id: 'my-app' }) },
collections: ['todos'],
versions: [{ version: '1.0.0', schema }],
cloud: { appId: 'my-app' },
})
const todos = cloud.collection<Todo>('todos')
cloud.connect('https://sync.example.com')
await cloud.login('user@example.com', 'password')
cloud.startSync()
Core Concepts #
Configuration #
const cloud = new CloudStore({
// ── Store options (see @civility/store) ──
id: 'my-app', // inherited by unbound persistent adapters
storage: { db: backend, blobs: blobStorage }, // byte adapter; blobs optional
blobs: { history: { keepLast: 10 } }, // optional logical-history policy
collections: ['todos'],
documents: ['settings'],
versions: [{ version: '1.0.0', schema }],
// ── Server options ──
cloud: {
appId: 'my-app',
manifest, // optional — registered on first sync
selectiveSync: ['todos'], // optional — Only sync some collections
syncInterval: 30000, // ms between sync cycles (default: 30s)
debounceDelay: 500, // ms after a local change (default: 500ms)
timeout: 30000, // request timeout in ms (default: 30s)
pushBatchSize: 200, // max changes per push (default: 200)
shared: { backend }, // optional — cross-app replicas
},
// ── CivShare publisher (§9) — omit to disable publishing ──
share: { appShareUrl: 'https://my-app.example/share' },
})
Every declared collection and document syncs by default. selectiveSync
narrows that; anything left out stays fully local on this device. Blob revision
history is likewise published only for logical blobs referenced by selected
entities, while a blob shared with a selected entity keeps its complete graph.
Omit it to sync everything (the default); [] syncs none.
Cloud synchronizes the stored JSON representation. An app that encrypts data supplies its own encrypted values and manages its keys and recovery.
Connection & Authentication #
// Reconnect a saved session and start syncing — restoreConnection → connect →
// setToken → startSync, in one call. Returns false when there's nothing saved.
if (!(await cloud.restore())) {
cloud.connect('https://sync.example.com')
await cloud.login('user@example.com', 'password')
// or: await cloud.signup(...) / cloud.setToken('existing-token')
cloud.startSync()
}
Sync Events #
Listen to sync events:
cloud.addEventListener('connected', () => console.log('Connected'))
cloud.addEventListener('auth', () => console.log('Authenticated'))
cloud.addEventListener('sync', (e) => {
const { pushed, pulled } = e.detail
console.log(`Synced: ${pushed} pushed, ${pulled} pulled`)
})
cloud.addEventListener('error', (e) => console.error('Sync error:', e.error))
Manual Sync #
// Force push without pulling
await cloud.forcePush()
// Force pull without pushing — returns the number of changes applied,
// or null if a 'force-sync' listener called preventDefault().
await cloud.forcePull()
// Full sync cycle
await cloud.sync()
await cloud.sync({ entities: ['playback'] }) // only these entities this cycle
Both force operations dispatch a cancelable 'force-sync' ForceSyncEvent first — call e.preventDefault() to abort. A normal sync() also invokes forcePull() automatically to recover when the server reports sync.history_pruned (the client's watermark aged out of the retention window); vetoing the 'force-sync' event there surfaces the error instead of auto-wiping.
Blob Sync #
If your documents contain binary data (images, files), provide a blob storage
adapter. The unified Store constructs the byte helper and logical history API:
import { IDBBlobStorage } from '@civility/store/blobs/idb'
const cloud = new CloudStore({
id: 'my-app',
storage: { db: backend, blobs: new IDBBlobStorage() },
collections: ['todos'],
versions: [{ version: '1.0.0', schema }],
cloud: { appId: 'my-app' },
})
// Blobs referenced in documents are automatically uploaded/downloaded
storage.blobs is the physical byte-storage adapter. An unbound adapter inherits
the Store's logical id; IDBBlobStorage then derives the physical database
my-app-blobs. Do not pass 'my-app-blobs' as the adapter ID unless the intended
physical name is my-app-blobs-blobs.
For a standalone byte-only helper, construct BlobStore with the logical ID:
import { BlobStore } from '@civility/store/blobs'
import { IDBBlobStorage } from '@civility/store/blobs/idb'
const byteStore = new BlobStore(new IDBBlobStorage({ id: 'my-app' }))
// Physical IndexedDB database: my-app-blobs
cloud.blobs is the inherited logical API: create, revise, delete,
restore, history/conflict inspection, retention, and availability. Use
cloud.blobs.byteStore only for byte-level put, get, has, and delete
operations. cloud.blob(hash) is different again: it is the cloud-aware,
local-first byte fetch used by lazy sync.
Blob references in documents should use this structure:
interface BlobRef {
hash: string // sha256:...
mime: string // image/png
size: number // bytes
}
Logical blob history sync is a separate, best-effort protocol. Registration
negotiates protocol v1 capabilities for revisions, ancestry skeletons, retention
records before either peer uses
POST /apps/:id/sync/blob-history/push or /pull. A normal cycle publishes
history metadata before changes that may reference it, then pulls the server
projection after change sync. History failure reports a blob_history sync error
but does not stop document or byte sync, so partial availability is expected:
metadata may identify a revision whose payload is remote-only, missing, or
expired. forcePush() republishes history after the server wipe and before full
changes; forcePull() pulls changes and then rebuilds local history from the
server projection.
Failed byte-transfer retry state, including permanent upload failures, persists
in _civ:blobTransfers on the sync metadata backend and survives reloads. An
explicit disconnect/logout clears it. Background transfers use exponential
backoff with jitter and honor Retry-After. Connectivity failures share a
server-wide gate, rather than starting a fresh retry loop for every hash.
After four failed automatic attempts, work stays queued but paused, including
across reloads. Authentication and non-transient failures require user action.
To resume failed work (including permanently rejected uploads after a server
limit changes), call await cloud.retryBlobTransfers() while no sync is active.
This retains pending work and sync cursors; it does not sign out or delete data.
On-demand blob(hash) calls remain explicit operations, outside the automatic
background retry budget.
Server GC derives references from the stored JSON and retains payload hashes protected by logical blob history. Apps that hide blob references inside encrypted values must manage retention outside ordinary reference extraction.
Blob-history capability negotiation is the client/server release boundary. Clients only publish history when the server echoes the full capability set; retained history becomes cross-device after both sides advertise the same protocol. Deploy the server capability before enabling retention on clients.
Application-owned encryption #
Store an encrypted envelope as an ordinary JSON value, with a schema that validates that envelope. Civility does not provision keys, decrypt values, or merge their plaintext. Keep nonce, ciphertext, and authentication tag together as one atomic value when concurrent updates are possible. For blobs, hash the exact bytes supplied to Civility; upload and download transfer those bytes unchanged. Applications own unlock, key distribution, rotation, and recovery.
API Reference #
CloudStore #
Everything Store exposes, inherited unchanged, plus:
| Method | Description |
|---|---|
restore() |
Resume a saved session and start syncing |
connect(url) |
Connect to sync server |
disconnect() |
Disconnect and stop sync |
login(email, password) |
Authenticate user |
signup(email, password) |
Create new account |
logout() |
Log out and stop sync |
setToken(token) |
Set auth token directly |
startSync() |
Begin periodic sync |
stopSync() |
Stop periodic sync |
sync({ entities? }) |
Run a full or entity-targeted sync cycle |
forcePush() |
Push without pulling |
forcePull() |
Pull without pushing |
shared(type) |
A cross-app read-only replica (§3.9.5) |
blobs |
Inherited logical blob/history API |
blob(hash) |
Fetch a blob, downloading it if absent |
retryBlobTransfers() |
Explicitly resume paused/failed transfers and sync |
share |
The CivShare publisher — requires share: + connect |
dispose() |
Release sync, publisher, and store |
Properties #
| Property | Type | Description |
|---|---|---|
connected |
boolean |
Whether connected to server |
authenticated |
boolean |
Whether authenticated |
syncing |
boolean |
Whether sync in progress |
lastSync |
string | null |
ISO timestamp of last sync |
appId |
string |
Current app ID |
api |
SyncApi | null |
Underlying API client |
canShare |
boolean |
share: configured + connected |
SyncApi #
| Method | Description |
|---|---|
login(email, password) |
Authenticate |
signup(email, password) |
Create account |
logout() |
Log out |
verifyToken() |
Check token validity |
getMe() |
Get current user profile |
listApps() |
List user's apps |
createApp(name, desc?) |
Create app |
getApp(id) |
Get app by ID |
deleteApp(id) |
Delete app |
listAppTokens(appId) |
List app tokens |
createAppToken(appId, name, perms?) |
Create token |
deleteAppToken(appId, tokenId) |
Delete token |
registerApp(appId, schema) |
Register schema |
pushChanges(appId, changesByStore, hlc) |
Combined push (all stores) |
pullChanges(appId, sinceHlc, stores) |
Combined pull (all stores) |
uploadBlob(hash, data) |
Upload blob |
downloadBlob(hash) |
Download blob |
hasBlobRemote(hash) |
Check blob exists (false on failure) |
checkBlobsRemote(hashes) |
Batched upload deduplication (fails open per chunk) |
checkBlobsRemoteStrict(hashes) |
Batched existence check; rejects on failure |
Use checkBlobsRemoteStrict for availability UI and transfer preflights. It
deduplicates hashes, sends at most 500 hashes per request, and stops on the
first failed chunk. A rejection means unknown, not missing, for the entire
operation; no partial result is returned. Only a successful response can prove
absence. The legacy checkBlobsRemote API remains available for callers that
explicitly want fail-open upload deduplication.
@civility/cloud/manifest #
Client-side AppManifest validation
import { AppManifestSchema, validateManifest } from '@civility/cloud/manifest'
const { ok, issues } = validateManifest(input) // flat, serialisable issues
CivShare #
Publish a Civility document as a public, read-only url that anyone can visit.
Publishers #
Publish, list, and revoke are local CRUD, so they work offline and take server effect on the next sync:
import { CloudStore } from '@civility/cloud'
const cloud = new CloudStore({
storage: { db: backend },
collections: ['recipes'],
versions: [{ version: '1.0.0', schema }],
cloud: { appId: 'cook.example' },
share: { appShareUrl: 'https://cook.example/share' }, // your rendering page
})
await cloud.restore() // or connect() + auth — `share` needs a connection
// Returns the URL immediately — works offline. Idempotent per (collection, docId).
const { id, url } = await cloud.share.publish({ collection: 'recipes', docId })
await cloud.share.list() // outgoing shares (local read)
await cloud.share.revoke(id) // tombstone → the public URL 404s once synced
await cloud.share.stats([id]) // owner-only view counts
Frozen (snapshot at share time) #
By default a share is live — it always reflects the current document. Pass
frozen: true to capture the doc once at share time and serve that fixed
snapshot forever:
const { url } = await cloud.share.publish(
{ collection: 'recipes', docId },
{ frozen: true },
)
Visitors #
getShare is the anonymous fetch an app's own rendering page calls — no session
needed. Validate the payload against a Zod schema (the trust boundary) and
optionally HTML-escape strings as defense-in-depth.
import { getShare } from '@civility/cloud'
const recipe = await getShare(appId, id, {
server, // from the share URL's ?server= param
schema: RecipeSchema, // required: the server is untrusted
escapeStrings: true, // optional defense-in-depth
})
Blobs in shares #
A shared doc may reference blobs:
import { getShareBlob } from '@civility/cloud'
const ref = workout.photo // a BlobRef { hash, mime, size } from the decoded doc
const blob = await getShareBlob(appId, id, ref.hash, {
server,
mime: ref.mime,
})
img.src = URL.createObjectURL(blob)
Shares publish the stored representation. Ordinary restricted recipients remain available; built-in encrypted share copies and recipient key exchange are not provided. Applications that need confidential sharing must design their own payload and key distribution before publishing.