charter
An agent harness for spec-driven development.
Star
Fork
Watch
charter spec work-queue.md
5.8 kB · 153 lines
Markdown
at main

id: work-queue kind: architecture title: Event Stream and Work Queue status: active #

Event Stream and Work Queue #

Charter standardizes on Jazz-native coordination primitives instead of rebuilding a parallel ledger beside them.

The central event stream is the semantic spine. The work queue is one projection from that stream for events that require action.

Layers #

1. Jazz row-version history #

Jazz 2 retains row versions with ancestry, authorship, timestamps, schema hash, environment, and user branch. Concurrent edits can produce multiple ancestors even though last-writer-wins selects the visible scalar value.

This is the raw provenance layer. It answers: who changed this row, along which ancestry, and when?

Charter should not copy raw storage history into a second canonical history table. Semantic documentVersions exist for reproducible context and content-addressed project meaning, not as a counterfeit copy of Jazz internals.

2. Project event stream #

Charter adds semantic project events when raw edits become meaningful to agents or UI.

In Jazz, semantic events are immutable rows in the project-scoped projectEvents table. Queries and subscriptions provide the shared stream.

Initial event sources:

  • document create, update, delete, archive, rename.
  • Markdown projection sync.
  • rich editor writes.
  • span creation.
  • comments and action comments.
  • prompt document changes.
  • consumer registration and subscription changes.
  • packet creation.
  • agent request lifecycle.
  • agent reports, failures, blocked states, ignored states, and no-op observations.
  • eventually, selected agent activity or reasoning traces.

A project event should carry:

  • event id.
  • event type.
  • affected document id and path.
  • span id or comment id when applicable.
  • work item id or request id when applicable.
  • source Jazz row id when known.
  • source field/path when known.
  • timestamp.
  • authority/source.

The current Jazz adapter persists these rows natively. .spec/state/jazz-runtime.json is only an inspection receipt.

3. Event classification #

Not every event needs an agent.

Each event should be classifiable as:

  • record only.
  • creates work.
  • updates existing work.
  • wakes a consumer without creating user-visible work.
  • invalidates a consumer, prompt, packet, or document version.
  • requires human review.

The classification should be visible. Hidden classification is where small automations go to become folklore.

4. Durable work items #

workItems contains mutable project-scoped rows. Work items are separate from event rows because they need claim/status mutation and bounded leases.

Work item lifecycle:

  • queued
  • claimed
  • running
  • done
  • ignored
  • failed
  • blocked

A work item should carry:

  • source event id.
  • affected document id.
  • affected span/comment when applicable.
  • matched consumer ids.
  • attached document ids.
  • source Jazz row id / field when known.
  • claim owner and claim timestamp.
  • linked packet id.
  • linked agent request id.
  • result summary.

This gives agents a durable queue without pretending the filesystem diff is the queue.

Consumers over the event stream #

Workers and agents should subscribe to the project root and event stream rather than poll .spec/state.

Expected shape:

  1. Join a project with a distinct Jazz identity.
  2. Query registered consumer and worker rows.
  3. Subscribe to project-scoped semantic events and matching work.
  4. Match events against consumer subscriptions.
  5. Create or update corresponding WorkItem records.
  6. Build a durable WorkPacket / AgentPacket for selected work.
  7. Claim work with a bounded workerLeases row only when execution starts.
  8. Link the packet and request.
  9. Mark packet, request, and work done, failed, blocked, ignored, or seen through the same report path.
  10. Emit follow-up events when the result itself matters.

The raw Jazz edit history remains available for deeper provenance, undo, blame, or UI display. The work queue is the agent-facing semantic projection over that history.

CLI #

charter events
charter events --document agent-harness
charter events --event-type spec.document.upserted
charter work
charter work --status queued
charter agent

charter sync currently projects Markdown into the configured database, appends project events, and creates queued work items from those events.

Consumer registration can also update the queue by rematching existing queued work. This is a backfill operation: find queued work items, load their source events, test the consumer subscription, and add the consumer id to matchedConsumerIds when it applies. It should be triggered explicitly from the UI or by an explicit matchExistingQueuedWork registration option. Silent retroactive matching is too magical, which is where small agent systems traditionally go to acquire a haunted crawlspace.

That is a compatibility path for the filesystem client. The destination shape is Jazz-native: documents as CoMaps/records, events as CoFeed entries, work items as mutable CoMaps, consumers as CoMaps, packets as CoMaps, requests as CoMaps.

No-op and failure events #

A consumer that sees an event and chooses no action should be able to report seen or ignored. This matters because no-op is often an important decision, not absence of work.

A consumer that fails or blocks should create an inspectable result and, when appropriate, a follow-up work item.

Examples:

  • failed to load a document version.
  • prompt document missing.
  • runner exited with non-zero code.
  • consumer lacks authority to edit target document.
  • span anchor went stale after document edits.
  • event cascade hit retry limit.

Failure is part of the graph. Terminal output is evidence, not the state model.