Realtime and canonicalization

The two SSE streams, the canonicalization outbox, and why exactly one backend runs per database today.

Realtime in UseAgent is deliberately simple: durable snapshots over one-way streams, plus an in-process event bus for ambient invalidations. It is not distributed pub/sub, and that shapes how it scales.

Two streams, two jobs

The thread stream’s frame types are snapshot, run, step, delta, native, and done, plus canonical and canonical-complete for the canonical lane; the full contract lives in Events and streaming. Two facts matter here: a done frame settles one run but does not close the stream, and a bounded queue (20,000 live frames) closes the connection on overflow so the browser reconnects to a fresh authoritative snapshot instead of growing without limit. The legacy per-run /:id/events route is retained as the rollback path.

The org-change stream is backed by an in-process event bus. Its change types are run, artifact, automation, provider_connection, and integration_connection, and provider-connection changes are filtered to the target user. It exists to tell a view “something changed, refetch,” not to carry state.

Canonicalization

Engine output is translated into provider-neutral canonical events before the canonical lane is authoritative to the UI. Native frames are retained alongside canonical events for fidelity and debugging. Four invariants govern the lane:

• Enqueue inside finalize. Canonicalization is itself the outbox work item, enqueued inside the run-finalization transaction, so the intent to translate commits atomically with the terminal run. Workers claim pending items with FOR UPDATE SKIP LOCKED.
• Watermark-stable seal. The worker marks a run complete only when the source watermark (max native frame seq plus a step content signature) is stable across the translate; until then it retries, never freezing a partial snapshot.
• Persist before publish. Canonical rows are written with an immutable thread-wide delivery cursor and append-only revisions, and only then emitted to live subscribers, so a reconnect replays exactly what live showed.
• Never trust provisional. The client trusts the canonical lane for a run only after the durable canonical-complete signal, never on the mere presence of provisional rows.

One backend per database

This is the single most important operational constraint, and it is intentional for the current release.

Boot acquires a per-database Postgres advisory lock. A duplicate backend warns by default and refuses to boot when REQUIRE_SINGLE_BACKEND=1, which production sets. Multi-replica realtime would require a durable, database-backed seal and org-event fan-out first; until that exists, adding replicas is not safe and is not claimed. See Operational invariants.

What this buys, and what it costs

• Buys: a simple, debuggable realtime path with durable reconnection and no external message broker.
• Costs: horizontal scale of the realtime tier waits on durable fan-out. It is a known, bounded piece of work, not an accident.