Durable engine

Every sync is a durable workflow. Siphon binds the engine's orchestration to Restate, so a run survives process crashes, deploys, and transient failures without re-doing completed work or double-delivering.

The pipeline

For each page of the source:

resolve auth → fetch page → change-filter → enrich (DAG) → transform → validate → deliver → advance watermark

Each of these is a journaled step. On resume, completed steps replay from the journal (no re-fetch, no re-delivery); only the step that was in flight when the crash happened re-executes. Pagination state lives in the journal too, so a backfill that dies on page 40 resumes at page 40 — not page 1.

The run, step by step

The worker's siphon-sync-run workflow does, in order:

1. prepare — load the connection, pinned spec, decrypted secrets, watermark, and destination; apply any connection override and re-validate the effective spec.
2. create-run — insert the run as running so it's visible in the inspector immediately.
3. load-fingerprints — if the resource uses client-side hash change detection, load the prior fingerprints.
4. ensure-destination — idempotent DDL (create/evolve tables); reset for overwrite mode.
5. page loop — the pipeline above, one journaled step group per page.
6. persist-run — write call logs, run errors, quarantine, the advanced watermark, and updated fingerprints; set the final status.
7. notify-host — fire outbound webhooks.

Delivery guarantees

Delivery is at-least-once and idempotent on (connectionId, resourceKey, externalId). Destinations upsert on that key, so a replayed page overwrites rather than duplicates. The watermark and record fingerprints advance only in persist-run, after delivery — so a run that crashes mid-flight re-processes the affected records next time (safe, idempotent) rather than skipping them.

Failure handling

A definitive HTTP failure (a 4xx, or a 5xx that exhausted retries) is mapped to a Restate terminal error — the run fails cleanly and records the failing call, instead of retrying forever. Transient failures stay retryable and back off through the HTTP client stack.

Running it

The data plane is three long-lived processes:

• the Restate runtime (the durable execution broker),
• the worker (packages/worker — binds the workflow; restart it after engine or worker changes, it does not hot-reload),
• the scheduler (packages/worker/src/scheduler.ts — see Schedules).

A run is dispatched fire-and-forget to the Restate ingress; the same path serves manual, scheduled, and webhook triggers.