Skip to content

feat(daemon): add a Connect API daemon and SQLite and PostgreSQL stores - #51

Open
erikreinert wants to merge 501 commits into
mainfrom
feature/docket-daemon
Open

erikreinert wants to merge 501 commits into
mainfrom
feature/docket-daemon

Conversation

@erikreinert

@erikreinert erikreinert commented Nov 19, 2026 •

Copy link
Copy Markdown
Member

Summary

Adds the pieces of a hosted daemon: a versioned Connect API served through a shared verb layer, a Store interface with SQLite and PostgreSQL backends, daemon transports, and client configuration. The daemon serve entry point serves HTTPS with no store attached yet, and the CLI still opens its local database directly.

Changes

  • Define the v1 Connect API schema, with a strict protojson codec, a handshake-first client, request size caps, API major-version windows, and daemon exit and error codes.
  • Add a Store interface with delegating SQLite and PostgreSQL implementations; port the schema migrations to PostgreSQL, serialize writes with an advisory lock, and enforce a DSN policy at open. Only tests use the PostgreSQL store so far.
  • Move engine, CLI, and verb reads and writes off raw database/sql handles onto Store and Tx methods, with read-snapshot transactions for read paths.
  • Add a shared verb layer that the Connect handlers call for every RPC, including author and working-directory headers, lease and CAS translation, and project resolution policies.
  • Wire HTTPS with a static bearer secret to daemon serve. Over HTTP, refuse client worktree paths, drop the workflow source path, refuse Doctor, and refuse conductor-gated RPCs on runs no conductor holds.
  • Add a Unix socket listener in a 0700 runtime directory with peer-uid checks, and a loopback-only plaintext listener with a bearer exemption, as library code covered by tests and not yet wired to a command.
  • Stream events by sequence cursor with long-poll, prune markers, and an in-process broadcast published after each write commit.
  • Persist pinned file bytes and render packets from them; fingerprint idempotency keys and reject reuse with different content.
  • Add client configuration in TOML, bearer token resolution from the environment or a _FILE form, a user-level link file and link command, and a warning on local daemon build mismatch.
  • Make bare --json select the v2 output and reject --json=v1.
  • Add design notes for the Connect API, the Connect verb layer, saga work items, the SQL handle migration, the hold-time report, and the begin-site audit.

Testing

  • CI runs the Go tests, the store contract suite against PostgreSQL, and the pgstore tests against a password-requiring PostgreSQL service.
  • CI's test job runs the QA shell suites against the built binary; the suites were updated for the v2 JSON contract and do not exercise the daemon.
  • No tests were run locally as part of opening this PR.

Notes

  • The branch is behind main and conflicts with it; it needs a sync before it can merge.
  • The branch carries 500 commits across 663 files; reviewing by commit is easier than by the whole diff.

(cherry picked from commit 50b6541b470b8969c11dad0bc1e5d355af9ac7cc)
…liases

(cherry picked from commit 783c597474274fecc81b84fff8ab796ed6fc46e8)
(cherry picked from commit 76a54d5384310491f9666637e1a751e10748bb8f)
(cherry picked from commit bba2a61c1103553ddd052a7558130eee24bd83da)
- pin that a refused fail releases its transaction

(cherry picked from commit dae59d85cfe6ee69f5d94d1b7f54989efd5680a9)
(cherry picked from commit f17179f629791555e61bf0f2b362cdf1262d486c)
… aliases

(cherry picked from commit 88bd5b2a282c9f927bf012dfbc7d15f29827d534)
(cherry picked from commit db06af9f9ebf7aa6ed08e92a9d8219b66ac502ae)
(cherry picked from commit e47eaa10fc2f60dab904c679e117b6aab6c28ebd)
…liases

(cherry picked from commit 0792d9986c740fd750575742f3c57983795c5bee)
(cherry picked from commit 64a16d8ae8da8fc44437261b8352facc2abbb2e1)
(cherry picked from commit 669e65528626505842624f8028ca94a84e767c70)
(cherry picked from commit 12285f73139c6c5c0968ea51d8866e1003255d55)
(cherry picked from commit 3adbdf7471bcceec3a26093a99ee583a590d09c0)
(cherry picked from commit 417e25d9c3586fd4f4c7d802d7600dec72ba26ac)
(cherry picked from commit e8548df6d7b8f446f7ed48321d972d77dfc0c968)
(cherry picked from commit 7e7b954389786ec974a436321293255a4ff5a64e)
(cherry picked from commit 4e4b7990dd0687b072d94113ff0ebd5e6138f7ba)
(cherry picked from commit 3381d629b18528957d235852063578150ba97e9a)
(cherry picked from commit bce51d1d07a77b76920fe74198859b92fd41e19d)
(cherry picked from commit 856918cacd9c5f8076c8686f333ab4df851932aa)
(cherry picked from commit 2618388143ce708f60161d5410c3dc250b73a35c)
(cherry picked from commit 1afdc60c6920c647fc418cd35ed3d53df849a0a2)
(cherry picked from commit 5651cd9c0a6ee2ce7a8713797acb10b808c08632)
(cherry picked from commit f7c0f89dff3ff84276ce76e661a603b16e79e123)
(cherry picked from commit 7cbf44018468820ba172b84032e321e5a5edf0ab)
(cherry picked from commit 6decc16bf2c4971f93fba88889b9ffb453b15b6e)
(cherry picked from commit b1e7a3ef1504026f641fc862e10fad533aca6caa)
(cherry picked from commit c8672d7a4f216ef8a7bb9b0040ab1b5de1b3243f)
(cherry picked from commit c42a52cd3ce43f815127abd9c2b419d524371736)
- A dry run stayed silent when a file-only issue met a glob-only labelled
  issue, or the reverse, because no pair formed and no count said so
- Count linted issues with no comparable pair and warn on them whenever a
  labelled issue is bound
- Correct the checked-nothing and unscoped warnings to state what makes an
  issue comparable and what was actually checked
- A detached gate child that died before its first gate left the step
  gated with no event until a later `next` resumed it
- Commit a saga-detach-started event, with the stranded deadline, before
  the launch; skip the launch and run inline if it cannot be written
- The Linux sandbox masks protected paths in a writer's worktree with
  null-device binds, or leaves their zero-length mount points behind, and
  each one made the worktree read as dirty, so integration refused every
  writer
- An untracked character device, or a zero-length file at a protected
  path, no longer counts; any other untracked or modified file still
  records a live head
(cherry picked from commit 361922d63d030ee53db7ae248238422d4f5f0540)
(cherry picked from commit 7c428f01bef3485c720e1cc48b894d9b7639d6c0)
(cherry picked from commit f1bf68ca8e26e3270821b49bfb9a94da96733f16)
- the daemon binding encodes any non-zero failed result as a deleted detail
- the sub-issue refusal now carries its count on IssueHasSubIssuesError

(cherry picked from commit 288f4b429b3848d3ae4d92a49e000f20c52ff35b)
(cherry picked from commit c8b83e2071bbb260a829510fdee7b89d548998ae)
…resume

(cherry picked from commit 3e457874c788ae2bc9317042a8561e4b97cb0e03)
(cherry picked from commit c64f0fe83bb35afc06a638da323b9d3f4d34e2e5)
(cherry picked from commit dadd0b643a1beb09ffc80ad2b9807fe0d31e2ca8)
- share the RunPause staging through one pauseRun helper
- record why driftPin and pinFile stay copies of the verb test helpers

(cherry picked from commit bf31899fe888fc313c597d28f441381aed21ea5e)
(cherry picked from commit 849daada870390dd3764fb682d69dff22b6f2bff)
(cherry picked from commit ec24b40fe13cb6f82660ec3e712a43a49e577f6d)
(cherry picked from commit 432ec547947c824f4f9d8d84273c411c3d992e57)
(cherry picked from commit c0b5b23dd6274cf79adef5ccfe232e5d66db5ec6)
(cherry picked from commit a2a67a58ea99b8d72718530cbe7d0b9c3ba9b387)
(cherry picked from commit b9802df5180828f9bd73319c8e0343a8d49b4670)
(cherry picked from commit 7685002bf80e679d0b9fda8c5510208fb41be027)
- verb.Env carries the command path as Verb; CLI helpers stop writing

(cherry picked from commit 621d30e27ad3427a185c2f986e0d9321683d7c02)
(cherry picked from commit 4668dfaa22ea8ad0572af11d65b46b874a2e6408)
(cherry picked from commit abb63d33a2abc00eb2788faf83b662bf9e18e24c)
(cherry picked from commit 885a9f5d4126c50fbe0ef75eea3fd656415777ab)
(cherry picked from commit ed083530cb5e6f73d735517c9be3047bb6993a9e)
@erikreinert erikreinert changed the title feat(daemon): serve a Connect API daemon backed by SQLite or PostgreSQL feat(daemon): add a Connect API daemon and SQLite and PostgreSQL stores Nov 19, 2026
- keep the branch side for the integration, recorded-diff and
  interrupted-gate fixes, which the branch already carries as 037d95b,
  ec7d918 and ff6ccc5 on top of the store-interface move
- keep the branch's newer module versions (sqlite v1.62.0)
- carry 795fd7f: the trust probe looks entries up with LookupAt under
  the probe worktree, with its declared-file regression tests
@erikreinert
erikreinert marked this pull request as ready for review November 19, 2026 04:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant