Skip to content

feat(quickstart): show it once, run it, break it - #1408

Merged
abrichr merged 16 commits into
mainfrom
feat/quickstart-three-acts
Oct 20, 2026
Merged

abrichr merged 16 commits into
mainfrom
feat/quickstart-three-acts

Conversation

@abrichr

@abrichr abrichr commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

openadapt quickstart is now one short story built around one moment: the screen said "Encounter saved", the database had nothing, a bot that trusts the screen would report Done, and OpenAdapt stopped.

The new quickstart, recorded with VHS

(Recorded with VHS, browser hidden, at load average 15–18 from other sessions on this Mac: 21.9 s in all.)

The three acts

  1. Show it once. The browser opens (when there's a display; --headless hides it) and the demonstration runs in the synthetic clinic app, MockMed. Each step ticks by in the terminal with its time, read from the recorder's own event log. Then: Compiled into a program: 6 steps.
  2. Run it. The program runs on 5 new notes, all at once: five worker processes get ready during Act 1 (each with its own browser and its own copy of the clinic's database), then each runs the program once through the same governed path as any run. The counter ticks as runs finish. Each row is the note, ✓ checked in the database, and that run's time. Totals: 5 of 5 saved and checked · 0 AI calls · $0.00 · <wall time>.
  3. Break it (default; --no-break skips it). On a terminal it asks "Press Enter to make the next save fail silently". The same program runs once more while the database rejects the write after the app has shown "Encounter saved". Side by side: the halted run's real screenshot (inline on iTerm2, WezTerm, Kitty and Ghostty; a quoted box elsewhere) and the database's 0 new records, A screen-trusting bot ✓ Done, OpenAdapt ■ Stopped before reporting success, and one sentence of why.

The finale is at most 12 lines: a receipt card (program, 5 of 5 checked, AI calls, cost and total time, Seal key), Check it: openadapt seal verify openadapt-quickstart, one Details: line, and exactly two next steps: record your own app, and serve it to an agent. Under uvx it adds Keep it: uv tool install --python 3.12 openadapt, and one warning when the shell's openadapt is another version (for example a Conda bin ahead of ~/.local/bin), naming its path and version. The side-by-side page from openadapt flow demo opens when a display is available. --verbose adds the engine detail that used to be the default output.

Every number comes from the runs: step times from events.jsonl, counts and AI calls and cost from each report.json, records from the app's database read out of band, the quoted screen text read off the halted run's screenshot. The tests feed the story different facts and check that the output follows.

Before and after

Before (openadapt 1.17.0 behavior, quickstart --break-it): 74 lines, three summaries, eight next steps, 42.4 s on this machine.

Old output
[1/5] Record the demonstration against a real persistence boundary
[2/5] Compile and propose a fixture-specific effect contract
      2 system-of-record effect proposal(s) derived from the fixture's record delta on step_005; review is required for a real deployment
[3/5] Certify against the clinical-write policy
[4/5] Admit and execute under the standard profile
      VERIFIED in 2.9s; 0 model calls; the system of record holds 1 record(s)
[5/5] Emit the local run receipt

[break-it] Rerun the SAME certified bundle, but this time the backend
[break-it] runs fault mode 'optimistic': it rejects the write,
[break-it] so the save never reaches the record.
[break-it] HALTED: the system of record holds 0 record(s); the screen said otherwise.

--- clean run: the certified bundle, against an honest backend ---

Result: Done and checked. It saved the entry and read the record back to confirm it.
VERIFIED: openadapt-quickstart/run/REPORT.md
  transaction     VERIFIED
  metering class  billable (this local tutorial was not reported or charged)
  profile         standard
  model calls     0
  effects         2/2 confirmed by an independent system-of-record read (Seal Oracle tier 2)
  bundle digest   d9a446e4f01bb1e901e565255053857582766fd28bcb42d831a4c4ca648d9f50

Shareable receipt: openadapt-quickstart/run/receipt.png
                   openadapt-quickstart/run/receipt.json

--- break-it: the same certified bundle, with the save set to fail ---

  Injected fault:      'optimistic' -- the server rejects the write
  On screen:           every on-screen check passed, including the saved message
                       (observed on screen: "Encountersaved-")
  The verifier found:  1/2 declared effect(s) REFUTED by an independent
                       read of the system of record, which holds 0 record(s)
  The engine did:      HALTED at the consequential step instead of claiming
                       success (transaction: RECONCILIATION_REQUIRED, billable: no)
  Result:              Check the record. A save may have gone through. A person checks the record before anything is retried.

  The screen looked the same as a good save. The record check found no new note,
  so the engine stopped instead of reporting done.

  Engine's own words:  System-of-record effect verification HALTED step 'step_005' (click 'Save Encounter'): record_written refuted against the rest system of record and could not be reconciled (escalated) — [rest] record_written: 0 records match the target selector, expected 1 (missing / phantom / rejected write -- the screen may show success but nothing landed) -- no compensator available for an irreversible refuted effect -- durably halt and escalate — run aborted

Caught-fault evidence (local report, NOT a success receipt):
  openadapt-quickstart/run-broken/REPORT.md

No shareable receipt for the halted run: only VERIFIED runs may use the
success rail. The halt itself is the demonstration.

See both runs side by side on one page:
  openadapt flow demo

Next: record your own workflow:
  openadapt flow record --backend web --url https://your-test-app.example --out my-task

Local quickstart complete.
Bundle: openadapt-quickstart/bundle
Run evidence: openadapt-quickstart/run
Outcome: VERIFIED under the Standard profile.
The synthetic write was confirmed through a read-only system-of-record API.
No model or Cloud call was enabled.
The rerun against the fault-injecting backend HALTED as designed: the form was submitted, but the system of record had no new note.
Caught-fault evidence: openadapt-quickstart/run-broken/REPORT.md
Result: check the record (transaction outcome RECONCILIATION_REQUIRED). A save may have gone through, so nothing is retried until a person checks the record.
Inspect qualification gaps: openadapt flow lint openadapt-quickstart/bundle
See it stop when the screen changes (a separate MockMed demo):
  openadapt flow demo-record --out openadapt-quickstart-drift/rec
  openadapt flow compile openadapt-quickstart-drift/rec --out openadapt-quickstart-drift/bundle --name drift-demo
  openadapt flow replay openadapt-quickstart-drift/bundle --drift modal --run-dir openadapt-quickstart-drift/run
Give a local agent the run tool (synthetic MockMed, no frames):
  openadapt agent serve --allow-run
A client sends authorized work and gets a receipt. Frames stay here.
Connect this computer when you want Cloud history and collaboration: https://app.openadapt.ai/dashboard/settings/ingest
Qualify a consequential workflow: https://openadapt.ai/qualify

After: the GIF above (non-TTY output is the same text, one line per fact; see tests/snapshots/quickstart_plain.snapshot).

The inline screenshot on terminals that draw images (WezTerm, iTerm2 protocol; the Kitty protocol renders the same):

Act 3 with the screenshot inline

Speed

Measured on this Mac (M2 Max, shared with other sessions), headless, not on a terminal, interleaved so load hits both sides (scripts/quickstart_timing.py; data in docs/quickstart/).

Version Before: engine tutorial with --break-it (median) After: openadapt quickstart (median) Load
Sequential runs (first push) 22.1 s 32.9 s 19–47
Five runs at once, final (5 rounds, timing-2026-10-20-parallel.json) 21.6 s 22.3 s 9–36

Rounds overlap (before 21.0–23.0 s, after 21.6–24.2 s), but the median is still about 0.7 s slower than the old tutorial, which does one checked run where this does five plus the screenshot read, the page and the Seals. What brought it down from 32.9 s:

  • Act 2's five runs at once (about 3.6 s instead of about 13 s).
  • Workers start early at lower CPU priority with two OCR threads each, and the bundled-browser check runs once, off the main thread.
  • The five Seals are signed and the workers shut down during Act 3; the halted run's screenshot is read while the engine waits on the database, and the page is written alongside.

What didn't help: three runs instead of five (23.7 s vs 23.8 s, the same), and keeping the workers until after Act 3 (27.8 s). The phases that remain slower than the old tutorial are compile (+0.5 s, sharing the CPU with the workers getting ready) and Act 3 (+0.3 s).

Workers use Playwright's own Chromium. Five installed-Chrome instances at once stalled 2 of 20 runs on this loaded Mac; the bundled build ran 30 of 30. Workers start only when the bundled build is already on disk (or the person set OPENADAPT_FLOW_BROWSER); otherwise nothing extra is downloaded and the five runs go one after another (the header then doesn't say "all at once"). A run that stalls for 45 s has its worker stopped and is repeated in the main process in a new folder, and its row says "(on a second try)". A run that raised is reported as not finished, not repeated.

An older openadapt earlier on PATH

On the founder's Mac, openadapt resolves to an Anaconda 1.5.1 launcher ahead of the 1.17.0 one that uv tool install put in ~/.local/bin, and the old one answers new commands with "invalid choice". Old releases can't be changed, so this launcher now names the problem whenever it does run:

  • openadapt --version and an unknown command check every openadapt on PATH. If the first is another version, one line names its path and version and the fix (for example /opt/homebrew/anaconda3/bin/python -m pip uninstall openadapt openadapt-flow, then run: hash -r); if a newer one is elsewhere on PATH, one line names it. Copies of the same version get no warning. Checked on this Mac: it names the Anaconda 1.5.1 install and stays quiet about the matching ~/.local/bin copy.
  • Under uvx, the quickstart finale adds the "Keep it" line and the shadow warning described above.
  • tests/test_launcher_conflicts.py and tests/test_cli_uvx_hints.py cover both.

Where I departed from the brief, with evidence

  • "5 synthetic patients" became 5 new notes for one synthetic patient. The compiled program is bound to the demonstrated patient: its first click is #open-p1, and its record check matches patient_id='p1' with the idempotency key mockmed-triage-p1-v1 (see bundle/workflow.json). Running it on other patients would be a wrong-patient write, which is what OpenAdapt exists to refuse. What the program does generalize over is its input, the note. Flow's run_tutorial_workflow gains params=...; the run authorization binds them and the bundle is unchanged; the record check reads the run's note (bind_effect_params). Picking the patient as a run input needs a recording that selects the patient by a typed value; that's a follow-up.
  • "Compiled into a program: N steps, 0 AI calls" prints the step count only. The compiler has no AI-call counter to read, and the brief asks for no hard-coded numbers. The AI-call count appears where it's measured: Act 2's totals and the receipt card, summed from the run reports.

What ships when

  • The launcher needs Flow 2.2.0 or newer for the three acts; older Flow and engine-only flags (--guided, --interactive-record) run the engine tutorial as before. --break-it and --headed stay as hidden aliases, so every documented command still works.
  • With Flow 2.2.0 from PyPI (no params yet), Act 2's header says "the program 5 times" and every run uses the demonstrated note. With the Flow in this PR, it says "on 5 new notes". Checked in a fresh environment with Flow 2.2.0 from PyPI: the quickstart runs, the Seal line appears, and seal verify reports 5 of 5 valid.
  • Offline Seals: Flow 2.2.0 already signs them; the quickstart uses only the calls 2.2.0 and main share. Each checked run's report.json is signed with this computer's Seal key (generated under ~/.openadapt/seal on first use, the offline-Seal default), and openadapt seal verify <folder> checks every sealed run in the folder (a tampered report shows run-3 not valid, exit 1).
  • README and docs.openadapt.ai are unchanged here, so no public page shows output that PyPI doesn't ship yet. The GIF is in docs/quickstart/ for the README swap at release.

Changes

  • openadapt/quickstart/: engine.py (drives Flow's tutorial pieces and returns facts), workers.py (Act 2's worker processes), story.py (the three acts), view.py (rich view and plain lines; plain when not a TTY, NO_COLOR or TERM=dumb; ASCII marks when the encoding lacks ✓ ■), images.py (iTerm2 and Kitty protocols; detection from the terminal's own variables, never inside tmux or screen, OPENADAPT_INLINE_IMAGES=0|iterm2|kitty to override).
  • openadapt/cli.py: the new quickstart (--headless, --no-break, --verbose), seal verify <folder>, help and the Python remedy point at openadapt quickstart. Commands print through _hint from fix(cli): quickstart's next steps reach the same version under uvx #1404 (this branch is stacked on it), so uvx users get commands that reach the same version.
  • pyproject.toml: rich (MIT; pulls markdown-it-py, mdurl and Pygments), gated to Python < 3.13 like the engine; install-weight allow list updated. Flow's dependencies are unchanged.
  • Flow: run_tutorial_workflow(params=...).
  • scripts/quickstart_lifecycle.py: the weekly lifecycle job checks every run-N and that the silent failure stopped.
  • scripts/quickstart_timing.py and docs/quickstart/timing-2026-10-20.json: the timing data; docs/quickstart/quickstart.tape re-records the GIF.

Tests

  • tests/test_quickstart_workers.py: the worker pool with stand-in workers in real processes: all runs finish in parallel, fewer workers take turns, a stalled run comes back in a new folder, a run that raised is reported and not repeated, workers that can't start hand every run back.
  • tests/test_quickstart.py: plain-output snapshot; acts in order; no internal vocabulary in default output; --verbose detail; finale ≤ 12 lines with exactly two next steps; numbers follow the runs (a fake engine with 4 of 5 checked, 3 AI calls, $0.04 and odd times changes every figure); the aha line only when the run showed all of it; no prompt without a terminal, prompt and uncounted wait with one, EOF continues; view choice for pipes, NO_COLOR, TERM=dumb; ASCII marks; a TTY render with colors and boxes; the screenshot inline for iTerm2, WezTerm, Kitty and Ghostty and the box fallback; image-protocol detection (15 cases); Kitty chunking; the event-log ticker with half-written lines; every printed next-step command run with --help in a fresh process with a minimal environment; the uvx spelling; seal verify on a folder (valid, tampered, empty); focus returning to the terminal.
  • packages/flow/tests/test_tutorial_run_params.py: a run with a new note verifies against that note and leaves the bundle unchanged.
  • test_real_quickstart (opt in with OPENADAPT_REAL_QUICKSTART=1) runs the real thing; it passed locally at load average 92.
  • Existing quickstart tests now say which path they test.

Not verified

  • Ghostty and Kitty themselves (the Kitty protocol was checked in WezTerm), iTerm2 itself (its protocol was checked in WezTerm), and Windows.
  • The visible-browser path was run in a real terminal (VHS); the macOS focus return is unit-tested, not observed.

🤖 Generated with Claude Code

abrichr and others added 14 commits October 20, 2026 00:13
openadapt quickstart now tells one story in about 30 seconds: a
demonstration in the synthetic clinic app ticks by step by step and
compiles into a program; the program runs five times with a new note
each time and every save is checked in the app's database; then one save
fails silently and the screen's saved message sits beside the database's
0 new records, a screen-trusting bot's Done, and OpenAdapt's stop.

- openadapt/quickstart: engine adapter (facts read from the runs), the
  three-act story, a rich terminal view with a plain-line fallback, and
  inline screenshots on iTerm2, WezTerm, Kitty and Ghostty.
- Each checked run's report gets an offline Seal; openadapt seal verify
  <folder> checks every sealed run in it.
- Flow: run_tutorial_workflow(params=...) gives a run its own inputs; the
  run authorization binds them and the bundle is unchanged.
- Engine-only flags and Flow older than 2.2.0 keep the engine tutorial.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Breaking a save is now the quickstart's last act by default; --break-it
stays as a hidden alias, so documented commands keep working.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A visible browser on macOS takes focus and keeps it after its window
closes, so the Act 3 prompt's Enter would go to the browser. Also lists
openadapt seal verify FOLDER in CAPABILITIES.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nadapt on PATH

The finale adds "Keep it: uv tool install --python X.Y openadapt" under
uvx, and one warning when the shell's openadapt is another version (for
example a Conda bin ahead of ~/.local/bin). The receipt's time joins the
cost row so the finale stays within 12 lines with every optional line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
openadapt --version and an unknown command now check every openadapt on
PATH. When the first one is another version (an old 1.x in a Conda bin
ahead of ~/.local/bin keeps answering every new command with "invalid
choice"), or a newer one is installed elsewhere, one line names the path
and version and how to remove the other install, then hash -r.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Worker processes start while Act 1 demonstrates and compiles: each loads
the engine, its own browser and its own copy of the clinic's database.
When the program is ready, each runs it once for one note through the same
governed path, and the counter ticks as runs finish. Act 2 drops from
about 13 s to about 4.5 s.

Workers drive Playwright's own Chromium: several installed-Chrome
instances at once stalled 2 of 20 runs on a loaded Mac, the bundled build
0 of 30. Without a bundled build on disk nothing is downloaded and the runs
go one after another. A run that stalls (45 s) has its worker stopped and is
repeated in the main process in a new folder; a run that raised is reported
as not finished.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- One background check for the bundled browser instead of a playwright
  subprocess in each worker; workers start at lower CPU priority with two
  OCR threads each, so they don't slow Act 1's compile.
- Workers shut down during Act 3; the five Seals are signed then too.
- The halted run's screenshot is read while the engine waits on the
  database, and the side-by-side page is written alongside.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Seals use the API Flow 2.2.0 already ships (seal_run with the Seal home,
  public_key_entry), so the receipt's Seal and seal verify FOLDER work with
  the released engine too.
- The display check no longer imports the engine, so the launcher's
  quickstart tests pass on Python 3.13+, where the engine isn't installed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@abrichr
abrichr marked this pull request as ready for review October 20, 2026 06:09
@abrichr
abrichr merged commit 677131c into main Oct 20, 2026
37 checks passed
@abrichr
abrichr deleted the feat/quickstart-three-acts branch October 20, 2026 15:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant