Skip to content

Second installation on Medplum and openIMIS: Phase A by hand (evidence, harness, fixtures) - #1548

Draft
abrichr wants to merge 8 commits into
feat/dup-check-record-channelfrom
bench/second-installation
Draft

abrichr wants to merge 8 commits into
feat/dup-check-record-channelfrom
bench/second-installation

Conversation

@abrichr

@abrichr abrichr commented Oct 25, 2026

Copy link
Copy Markdown
Member

Draft. Stacked on #1516, which carries the soak harness (#1513) and the FHIR duplicate check.

Second installation: bringing OpenAdapt up on two systems that aren't OpenEMR, and measuring the work from nothing to an admitted, running workflow under the production configuration. Phase A is by hand; Phase B regenerates the parts with generate-parts (#1520) and compares the two. Full write-up: packages/flow/benchmark/second_installation/README.md.

Where it stands

  • Medplum 5.2.8 (FHIR): the fixture, the contract and the teach all work on the live server.
    • Three demonstrations with different values each saved exactly one Patient, and every field read back right.
    • The program compiled to 37 steps and 10 parameters.
    • Admission is blocked. qualify refuses the save because a labeled Create button alone on its row gets no identity check (bug 17, designed in feat: arm a create-new form's save by the form the run typed into #1543). The web fault driver also needs a CSS selector (bug 18, in its own PR).
    • No soak has run yet.
  • openIMIS 25.10 (claims): blocked on the environment. Its amd64 backend never answered under qemu-user on the Podman VM; the next lease turns on Rosetta. The contract, hooks and soak adapter are written but have not run.

In this PR

  • Soak harness: an app registry, and an oracle schema supplied by each app (columns, value maps, identity rule, string ids). There is also a claims workload, plus soak adapters for openIMIS and Medplum.
  • Lifecycle proof: an AppProfile (contract, program, user, test copy, signed-in check, policy), and the production arm's app_profile option.
  • benchmark/fhir_medplum:
    • a pinned, loopback-only compose file and a fixture: admin token, client applications, a dedicated user, a Postgres oracle;
    • a FHIR verify.yaml, session and reset hooks;
    • the teach script.
  • benchmark/openimis_claims: a claims verify.yaml (SQL on tblClaim through a read-only role) and its hooks.
  • Product fixes, with tests:
    • FHIR extract_path can filter a list (telecom[system=email].value).
    • PlaywrightBackend sets an ISO date into <input type=date> with fill.
  • Evidence: results/medplum-phase-a/, holding the qualify refusal, the compiled program's steps, and the Phase B inputs (three before/after FHIR snapshots). There is also the hours ledger, and 20 generality bugs, each marked fixed or open.

Measurement

The hours ledger records one AI agent's wall-clock time, which isn't an engineer's hours. So the README also gives lines written per step: for Medplum, 428 lines for the fixture, 178 for teach, and 299 for the contract and hooks.

🤖 Generated with Claude Code

abrichr and others added 7 commits October 24, 2026 20:16
The oracle's columns, value maps, name keys and identity rule come from the
app (OpenEMR's patient_data stays the default), row ids may be strings, the
workload takes the app's name, and the stops report reads it from the run.
Adds the second-installation hours ledger.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The verify.yaml, source bundle, dedicated user, workflow id, test copy,
signed-in-user check and policy come from an AppProfile (OpenEMR is the
default). The production arm takes app_profile. Adds the claims workload
for openIMIS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…IMIS and Medplum (offline drafts)

- benchmark/fhir_medplum: a pinned, loopback-only Medplum 5.2.8 compose, a
  fixture (admin token, client applications, a Postgres oracle), a FHIR
  verify.yaml and session/reset hooks.
- benchmark/openimis_claims: a claims verify.yaml (SQL on tblClaim through
  a read-only role) and session/reset hooks.
- benchmark/second_installation: AppProfiles for both apps.
- The FHIR verifier's extract_path can filter a list
  (telecom[system=email].value): a Patient keeps phone and email in one list.

Nothing here has run against the live apps yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… bugs found so far

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… native date inputs

- benchmark/fhir_medplum/demo.py records Medplum's new-patient form (the
  gender autocomplete takes ArrowDown + Enter; Enter alone submits the form)
  and compiles it (37 steps, 10 parameters).
- The fixture creates the dedicated user through a project invite.
- PlaywrightBackend sets an ISO date into <input type=date> with fill: Chromium
  splits the input into locale-ordered segments that a typed ISO string can't
  fill. Tested on a local page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ed), evidence and generality bugs

Medplum was taught (three demonstrations, every field read back) and its
contract written; qualify refused the save step: a labeled Create button alone
on its row gets neither an identity check nor a submit binding (bug 17, a
safety design question), and the web fault driver needs a CSS selector the
recorder captures only for an #id (bug 18). openIMIS's amd64 backend hung
under qemu-user on the Podman VM. Evidence, the hours ledger and 20 generality
bugs are in benchmark/second_installation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Create now carries a form identity (bug 17, #1543). The run gate then refused seven unarmed field clicks on Medplum's placeholder-only form (bug 21), so the teach reaches those fields by keyboard from armed clicks; all 10 clicks are armed. Certification still refuses two key steps that only move the focus and assert no postcondition (bug 22). Evidence in results/medplum-phase-a/lease-2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant