Repository navigation
Conversation
…erified openadapt-flow generate-parts builds the four parts of a workflow that are specific to one application at setup time, and admits each only by a check that does not depend on the generator. Healthy runs still make no model calls; nothing here calls a model. - Read channel (generate/channel.py): candidates from an OpenAPI document, a FHIR CapabilityStatement or teach-time traffic. The channel test reads each demonstration's record, every page, and must read as unreadable through a fault proxy (no auth, 401, 500, truncated, not JSON, a failed next page). - Record contract (generate/contract.py): from the before/after difference across 2-3 varied demonstrations (bound fields, constants, volatile fields, match keys, frame), refined on a test copy by the fault gate until every fault class is killed or reported. - Duplicate rule (generate/identity.py): roles from the schema, clauses calibrated on the system's own records with a generic error model and per-comparison false-match rates; a person labels the pairs that block a clause; unknown goes to a person. - Input types (generate/inputs.py): labels, dates and the source's date order, code tables, admitted by a round trip; ambiguity halts. - A floor generated parts can't go below (generate/baseline.py); a plain- language review, a digest manifest and a person's approval bound to it (generate/build.py, generate/cli.py). - Verifiers: the FHIR read follows next links on the same origin and refuses a short read; the REST read follows records_next_path. - benchmark/generated_parts: the generated duplicate rule on #1516's stream (676 items, 223 duplicates): 54 missed, 0 false stops, against the hand-built check's 24 and 2. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A missed duplicate costs far more than a false stop, so the duplicate rule is now chosen for the fewest calibration misses within an estimated false-stop budget (a knapsack: gain-first and gain-per-cost greedy, the better kept). New signals: a generic, model-made table of given-name variants (common names only; no record is sent), the medical record number as an exact match, and typos in both names in the error model. Adoption rule: a generated rule replaces a system's hand-built duplicate check only if it misses no more on the same calibration set (identity.adopt; generate-parts build --reference-duplicate-check). On #1516's stream it does not (it misses 45 or more of 604 calibration duplicates where the hand-built check misses 3), so the hand-built check is kept. The frontier, the reasons and every arm are in benchmark/generated_parts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Calibrating a matching rule on one site's few dozen records can't learn which near-matches are one person, so identity is now a reviewed template (patient-identity/1: #1516's record check, never generated) fed by a generated per-site mapping: which record fields feed each slot (given, family, prior last name, date of birth and its format, MRN, phone, email, street) and which task parameter feeds each input. Fields are chosen by grounding on the demonstrations, the slot's name and its value shape; FHIR coded lists are read by code (telecom[system=email].value). The mapping is verified against the demonstrations and a perturbation suite built from the site's own records, with one case per slot that only that slot can catch, and adopted only if it does no worse than the site's hand-written adapter (adopt_mapping; build --reference-identity). Back-test on #1516's people and stream, in three site shapes (#1516's rows, OpenEMR REST, FHIR): the generated mapping matched the hand-written adapters on the suite and the stream (24 of 223 missed, 2 false stops). A FHIR mapping read by first list item only (no email, no prior name) misses 30 suite duplicates against 5 and is refused. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- baseline: a contract without a frame, or a frame without an identity field, is below the floor. - adoption: a generated rule or mapping must catch every duplicate the reference catches (a superset), not just miss fewer; the README says the yardsticks are the generator's own synthetic sets and lean toward generation. - a REST next link to another origin reads as unreadable (test); the FHIR one had a test. - CI: import order in rest.py; generate-parts channels' URL flags in BACKEND_PARITY.md; the approve hint uses the CLI spelling helper. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
abrichr
enabled auto-merge
October 25, 2026 01:38
3 tasks done
abrichr
disabled auto-merge
October 25, 2026 03:13
This was referenced Oct 25, 2026
abrichr
marked this pull request as draft
October 25, 2026 03:48
This was referenced Oct 25, 2026
Draft
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft, parked. Before review it needs two things: the live OpenEMR back-test (queued for the shared fixture slot), and
approvesigning with a registered approver key on the #1486 path instead of a typed name. The pagination engine fix moved to #1538; this branch drops it when #1538 merges.openadapt-flow generate-partsbuilds the four parts of a workflow that are specific to one application at setup time. Each is admitted only by a check that does not depend on the generator. Healthy runs still make no model calls, and nothing in this PR calls a model. Design:packages/flow/docs/GENERATED_PARTS.md.Generated, and how each is admitted
generate/channel.py)generate/contract.py)generate/identity_template.py)patient-identity/1: #1516's record check, never generated) plus a generated mapping of this system's fields and the task's parameters onto itgenerate/inputs.py)Massachusetts->MA)generate/baseline.py): every typed parameter is checked, the frame is on, nothing ignores or drops a written field, no collateral rule is off, and there is no attribution. Generation can add checks, never remove one.parts.review.mdstates every check in plain words, including what it can't catch.generate-parts approveneeds a person at a terminal. It refuses an agent's name and any part that changed after generation, because the approval is bound to the manifest digest.nextlinks on the same origin, and a failed page, another origin, more than 50 pages, or atotallarger than what was read all read as unreadable.records_next_path(new, optional; digests unchanged when unset).Credit
FHIR/REST pagination also proposed by @yunusarda-sudo in #1511 (superseded by this PR).
Changelog entry (for the next openadapt-flow release)
openadapt-flow generate-partsgenerates the app-specific parts of a workflow (read channel, record contract, identity mapping onto a reviewed template, input types) from two or three demonstrations, admits each by a check that doesn't depend on the generator, and records a person's approval of the plain-language review (#1520). FHIR reads follow everynextpage on the same origin, REST reads followrecords_next_path, and a broken, cross-origin or short page reads as unreadable. FHIR/REST pagination also proposed by @yunusarda-sudo in #1511 (superseded by this PR).Measured
Identity: a generated mapping onto the reviewed template, against #1516's hand-written adapters. #1516's people and its 676-item stream (223 true duplicates, 440 new patients) are rendered in each site shape. The mapping is generated from the site's rows and three demonstrations (
benchmark/generated_parts/identity_mapping_backtest.py).The generated mapping matches the hand-written adapters exactly on all three shapes. The fourth row loses the email and the prior name. The suite catches that where the demonstrations can't, and the adoption rule keeps the reference.
A rule calibrated on one site's records (the approach the template replaces): at every false-stop budget it missed 52-54 of 223 against the hand-built 24, with 0 false stops against 2. With 30 records, calibration can't learn which near-matches are one person. The frontier, and both adoption rules (
--reference-identity,--reference-duplicate-check), are in the README.Not in this PR
The live OpenEMR back-test is not in this PR; it waits for the fixture lease (
autogen-backtestis in the queue). It will generate all four parts from three demonstrations on the fixture and run the fault gate on the generated and the hand-built contract with the same copy. It will report coverage per class, false stops, cost and time, review effort, and setup effort.Tests
test_generate_{contract,channel,inputs,identity,identity_mapping,parts_pipeline}.py(37 tests).🤖 Generated with Claude Code