Skip to content

Duplicate check asks the system of record (FHIR/REST, SQL fallback) instead of the screen list - #1516

Draft
abrichr wants to merge 13 commits into
mainfrom
feat/dup-check-record-channel
Draft

abrichr wants to merge 13 commits into
mainfrom
feat/dup-check-record-channel

Conversation

@abrichr

@abrichr abrichr commented Oct 24, 2026 •

Copy link
Copy Markdown
Member

Stacked on #1513 (which is stacked on #1514). The base is measure/production-lifecycle, so the diff shows only this change.

In the production soak (#1513), the duplicate check was the largest operator cost after approvals. no_duplicate_candidate reads OpenEMR's on-screen match list and stops any new patient who shares a last name or a date of birth with an existing chart.

What changes

A check whose source is the record (runtime/checks/duplicate_record.py; the S5 seam in docs/design/DECOMPOSITION.md). It has three channels:

  • FhirPatientSource: generic FHIR R4 Patient search;
  • RestPatientSource: an OpenEMR adapter (/api/patient);
  • SqlPatientSource: an OpenEMR adapter, the fallback (patient_data, MySQL %s parameters), wired by hand only.

It searches:

  • by the date of birth (and with day and month swapped);
  • by each part of the last and first name, and by first name;
  • by MRN, phone and email.

It reads every page and decides:

Decision Result
exact: same first name, last name and date of birth, or the same MRN stop
close: the rules in docs/DUPLICATE_CHECK.md (name variants, nicknames, initials, prior names, near dates with a tie-breaker) stop for a person; the halt names each record id and why
none proceed
unreadable: see below stop (fail closed)

The check is unreadable when:

  • the channel is unreachable or refuses the credentials;
  • the reply is not a result set, or a REST body carries any error field or no data list;
  • a reported total doesn't match what was read;
  • the search runs past 50 pages, or a REST page is full with no next link;
  • a next-page link points to another host;
  • a record has no id;
  • the run's date of birth is missing or not ISO.

A record date that is only a year or a year and month counts as no date, so a name match is close.

One source for every path. Replayer resolves the source itself, from its own effect verifier and the program's pin (resolve_duplicate_source). So qualification (qualify/campaign.py), a run (build_replayer), and the lifecycle proof's run, execute and resume all use the same check:

  • no path can pass another;
  • a qualification replayer_kwargs hook that tries is refused;
  • a verifier the resolution doesn't recognize, such as a test double without records_path, gives no source instead of an error.

There is no deployment-level switch any more, so nothing outside the program can make one path differ.

The program decides. The postcondition field candidate_source is part of the qualified program:

  • record: FHIR or OpenEMR REST. A run without the channel fails closed, never the screen.
  • screen: the on-screen list.
  • unset: FHIR when the run has it, else the screen.

So REST is opt-in: only a program pinned to record uses it, and FHIR is the only automatic channel. The soak's OpenEMR program change pins record. candidate_record_fields maps a program's own parameter names; the default is OpenEMR's.

The stop reason reads failed: no_duplicate_candidate (record check via <channel>: <decision>, ...). unreadable is a program or channel problem, and exact and close are about the item; I've told #1517's author the exact form.

The exact numbers (offline)

A miss is a second chart for a patient who already has one.

Test set: the soak's seed-7 generator, 500 items with planted duplicates and extra charts. That makes 694 items, 440 new patients and 241 true duplicates: 47 from the generator, 194 planted.

screen check record check, first version record check, now
missed, of 241 42 90 24
missed, of the 47 from the generator 0 0 0
false stops, of 440 new patients 252 (36.3 per 100 items) 2 (0.3) 2 (0.3)

Known limit: the record check's 24 misses, uncatchable without extra data:

  • 12 are a new last name, new address, phone and email, and a mistyped day of birth (only the first name is shared);
  • 12 are new first and last names and new contact details (only the date of birth is shared).

Catching them needs an MRN, an insurance id or a person.

The README (benchmark/dup_check/README.md) has every kind: how many were planted, the misses by check, and the rule that catches it. It also covers the two cases the review reproduced, a nickname plus a date typo and a last-name typo plus a date typo. Both now stop, and the screen misses the second. The planted kinds for rounds two to four were written alongside their rules, so only round one's held-out kinds were a blind test.

Limits

  • Live OpenEMR 8.0.0.3 (benchmark/dup_check/live.py; raw output private, OpenAdaptAI/openadapt-corpus#18):
    • FHIR: decided 6 of 6 referrals with known answers right (exact, married name, nickname, swapped date, shared birth date only, namesake). OpenEMR's FHIR has no next link, and its Bundle.total equals the rows returned, so _count truncates silently. The check sends no _count, and OpenEMR returned all 120 of 120 rows. A larger set is untested.
    • REST: 6 of 6 at 500 rows a page. Paging works (_limit, links.next), but the links carry OpenEMR's configured site address. At 5 a page, the check refused the other-origin link and answered unreadable (fail closed). That's why REST stays opt-in.
    • Both match last names by prefix. FHIR doesn't expose a birth name, so prior names are searchable only through the SQL adapter.
  • One seed.
  • No change to the soak's operator-minute headline is claimed until the soak reruns with this on.
  • StepResult.duplicate_check (exact, close, candidate, unreadable) is set on the halted step whenever the check fails, for problems_only switch policy: input problems leave it on, a person re-arms with one signed tap #1517's switch policy; omitted when unset.

Tests

  • test_duplicate_record_check.py (52) and test_duplicate_source_paths.py (11).
  • test_no_duplicate_candidate.py, and the deployment, approval, profile, run-gate, qualification and lifecycle tests that build a replayer.
  • The launcher copy test (test_shipped_only_copy.py).
  • ruff check, ruff format --check and both mypy gates report nothing in the changed files.
  • The public artifact inventory is regenerated. It also registers Production configuration soak: verified items per operator-minute, before and after (evidence) #1513's soak results, which are in this PR's base.

🤖 Generated with Claude Code

abrichr and others added 6 commits October 24, 2026 19:46
…; SQL fallback)

no_duplicate_candidate stopped every new patient who shares a last name or a
date of birth with any chart. When the deployment's effects read a FHIR
server or OpenEMR's patient API, the check now searches the record channel
(every page, fail closed) and stops only on an exact or close match. The
on-screen list stays the fallback and runtime.duplicate_check: screen keeps
it. Offline on the soak's seed-7 stream: false stops per 100 items 44.8
(screen) against 0.4 (record), the same 12 of 110 true duplicates missed by
both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…reakers

The first version missed every married name with a mistyped day of birth;
in each, the phone, street and email matched the chart. The record check now
also searches by first name, phone and email, knows common nicknames, swapped
first and last names and the chart's birth name, and uses shared contact
details to tie a mistyped date of birth to a chart. Every new match stops for
a person.

Extended seed-7 stream (622 items, 169 true duplicates): misses 60 -> 24, all
24 a name change plus a move that shares nothing with the chart; false stops
unchanged at 2 (0.3 per 100 items). Screen: 252 false stops, 30 misses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ate prefix

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lifecycle's six cases

All six cases production-lifecycle named stop as close matches: a married
name with a new address, a mistyped day, a second surname, an initial, a
crossed name, and both names misspelled with the same contact. The initial
was the one gap. Extended stream (676 items, 223 true duplicates): false
stops 2 (0.3 per 100 items), misses 24, all name change plus move.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… record check

The record source was set only in deployment.build_replayer, so
qualification (qualify/campaign.py), the lifecycle proof's run, execute and
resume built Replayers that silently used the on-screen list. Replayer now
resolves the source itself from the effect verifier it is given
(resolve_duplicate_source), so every path that reads the same system of
record uses the same check. The postcondition's candidate_source: record
pins the record check into the qualified program; without a channel the
step fails closed instead of reading the screen. The soak's OpenEMR program
change pins it.

tests/test_duplicate_source_paths.py scans every Replayer(...) construction:
none picks a source of its own, the governed paths pass their verifier, the
same verifier gives the same source through build_replayer and directly, and
a pinned program fails closed without a channel. RestPatientSource and
SqlPatientSource are labeled OpenEMR adapters; FhirPatientSource generic R4.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
abrichr added a commit that referenced this pull request Oct 25, 2026
A missed duplicate costs far more than a false stop, so the duplicate
rule is now chosen for the fewest calibration misses within an estimated
false-stop budget (a knapsack: gain-first and gain-per-cost greedy, the
better kept). New signals: a generic, model-made table of given-name
variants (common names only; no record is sent), the medical record
number as an exact match, and typos in both names in the error model.

Adoption rule: a generated rule replaces a system's hand-built duplicate
check only if it misses no more on the same calibration set (identity.adopt;
generate-parts build --reference-duplicate-check). On #1516's stream it
does not (it misses 45 or more of 604 calibration duplicates where the
hand-built check misses 3), so the hand-built check is kept. The frontier,
the reasons and every arm are in benchmark/generated_parts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
abrichr added a commit that referenced this pull request Oct 25, 2026
- An explicit switch-off from the gate, R2 or the engine is never classed
  down to an input problem.
- A duplicate stop counts as an input problem only from the duplicate
  check's structured outcome (StepResult.duplicate_check == 'candidate',
  set by the record-channel check, #1516), never from the error text. An
  unreadable list or channel turns the switch off (record_check_lost); a
  duplicate stop that doesn't say is a program problem.
- A re-arm's approver must be its key's registered name.
- Only a re-arm that verifies ends an off period, so a forged event can't
  hide a switch-off or reset the count.
- Trusting the registered approvers by default is documented as deliberate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
abrichr added a commit that referenced this pull request Oct 25, 2026
Calibrating a matching rule on one site's few dozen records can't learn
which near-matches are one person, so identity is now a reviewed template
(patient-identity/1: #1516's record check, never generated) fed by a
generated per-site mapping: which record fields feed each slot (given,
family, prior last name, date of birth and its format, MRN, phone, email,
street) and which task parameter feeds each input. Fields are chosen by
grounding on the demonstrations, the slot's name and its value shape; FHIR
coded lists are read by code (telecom[system=email].value).

The mapping is verified against the demonstrations and a perturbation
suite built from the site's own records, with one case per slot that only
that slot can catch, and adopted only if it does no worse than the site's
hand-written adapter (adopt_mapping; build --reference-identity).

Back-test on #1516's people and stream, in three site shapes (#1516's rows,
OpenEMR REST, FHIR): the generated mapping matched the hand-written
adapters on the suite and the stream (24 of 223 missed, 2 false stops). A
FHIR mapping read by first list item only (no email, no prior name) misses
30 suite duplicates against 5 and is refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
abrichr and others added 4 commits October 24, 2026 21:55
…t-in

CI: resolution no longer reads attributes a verifier double lacks (unit
shards), the CAPABILITIES row states what was measured (copy test), and the
public artifact inventory is regenerated (lint).

Fail-closed reads: a REST body with any error field or no data list, a
partial record birth date (now no date, so a name match is close), a record
with no id, and a next-page link to another host are all unreadable instead
of a silent none.

One resolution: the deployment switch is gone; the program's
candidate_source (part of the qualified program) and the run's own verifier
decide on every path, and a qualify hook can't set the source. FHIR is the
only automatic channel; OpenEMR's REST adapter needs candidate_source:
record. candidate_record_fields maps a program's parameter names. The SQL
adapter uses MySQL %s parameters and a bound wildcard.

Recall: a nickname or a last-name typo with a mistyped date of birth now
stops. Extended stream (694 items, 241 duplicates): misses 24 (all name
change plus move), false stops 2 (0.3 per 100); screen 42 and 252.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
StepResult.duplicate_check is set whenever no_duplicate_candidate fails:
exact or close (a record matched) and candidate (a listed row matched) are
about the item; unreadable (the record channel or the screen list could not
answer, or a record-pinned check had no channel) is about the program. None
otherwise and then omitted, so existing reports serialize unchanged. #1517
classes the problems_only switch from it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
benchmark/dup_check/live.py seeds synthetic charts on the pinned fixture and
probes OpenEMR's REST and FHIR patient search. REST honors _limit and gives
links.next, but on OpenEMR's configured site address, which the check refuses
as another origin (fail closed): REST stays opt-in. FHIR gives no next link
and a total equal to the rows returned; without _count it returned every row
(120 tested). The check decided 6 of 6 referrals right through FHIR and REST
at 500 a page. Raw output is private (openadapt-corpus#18).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ord check

candidate_channel: fhir with a RestRecordVerifier on .../apis/default/api/patient
searches OpenEMR's FHIR API on the same server (.../apis/default/fhir) with
the record check's bearer token; rest uses only the REST adapter. Live, REST's
next-page links carry OpenEMR's configured site address, so FHIR is the
channel that answers completely there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
abrichr and others added 3 commits October 24, 2026 23:25
The offline measurement's per-item rows (about 8.5k lines) move to the
private openadapt-corpus; the README keeps the per-kind tables. measure.py
now requires --out so it never writes into the repository.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@abrichr
abrichr marked this pull request as draft October 25, 2026 04:06
abrichr added a commit that referenced this pull request Oct 25, 2026
…e only)

Qualification fault injection, consequential delivery, effect checks and
read-back, the API tier, acting by kind, settling and confirming move
verbatim into mixins (107 units; verify_moved_code finds no edits).
replayer.py goes from 18,300 to 10,926 lines.

Left in place because open pull requests change them: identity checks
(#1518, #1504), _resolve_step (#1504), postconditions and step evidence
(#1516, #1514), the starting-state settle and qualification boundary
(#1449), and the run and step drivers. Two methods that name Replayer
stay too.

Two tests patched module globals the moved code now reads from its new
module, and patch them there as well: resolve (scroll readiness is in
replay.acting, interstitials in replay.settling) and _DIALOG_WAIT_S
(replay.acting).

Checked: strict mypy on runtime/core, runtime/replay and replayer.py;
package-wide mypy; both import contracts; 3,149 tests across the 146
files that import the replayer plus the real-browser e2e files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
abrichr added a commit that referenced this pull request Oct 25, 2026
…s halts a pixel-resolved form field)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
abrichr added a commit that referenced this pull request Oct 25, 2026
…s halts a pixel-resolved form field)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
abrichr added a commit that referenced this pull request Oct 25, 2026
…min/item, 0 silent wrongs; Phase 0 gate met)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Base automatically changed from measure/production-lifecycle to main October 25, 2026 15:31

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant