diff --git a/CAPABILITIES.md b/CAPABILITIES.md index e541d413f..f66900f16 100644 --- a/CAPABILITIES.md +++ b/CAPABILITIES.md @@ -227,6 +227,9 @@ improvising or retrying when something is off. | Pre-approved repairs: a signed policy lists low-risk changes that apply without a person, a test-record run before each queue, and one approval for every item halted the same way | On main ([#1294](https://github.com/OpenAdaptAI/OpenAdapt/pull/1294)) | Synthetic | standard changes | | Repair a pinned bundle: the fix runs the pinned bundle's own qualification campaign on a test copy, keeps every write, effect, identity rule and record check, and is re-admitted with signed lineage (at most three repairs, no revoked key) that the pin verifies | On main | Synthetic | [SELF_REPAIR.md](packages/flow/docs/SELF_REPAIR.md#a-pinned-bundle-re-qualify-and-re-admit) | | Repair by default: once a repair agent is configured, every stop that wrote nothing goes to it without anyone starting it; a stop after a possible write goes to reconciliation; every stop is a row in a ledger with the funnel from stop to resumed item | On main ([#1360](https://github.com/OpenAdaptAI/OpenAdapt/pull/1360)) | Synthetic (MockMed, 22 drift modes, real Claude agent: 6 of 21 repairable stops resumed and verified, 0 wrong saves in 66 runs); routing over recorded OpenEMR stops; live openIMIS relabel: 3 of 3 reached repair, 0 resumed; silent-saves 1.1.0 UI-change blocks: 18 of 28 items that always stopped now correct, 0 wrong | `run` (on by default, `--no-auto-repair`), `repair:` in `deploy.yaml`, [benchmark/auto_repair](packages/flow/benchmark/auto_repair/README.md) | +| Repair a relabeled save button: prove from the recorded identity hash that only the label changed (same control id, kind and form), re-qualify the fix on the new release, and have a person admit the certified copy afresh, since a save button's identity is part of the record check | On main | Real app (local OpenEMR, synthetic data): after the save button was relabeled, the stopped item re-ran and saved once after 8.3 operator minutes and 763 s of re-qualification; 896 s from stop to done ([#1513](https://github.com/OpenAdaptAI/OpenAdapt/pull/1513)) | `openadapt_flow/repair/relabel.py`, [SELF_REPAIR.md](packages/flow/docs/SELF_REPAIR.md#a-relabeled-control-proof-that-only-the-label-changed) | +| A duplicate check that reads the app's match list (`no_duplicate_candidate`): stop only on a listed record with the item's last name or date of birth, fail closed on an unreadable or unloaded list | On main | Real app (local OpenEMR, synthetic data): false stops per 100 items 70.4 before, 16.4 after; 0 silent wrongs in 30 saves ([#1513](https://github.com/OpenAdaptAI/OpenAdapt/pull/1513)) | [DUPLICATE_CHECK.md](packages/flow/docs/DUPLICATE_CHECK.md) | +| A switched-off `problems_only` gate asks a person before the save instead of stopping the item | On main | Synthetic; real app in [#1513](https://github.com/OpenAdaptAI/OpenAdapt/pull/1513) | [APPROVAL_MODES.md](packages/flow/docs/APPROVAL_MODES.md#when-the-mode-turns-itself-off) | | Fleet drift patches: a verified fix with no patient data, shared across sites under policy | Released, local reference hub | Simulator | `openadapt-flow fleet`, [FLEET_DRIFT_PATCHES.md](packages/flow/docs/FLEET_DRIFT_PATCHES.md) | ## Computer-use agents diff --git a/llms-full.txt b/llms-full.txt index 02ba1a00d..acb6358d3 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -563,6 +563,7 @@ improvising or retrying when something is off. | Learn one program with branches from two or more recordings, or say which recordings are missing | Released | Synthetic | `openadapt-flow induce`, `disambiguate`; [CAPABILITIES.md (flow)](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/CAPABILITIES.md) | | Learn a guarded branch from several halts fixed the same way | Released | Synthetic | `openadapt-flow teach-branch`, `learning/branch_teach.py` | | Run a program over a worklist | Released | Real app | `openadapt-flow for-each` | +| Learn a worklist loop from recordings that search a record, open it, work it and go back to the list, even when the last row skips the return or rows differ (a field some rows lack, a write some rows get behind a screen check a person accepts). Each pass selects the row's own record by its values, verifies it, and checks that row's write. Certified and governed runs refuse it until qualification covers it | On main ([#1435](https://github.com/OpenAdaptAI/OpenAdapt/pull/1435)) | Synthetic (held-out rows run through the interpreter against an in-memory app) | `openadapt-flow induce`; `Step.target_param` | | Typed business decisions inside a run | Released | Synthetic | `business_decision` states, `openadapt-flow business-decisions` | | Watch many real runs in the background and mine candidate branches (shadow mode) | On main ([#1212](https://github.com/OpenAdaptAI/OpenAdapt/pull/1212)) | Synthetic | [SHADOW_MINING.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/capture/docs/SHADOW_MINING.md), [LEARN_FROM_MANY_RUNS.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/LEARN_FROM_MANY_RUNS.md) | | Fill a workflow's inputs from a referral PDF, fax image or email | Released | Synthetic | `openadapt-flow intake`, [INTAKE.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/INTAKE.md) | @@ -623,6 +624,9 @@ improvising or retrying when something is off. | Pre-approved repairs: a signed policy lists low-risk changes that apply without a person, a test-record run before each queue, and one approval for every item halted the same way | On main ([#1294](https://github.com/OpenAdaptAI/OpenAdapt/pull/1294)) | Synthetic | standard changes | | Repair a pinned bundle: the fix runs the pinned bundle's own qualification campaign on a test copy, keeps every write, effect, identity rule and record check, and is re-admitted with signed lineage (at most three repairs, no revoked key) that the pin verifies | On main | Synthetic | [SELF_REPAIR.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/SELF_REPAIR.md#a-pinned-bundle-re-qualify-and-re-admit) | | Repair by default: once a repair agent is configured, every stop that wrote nothing goes to it without anyone starting it; a stop after a possible write goes to reconciliation; every stop is a row in a ledger with the funnel from stop to resumed item | On main ([#1360](https://github.com/OpenAdaptAI/OpenAdapt/pull/1360)) | Synthetic (MockMed, 22 drift modes, real Claude agent: 6 of 21 repairable stops resumed and verified, 0 wrong saves in 66 runs); routing over recorded OpenEMR stops; live openIMIS relabel: 3 of 3 reached repair, 0 resumed; silent-saves 1.1.0 UI-change blocks: 18 of 28 items that always stopped now correct, 0 wrong | `run` (on by default, `--no-auto-repair`), `repair:` in `deploy.yaml`, [benchmark/auto_repair](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/benchmark/auto_repair/README.md) | +| Repair a relabeled save button: prove from the recorded identity hash that only the label changed (same control id, kind and form), re-qualify the fix on the new release, and have a person admit the certified copy afresh, since a save button's identity is part of the record check | On main | Real app (local OpenEMR, synthetic data): after the save button was relabeled, the stopped item re-ran and saved once after 8.3 operator minutes and 763 s of re-qualification; 896 s from stop to done ([#1513](https://github.com/OpenAdaptAI/OpenAdapt/pull/1513)) | `openadapt_flow/repair/relabel.py`, [SELF_REPAIR.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/SELF_REPAIR.md#a-relabeled-control-proof-that-only-the-label-changed) | +| A duplicate check that reads the app's match list (`no_duplicate_candidate`): stop only on a listed record with the item's last name or date of birth, fail closed on an unreadable or unloaded list | On main | Real app (local OpenEMR, synthetic data): false stops per 100 items 70.4 before, 16.4 after; 0 silent wrongs in 30 saves ([#1513](https://github.com/OpenAdaptAI/OpenAdapt/pull/1513)) | [DUPLICATE_CHECK.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/DUPLICATE_CHECK.md) | +| A switched-off `problems_only` gate asks a person before the save instead of stopping the item | On main | Synthetic; real app in [#1513](https://github.com/OpenAdaptAI/OpenAdapt/pull/1513) | [APPROVAL_MODES.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/APPROVAL_MODES.md#when-the-mode-turns-itself-off) | | Fleet drift patches: a verified fix with no patient data, shared across sites under policy | Released, local reference hub | Simulator | `openadapt-flow fleet`, [FLEET_DRIFT_PATCHES.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/FLEET_DRIFT_PATCHES.md) | ## Computer-use agents diff --git a/packages/flow/docs/APPROVAL_MODES.md b/packages/flow/docs/APPROVAL_MODES.md index 79c86b118..963612848 100644 --- a/packages/flow/docs/APPROVAL_MODES.md +++ b/packages/flow/docs/APPROVAL_MODES.md @@ -229,6 +229,10 @@ and a missing or broken log also reads as off. These thresholds are the public defaults. A deployment can only make them stricter. +While the switch reads off, each item waits for a person to approve it before +the save, as in `each_item`, when the run has an approval store. Without one, +the item stops before the save and nothing is saved. + ## Requirements `evaluate_requirements` in `openadapt_flow/approval_mode/requirements.py` diff --git a/packages/flow/docs/DUPLICATE_CHECK.md b/packages/flow/docs/DUPLICATE_CHECK.md new file mode 100644 index 000000000..02fffc8ae --- /dev/null +++ b/packages/flow/docs/DUPLICATE_CHECK.md @@ -0,0 +1,63 @@ +# A duplicate check that reads the match list + +Many registration forms run a duplicate search before they save. OpenEMR's New +Patient form, for example, opens a list of existing charts that might be the +same person. A recorded program usually expects the empty list ("No matches +were found."), so any listed chart stops the item before the save. + +OpenEMR lists a chart that shares only a first name with the new patient. On +the lifecycle soak's synthetic worklist, most of those stops were false: the +listed chart was clearly someone else, and a person entered the item by hand. + +## The postcondition + +`no_duplicate_candidate` replaces the "empty list" check on the step that +opens the search: + +```json +{ + "kind": "no_duplicate_candidate", + "text": "No matches were found.", + "candidate_rows": "tr.oneresult", + "candidate_frame_path": ["#modalframe"], + "candidate_params": ["lname", "DOB"], + "timeout_s": 10.0 +} +``` + +It passes only when the list has loaded and names no real candidate: + +- A listed row is a **candidate** when it contains every word of this run's + value of any `candidate_params` parameter (case-insensitive), or, for a + date, shows the same date in any format. With the example above, a chart + with the same last name or the same date of birth stops the item, even + under another last name (a married-name change). A chart that shares only + a first name doesn't. +- An **empty list** passes only when its "no matches" marker (`text`) is + shown, so a list that hasn't loaded yet never passes. +- It **fails closed**: an unreadable list (frame missing or ambiguous, or a + backend that can't read rows), a missing or empty parameter value, or any + candidate row all fail the step before anything is saved. + +The rule only lets an item continue past rows that share none of the identity +values. A same-name patient with a mistyped date of birth still stops, and a +person decides. The rows are read from the DOM (`element_texts` on the browser +backend), never from OCR. + +## Changing a program to use it + +The compiler doesn't propose this postcondition yet. Changing it is a program +change, so the program needs a fresh qualification and admission. The +lifecycle soak's `production_arm.with_duplicate_list_check` makes the change +on the OpenEMR referral program, and the soak admits that version afresh +(`"duplicate_check": true`). + +## Limits + +- The list must render its rows together, as OpenEMR's server-rendered list + does. A list that streams rows in one by one could pass on a partial list. +- A date (a value that is an ISO date) is compared as a date. Every date the + row shows is read the way intake reads dates (`Apr 20, 1944`, `20.04.1944`, + `04/20/1944`, ISO), and a row date that reads either way counts when either + reading matches. Other values are matched word for word, so name only + parameters the list shows as they were typed. diff --git a/packages/flow/docs/INTAKE.md b/packages/flow/docs/INTAKE.md index 9230d3304..9d21cd775 100644 --- a/packages/flow/docs/INTAKE.md +++ b/packages/flow/docs/INTAKE.md @@ -20,7 +20,18 @@ openadapt-flow run ./new-referral --params-file params.json `Insurance No:` and `Insurance Number:`. Add more labels with `--alias insurance_no="Member ID"`. 3. Validates each value against the parameter's type (`date`, `number`, - `boolean`, `enum`, `string`) and normalizes it (dates become `YYYY-MM-DD`). + `boolean`, `enum`, `string`) and normalizes it. Dates become `YYYY-MM-DD` + only when their order can be told: + - an ISO date, a spelled-out month (`25 Dec 1970`) or `YYYY/MM/DD`; + - a numeric date with a part over 12 (`12/25/1970`), or equal parts; + - a numeric date whose order you declare with + `--date-format DOB=MM/DD/YYYY` (or `DD/MM/YYYY`). + + A numeric date that could be read either way (`03/04/1970`) with no + declared format, or a date that doesn't exist (`1970-02-30`), needs a + person. A converted value records its original text and the conversion in + the provenance file (`raw_value`, `conversion`). The run checks again that + every `date` parameter is an ISO date before its first step. 4. Holds back any field that is missing, invalid, ambiguous (two different values), or below the confidence threshold (default 0.9). On a terminal it asks you to confirm or correct each one. Otherwise it prints the pending diff --git a/packages/flow/docs/PARAMETERS.md b/packages/flow/docs/PARAMETERS.md index 1ae1ef02b..703cb0350 100644 --- a/packages/flow/docs/PARAMETERS.md +++ b/packages/flow/docs/PARAMETERS.md @@ -28,6 +28,12 @@ one config. To fill parameters from a referral PDF, fax image, or email, see [`INTAKE.md`](INTAKE.md). +A parameter declared as a `date` (`ParamSpec.type`) must be a canonical ISO +date (`1990-03-14`) before a run starts. A date written another way, such as +`03/14/1990`, refuses the run before any step, so it is never typed into a +form and saved as whatever the app makes of it. Hosted dispatch applies the +same check to every typed parameter. + ## Choose an execution profile Select `--profile regulated` for encrypted, fail-closed production execution, diff --git a/packages/flow/docs/SELF_REPAIR.md b/packages/flow/docs/SELF_REPAIR.md index 543c3e86e..7e4521f74 100644 --- a/packages/flow/docs/SELF_REPAIR.md +++ b/packages/flow/docs/SELF_REPAIR.md @@ -404,6 +404,56 @@ self-repair runs without a named profile or under Demo, and the record check still runs. The Python API can run the Standard profile contract directly, as the tutorial does. +### A relabeled control: proof that only the label changed + +Self-repair never repairs a stop on a failed identity check, because a +renamed control usually can't be told from a different control. One case can +be told apart. When an app release renames a button, the ladder finds the same +control on the structural rung, and the run stops because the control's +structured identity (its id, kind, form and label) no longer matches. + +The recorded identity is a salted hash, but the bundle also keeps the recorded +label in plaintext (the step's intent and OCR text). +`openadapt_flow/repair/relabel.py` rebuilds the recorded identity from the +observed one with only the label swapped back. If that matches the recorded +hash, the id, kind and form are proven unchanged and only the label differs. +It refuses everything else: + +- a step that received input, or any earlier save that did (a step that may + have written is never retried); +- an identity check that failed outside the structured tier, or one that names + a record value rather than a control; +- a target found below the structural rung; +- an identity that doesn't match under any recorded label. + +The repair then follows the pinned-bundle path above: + +1. The candidate bundle changes only that step's identity hash, intent and + OCR text. +2. It is re-qualified with the parent's own campaign on the test copy, which + now runs the new release. Each re-qualification names its case runs + afresh, because a process refuses a run's authorization id once it has + been used, and the parent may have been qualified in the same process. +3. A person approves once, with the old and new label and the + re-qualification in front of them. +4. The certified copy runs next: + - For a control that is not part of the record check, it is re-admitted + with signed lineage, and the pin accepts it through the chain. + - A save button's identity is part of the record check, so no lineage hop + may carry it (`fixed_contract_refusal`). The person's approval is then a + fresh admission of the certified copy, and the deployment pins its new + digest. `readmit` returns `fresh_admission` and never re-pins by itself. + +The stopped item never wrote, so it runs again on the re-admitted version as a +retry; `problems_only` sends a retry to a person. Under a production +qualification admission or a managed dispatch the repair stays refused, as for +every other repair. + +On the local OpenEMR referral (proof-15's configuration), a relabeled save +button stopped every item until a person re-taught the step. The lifecycle +soak measures this path: see +the lifecycle soak (`benchmark/lifecycle`). + ## What is recorded | Where | What | diff --git a/packages/flow/openadapt_flow/approval_mode/gate.py b/packages/flow/openadapt_flow/approval_mode/gate.py index df3ec9a33..a081e698e 100644 --- a/packages/flow/openadapt_flow/approval_mode/gate.py +++ b/packages/flow/openadapt_flow/approval_mode/gate.py @@ -569,12 +569,33 @@ def _needs_approval(self) -> bool: if self.mode is ApprovalMode.EACH_ITEM: return True if self.mode is ApprovalMode.PROBLEMS_ONLY: - return self._item_fallback is not None or ( - self._decision is not None - and self._decision.kind is ItemDecisionKind.APPROVAL_REQUIRED + return ( + self._item_fallback is not None + or ( + self._decision is not None + and self._decision.kind is ItemDecisionKind.APPROVAL_REQUIRED + ) + or self._switch_reads_off() ) return False + def _switch_reads_off(self) -> bool: + """The local switch log already reads off (P1 would fail at the save). + + A switched-off mode waits for a person to approve the item, as in + ``each_item``. P1 runs at the save, after the last point a person can + be asked, so the ask happens here; without this, every item after the + switch turned off stopped at the save and nobody was asked. A missing + or unreadable log reads as off, as P1 does. + """ + + if self.record is None or self.log is None: + return True + try: + return bool(self.log.state().is_off) + except (SwitchLogError, OSError): + return True + def _before_step(self, ctx: StepContext) -> GateVerdict: if not self._began: raise RuntimeError("begin_item was not called for this run") diff --git a/packages/flow/openadapt_flow/backends/playwright_backend.py b/packages/flow/openadapt_flow/backends/playwright_backend.py index 8e95edacb..5778c218b 100644 --- a/packages/flow/openadapt_flow/backends/playwright_backend.py +++ b/packages/flow/openadapt_flow/backends/playwright_backend.py @@ -1691,6 +1691,26 @@ def focused_element_matches(self, selector: str) -> Optional[bool]: ) return result if isinstance(result, bool) else None + def element_texts( + self, selector: str, frame_path: tuple[str, ...] = () + ) -> Optional[list[str]]: + """The rendered text of every element matching ``selector``. + + Read in the frame ``frame_path`` names (outermost first). ``None`` + when the frame is missing or ambiguous, or the read fails; an empty + list means the frame is there and nothing matches. Read-only; never + raises. + """ + + try: + scope = self._resolve_scope(tuple(frame_path)) + if scope is None: + return None + texts = scope.locator(selector).all_inner_texts() + except Exception: # noqa: BLE001 - unreadable is None + return None + return [str(text) for text in texts] + def focused_select_label(self) -> Optional[str]: """The focused ``