Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CAPABILITIES.md
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,9 @@ improvising or retrying when something is off.
| Pre-approved repairs: a signed policy lists low-risk changes that apply without a person, a test-record run before each queue, and one approval for every item halted the same way | On main ([#1294](https://github.com/OpenAdaptAI/OpenAdapt/pull/1294)) | Synthetic | standard changes |
| Repair a pinned bundle: the fix runs the pinned bundle's own qualification campaign on a test copy, keeps every write, effect, identity rule and record check, and is re-admitted with signed lineage (at most three repairs, no revoked key) that the pin verifies | On main | Synthetic | [SELF_REPAIR.md](packages/flow/docs/SELF_REPAIR.md#a-pinned-bundle-re-qualify-and-re-admit) |
| Repair by default: once a repair agent is configured, every stop that wrote nothing goes to it without anyone starting it; a stop after a possible write goes to reconciliation; every stop is a row in a ledger with the funnel from stop to resumed item | On main ([#1360](https://github.com/OpenAdaptAI/OpenAdapt/pull/1360)) | Synthetic (MockMed, 22 drift modes, real Claude agent: 6 of 21 repairable stops resumed and verified, 0 wrong saves in 66 runs); routing over recorded OpenEMR stops; live openIMIS relabel: 3 of 3 reached repair, 0 resumed; silent-saves 1.1.0 UI-change blocks: 18 of 28 items that always stopped now correct, 0 wrong | `run` (on by default, `--no-auto-repair`), `repair:` in `deploy.yaml`, [benchmark/auto_repair](packages/flow/benchmark/auto_repair/README.md) |
| Repair a relabeled save button: prove from the recorded identity hash that only the label changed (same control id, kind and form), re-qualify the fix on the new release, and have a person admit the certified copy afresh, since a save button's identity is part of the record check | On main | Real app (local OpenEMR, synthetic data): after the save button was relabeled, the stopped item re-ran and saved once after 8.3 operator minutes and 763 s of re-qualification; 896 s from stop to done ([#1513](https://github.com/OpenAdaptAI/OpenAdapt/pull/1513)) | `openadapt_flow/repair/relabel.py`, [SELF_REPAIR.md](packages/flow/docs/SELF_REPAIR.md#a-relabeled-control-proof-that-only-the-label-changed) |
| A duplicate check that reads the app's match list (`no_duplicate_candidate`): stop only on a listed record with the item's last name or date of birth, fail closed on an unreadable or unloaded list | On main | Real app (local OpenEMR, synthetic data): false stops per 100 items 70.4 before, 16.4 after; 0 silent wrongs in 30 saves ([#1513](https://github.com/OpenAdaptAI/OpenAdapt/pull/1513)) | [DUPLICATE_CHECK.md](packages/flow/docs/DUPLICATE_CHECK.md) |
| A switched-off `problems_only` gate asks a person before the save instead of stopping the item | On main | Synthetic; real app in [#1513](https://github.com/OpenAdaptAI/OpenAdapt/pull/1513) | [APPROVAL_MODES.md](packages/flow/docs/APPROVAL_MODES.md#when-the-mode-turns-itself-off) |
| Fleet drift patches: a verified fix with no patient data, shared across sites under policy | Released, local reference hub | Simulator | `openadapt-flow fleet`, [FLEET_DRIFT_PATCHES.md](packages/flow/docs/FLEET_DRIFT_PATCHES.md) |

## Computer-use agents
Expand Down
4 changes: 4 additions & 0 deletions llms-full.txt
Original file line number Diff line number Diff line change
Expand Up @@ -563,6 +563,7 @@ improvising or retrying when something is off.
| Learn one program with branches from two or more recordings, or say which recordings are missing | Released | Synthetic | `openadapt-flow induce`, `disambiguate`; [CAPABILITIES.md (flow)](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/CAPABILITIES.md) |
| Learn a guarded branch from several halts fixed the same way | Released | Synthetic | `openadapt-flow teach-branch`, `learning/branch_teach.py` |
| Run a program over a worklist | Released | Real app | `openadapt-flow for-each` |
| Learn a worklist loop from recordings that search a record, open it, work it and go back to the list, even when the last row skips the return or rows differ (a field some rows lack, a write some rows get behind a screen check a person accepts). Each pass selects the row's own record by its values, verifies it, and checks that row's write. Certified and governed runs refuse it until qualification covers it | On main ([#1435](https://github.com/OpenAdaptAI/OpenAdapt/pull/1435)) | Synthetic (held-out rows run through the interpreter against an in-memory app) | `openadapt-flow induce`; `Step.target_param` |
| Typed business decisions inside a run | Released | Synthetic | `business_decision` states, `openadapt-flow business-decisions` |
| Watch many real runs in the background and mine candidate branches (shadow mode) | On main ([#1212](https://github.com/OpenAdaptAI/OpenAdapt/pull/1212)) | Synthetic | [SHADOW_MINING.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/capture/docs/SHADOW_MINING.md), [LEARN_FROM_MANY_RUNS.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/LEARN_FROM_MANY_RUNS.md) |
| Fill a workflow's inputs from a referral PDF, fax image or email | Released | Synthetic | `openadapt-flow intake`, [INTAKE.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/INTAKE.md) |
Expand Down Expand Up @@ -623,6 +624,9 @@ improvising or retrying when something is off.
| Pre-approved repairs: a signed policy lists low-risk changes that apply without a person, a test-record run before each queue, and one approval for every item halted the same way | On main ([#1294](https://github.com/OpenAdaptAI/OpenAdapt/pull/1294)) | Synthetic | standard changes |
| Repair a pinned bundle: the fix runs the pinned bundle's own qualification campaign on a test copy, keeps every write, effect, identity rule and record check, and is re-admitted with signed lineage (at most three repairs, no revoked key) that the pin verifies | On main | Synthetic | [SELF_REPAIR.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/SELF_REPAIR.md#a-pinned-bundle-re-qualify-and-re-admit) |
| Repair by default: once a repair agent is configured, every stop that wrote nothing goes to it without anyone starting it; a stop after a possible write goes to reconciliation; every stop is a row in a ledger with the funnel from stop to resumed item | On main ([#1360](https://github.com/OpenAdaptAI/OpenAdapt/pull/1360)) | Synthetic (MockMed, 22 drift modes, real Claude agent: 6 of 21 repairable stops resumed and verified, 0 wrong saves in 66 runs); routing over recorded OpenEMR stops; live openIMIS relabel: 3 of 3 reached repair, 0 resumed; silent-saves 1.1.0 UI-change blocks: 18 of 28 items that always stopped now correct, 0 wrong | `run` (on by default, `--no-auto-repair`), `repair:` in `deploy.yaml`, [benchmark/auto_repair](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/benchmark/auto_repair/README.md) |
| Repair a relabeled save button: prove from the recorded identity hash that only the label changed (same control id, kind and form), re-qualify the fix on the new release, and have a person admit the certified copy afresh, since a save button's identity is part of the record check | On main | Real app (local OpenEMR, synthetic data): after the save button was relabeled, the stopped item re-ran and saved once after 8.3 operator minutes and 763 s of re-qualification; 896 s from stop to done ([#1513](https://github.com/OpenAdaptAI/OpenAdapt/pull/1513)) | `openadapt_flow/repair/relabel.py`, [SELF_REPAIR.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/SELF_REPAIR.md#a-relabeled-control-proof-that-only-the-label-changed) |
| A duplicate check that reads the app's match list (`no_duplicate_candidate`): stop only on a listed record with the item's last name or date of birth, fail closed on an unreadable or unloaded list | On main | Real app (local OpenEMR, synthetic data): false stops per 100 items 70.4 before, 16.4 after; 0 silent wrongs in 30 saves ([#1513](https://github.com/OpenAdaptAI/OpenAdapt/pull/1513)) | [DUPLICATE_CHECK.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/DUPLICATE_CHECK.md) |
| A switched-off `problems_only` gate asks a person before the save instead of stopping the item | On main | Synthetic; real app in [#1513](https://github.com/OpenAdaptAI/OpenAdapt/pull/1513) | [APPROVAL_MODES.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/APPROVAL_MODES.md#when-the-mode-turns-itself-off) |
| Fleet drift patches: a verified fix with no patient data, shared across sites under policy | Released, local reference hub | Simulator | `openadapt-flow fleet`, [FLEET_DRIFT_PATCHES.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/FLEET_DRIFT_PATCHES.md) |

## Computer-use agents
Expand Down
4 changes: 4 additions & 0 deletions packages/flow/docs/APPROVAL_MODES.md
Original file line number Diff line number Diff line change
Expand Up @@ -229,6 +229,10 @@ and a missing or broken log also reads as off.
These thresholds are the public defaults. A deployment can only make them
stricter.

While the switch reads off, each item waits for a person to approve it before
the save, as in `each_item`, when the run has an approval store. Without one,
the item stops before the save and nothing is saved.

## Requirements

`evaluate_requirements` in `openadapt_flow/approval_mode/requirements.py`
Expand Down
63 changes: 63 additions & 0 deletions packages/flow/docs/DUPLICATE_CHECK.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# A duplicate check that reads the match list

Many registration forms run a duplicate search before they save. OpenEMR's New
Patient form, for example, opens a list of existing charts that might be the
same person. A recorded program usually expects the empty list ("No matches
were found."), so any listed chart stops the item before the save.

OpenEMR lists a chart that shares only a first name with the new patient. On
the lifecycle soak's synthetic worklist, most of those stops were false: the
listed chart was clearly someone else, and a person entered the item by hand.

## The postcondition

`no_duplicate_candidate` replaces the "empty list" check on the step that
opens the search:

```json
{
"kind": "no_duplicate_candidate",
"text": "No matches were found.",
"candidate_rows": "tr.oneresult",
"candidate_frame_path": ["#modalframe"],
"candidate_params": ["lname", "DOB"],
"timeout_s": 10.0
}
```

It passes only when the list has loaded and names no real candidate:

- A listed row is a **candidate** when it contains every word of this run's
value of any `candidate_params` parameter (case-insensitive), or, for a
date, shows the same date in any format. With the example above, a chart
with the same last name or the same date of birth stops the item, even
under another last name (a married-name change). A chart that shares only
a first name doesn't.
- An **empty list** passes only when its "no matches" marker (`text`) is
shown, so a list that hasn't loaded yet never passes.
- It **fails closed**: an unreadable list (frame missing or ambiguous, or a
backend that can't read rows), a missing or empty parameter value, or any
candidate row all fail the step before anything is saved.

The rule only lets an item continue past rows that share none of the identity
values. A same-name patient with a mistyped date of birth still stops, and a
person decides. The rows are read from the DOM (`element_texts` on the browser
backend), never from OCR.

## Changing a program to use it

The compiler doesn't propose this postcondition yet. Changing it is a program
change, so the program needs a fresh qualification and admission. The
lifecycle soak's `production_arm.with_duplicate_list_check` makes the change
on the OpenEMR referral program, and the soak admits that version afresh
(`"duplicate_check": true`).

## Limits

- The list must render its rows together, as OpenEMR's server-rendered list
does. A list that streams rows in one by one could pass on a partial list.
- A date (a value that is an ISO date) is compared as a date. Every date the
row shows is read the way intake reads dates (`Apr 20, 1944`, `20.04.1944`,
`04/20/1944`, ISO), and a row date that reads either way counts when either
reading matches. Other values are matched word for word, so name only
parameters the list shows as they were typed.
13 changes: 12 additions & 1 deletion packages/flow/docs/INTAKE.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,18 @@ openadapt-flow run ./new-referral --params-file params.json
`Insurance No:` and `Insurance Number:`. Add more labels with
`--alias insurance_no="Member ID"`.
3. Validates each value against the parameter's type (`date`, `number`,
`boolean`, `enum`, `string`) and normalizes it (dates become `YYYY-MM-DD`).
`boolean`, `enum`, `string`) and normalizes it. Dates become `YYYY-MM-DD`
only when their order can be told:
- an ISO date, a spelled-out month (`25 Dec 1970`) or `YYYY/MM/DD`;
- a numeric date with a part over 12 (`12/25/1970`), or equal parts;
- a numeric date whose order you declare with
`--date-format DOB=MM/DD/YYYY` (or `DD/MM/YYYY`).

A numeric date that could be read either way (`03/04/1970`) with no
declared format, or a date that doesn't exist (`1970-02-30`), needs a
person. A converted value records its original text and the conversion in
the provenance file (`raw_value`, `conversion`). The run checks again that
every `date` parameter is an ISO date before its first step.
4. Holds back any field that is missing, invalid, ambiguous (two different
values), or below the confidence threshold (default 0.9). On a terminal it
asks you to confirm or correct each one. Otherwise it prints the pending
Expand Down
6 changes: 6 additions & 0 deletions packages/flow/docs/PARAMETERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,12 @@ one config.
To fill parameters from a referral PDF, fax image, or email, see
[`INTAKE.md`](INTAKE.md).

A parameter declared as a `date` (`ParamSpec.type`) must be a canonical ISO
date (`1990-03-14`) before a run starts. A date written another way, such as
`03/14/1990`, refuses the run before any step, so it is never typed into a
form and saved as whatever the app makes of it. Hosted dispatch applies the
same check to every typed parameter.

## Choose an execution profile

Select `--profile regulated` for encrypted, fail-closed production execution,
Expand Down
50 changes: 50 additions & 0 deletions packages/flow/docs/SELF_REPAIR.md
Original file line number Diff line number Diff line change
Expand Up @@ -404,6 +404,56 @@ self-repair runs without a named profile or under Demo, and the record check
still runs. The Python API can run the Standard profile contract directly, as
the tutorial does.

### A relabeled control: proof that only the label changed

Self-repair never repairs a stop on a failed identity check, because a
renamed control usually can't be told from a different control. One case can
be told apart. When an app release renames a button, the ladder finds the same
control on the structural rung, and the run stops because the control's
structured identity (its id, kind, form and label) no longer matches.

The recorded identity is a salted hash, but the bundle also keeps the recorded
label in plaintext (the step's intent and OCR text).
`openadapt_flow/repair/relabel.py` rebuilds the recorded identity from the
observed one with only the label swapped back. If that matches the recorded
hash, the id, kind and form are proven unchanged and only the label differs.
It refuses everything else:

- a step that received input, or any earlier save that did (a step that may
have written is never retried);
- an identity check that failed outside the structured tier, or one that names
a record value rather than a control;
- a target found below the structural rung;
- an identity that doesn't match under any recorded label.

The repair then follows the pinned-bundle path above:

1. The candidate bundle changes only that step's identity hash, intent and
OCR text.
2. It is re-qualified with the parent's own campaign on the test copy, which
now runs the new release. Each re-qualification names its case runs
afresh, because a process refuses a run's authorization id once it has
been used, and the parent may have been qualified in the same process.
3. A person approves once, with the old and new label and the
re-qualification in front of them.
4. The certified copy runs next:
- For a control that is not part of the record check, it is re-admitted
with signed lineage, and the pin accepts it through the chain.
- A save button's identity is part of the record check, so no lineage hop
may carry it (`fixed_contract_refusal`). The person's approval is then a
fresh admission of the certified copy, and the deployment pins its new
digest. `readmit` returns `fresh_admission` and never re-pins by itself.

The stopped item never wrote, so it runs again on the re-admitted version as a
retry; `problems_only` sends a retry to a person. Under a production
qualification admission or a managed dispatch the repair stays refused, as for
every other repair.

On the local OpenEMR referral (proof-15's configuration), a relabeled save
button stopped every item until a person re-taught the step. The lifecycle
soak measures this path: see
the lifecycle soak (`benchmark/lifecycle`).

## What is recorded

| Where | What |
Expand Down
27 changes: 24 additions & 3 deletions packages/flow/openadapt_flow/approval_mode/gate.py
Original file line number Diff line number Diff line change
Expand Up @@ -569,12 +569,33 @@ def _needs_approval(self) -> bool:
if self.mode is ApprovalMode.EACH_ITEM:
return True
if self.mode is ApprovalMode.PROBLEMS_ONLY:
return self._item_fallback is not None or (
self._decision is not None
and self._decision.kind is ItemDecisionKind.APPROVAL_REQUIRED
return (
self._item_fallback is not None
or (
self._decision is not None
and self._decision.kind is ItemDecisionKind.APPROVAL_REQUIRED
)
or self._switch_reads_off()
)
return False

def _switch_reads_off(self) -> bool:
"""The local switch log already reads off (P1 would fail at the save).

A switched-off mode waits for a person to approve the item, as in
``each_item``. P1 runs at the save, after the last point a person can
be asked, so the ask happens here; without this, every item after the
switch turned off stopped at the save and nobody was asked. A missing
or unreadable log reads as off, as P1 does.
"""

if self.record is None or self.log is None:
return True
try:
return bool(self.log.state().is_off)
except (SwitchLogError, OSError):
return True

def _before_step(self, ctx: StepContext) -> GateVerdict:
if not self._began:
raise RuntimeError("begin_item was not called for this run")
Expand Down
20 changes: 20 additions & 0 deletions packages/flow/openadapt_flow/backends/playwright_backend.py
Original file line number Diff line number Diff line change
Expand Up @@ -1691,6 +1691,26 @@ def focused_element_matches(self, selector: str) -> Optional[bool]:
)
return result if isinstance(result, bool) else None

def element_texts(
self, selector: str, frame_path: tuple[str, ...] = ()
) -> Optional[list[str]]:
"""The rendered text of every element matching ``selector``.

Read in the frame ``frame_path`` names (outermost first). ``None``
when the frame is missing or ambiguous, or the read fails; an empty
list means the frame is there and nothing matches. Read-only; never
raises.
"""

try:
scope = self._resolve_scope(tuple(frame_path))
if scope is None:
return None
texts = scope.locator(selector).all_inner_texts()
except Exception: # noqa: BLE001 - unreadable is None
return None
return [str(text) for text in texts]

def focused_select_label(self) -> Optional[str]:
"""The focused ``<select>``'s selected option label, or None."""

Expand Down
Loading
Loading