Skip to content

problems_only switch policy: input problems leave it on, a person re-arms with one signed tap - #1517

Merged
abrichr merged 8 commits into
mainfrom
fix/switch-problem-classes
Oct 25, 2026
Merged

abrichr merged 8 commits into
mainfrom
fix/switch-problem-classes

Conversation

@abrichr

@abrichr abrichr commented Oct 24, 2026 •

Copy link
Copy Markdown
Member

This PR implements the coordinator's decision on the problems_only switch policy: classify every problem, and let a person re-arm only with a signed tap. It is stacked on OpenAdaptAI/OpenAdapt#1514.

Why: in the measured soak (OpenAdaptAI/OpenAdapt#1513), one caught problem turned the switch off for the rest of the run. Every later save then needed an approval: 25 of 125.5 operator minutes.

1. Which problems turn it off (post.problem_class, classify_decision)

Item-input problems leave the switch on. The item goes to a person and doesn't count toward too_many_problems. These are problems caught before anything was written:

Program and environment problems turn it off at once. For example:

  • a wrong write the record check caught;
  • a failed identity check;
  • a postcondition that failed after delivery;
  • a changed screen;
  • the wrong signed-in user;
  • any unnamed cause.

Any outcome that doesn't consistently read "nothing was written" is a program problem (fail closed). A switch-off the gate, R2 or the engine already named is never classed down to an input problem. Gate pre-check failures map to closed off reasons: signed_in_user → environment_changed, right_record and same_screens → screen_changed, record_check_ready → record_check_lost. too_many_problems stays as a backstop.

2. A person re-arms with one signed tap (switch.rearm_request, sign_rearm, rearm)

  • Never automatic, under any profile.
  • Offered only after 10 consecutive verified items since the switch went off. These are no_person or approved items; any other finished item resets the count.
  • What the person's approver key signs: the off event and its reason, the recorded cause, and those 10 run refs.
  • What the log records: a rearmed event with who signed, the key, and what they saw. The field is left out of every other event, so existing log lines and hashes are unchanged.
  • Who honours it: a gate that trusts the signing key. By default, and deliberately, those are the approvers registered in the JIT key directory's principal registry. A person registers each at a terminal (jit register --role approver, fix: a person's approval needs a registered approver key's signature #1486), and the registry refuses agent and model names. ApprovalGate(pinned_approver_keys=...) names keys explicitly instead. Keys may be hex (the registry's form) or base64.
  • Binding: a re-arm's approver_id must equal its key_id (the key's registered name).
  • Only a verified re-arm ends an off period. Every reader checks the re-arm again against the log before it, so a forged, tampered, replayed or early re-arm appended to the log is ignored and doesn't reset the count.
  • Signing outside Python: rearm_to_sign returns the exact bytes, and rearm_evidence wraps a signature made elsewhere. Desktop's Touch ID card (feat(desktop): 0.24.0, the Touch ID approver key and one-tap stop review #1519) uses these.
  • Before fix: a person's approval needs a registered approver key's signature #1486 lands, main's registry has no approvers, so no re-arm counts.

3. A bug the soak exposed

_unapproved_delivered_saves classified saves without the certifying policy, as the gate's own save check did before #1455. On OpenEMR, "Create New Patient" (certified as not saving) counted as an unapproved delivered save. So after the switch turned off, every approved, verified item was logged as a problem (save_not_approved), and it could never have counted toward a re-arm.

Tests

  • New test_approval_mode_switch_policy.py (23). Covers:
    • review cases: an explicit switch-off with an input-class halt stays off; a duplicate stop counts as input only from its structured outcome, and an unreadable one turns it off; a replayed old re-arm after a second switch-off; a re-arm for another switch log; a forged re-arm, then a real one; a mismatched approver; the gate reading on after a valid re-arm, and a gate that doesn't trust the key reading off;
    • registry keys in hex; an unreadable registry gives no keys; signing with a key that never leaves its keychain;
    • an invalid date, a missing value and a duplicate candidate leave the switch on;
    • a caught wrong write turns it off (uncertain_result);
    • an unnamed stop before any effect turns it off;
    • a duplicate stop after a delivered save, or an inconsistent outcome, is a program problem;
    • re-arm: signed after 10 verified items turns it on;
    • refused before 10, or after a problem resets the count;
    • refused unsigned, with an unpinned key, tampered, or forged;
    • fifteen clean items alone change nothing;
    • a later problem turns it off again;
    • a forged rearmed event appended to the log is ignored.
  • Old-policy tests updated (4). They expected a counted problem without a switch-off, or three in a row before one.
  • Locally: every test that touches the switch log or approval modes passed (797 in 32 files); after the review fixes, the approval-mode suites pass (360), and ruff and mypy pass.

🤖 Generated with Claude Code

abrichr and others added 3 commits October 24, 2026 19:46
…program problems turn it off, a person re-arms with one signed tap

- post.problem_class sorts every problem. Item-input problems caught before
  anything was written (a missing or invalid value refused before the first
  step, an input item check, a real duplicate candidate) send the item to a
  person and leave the switch on. Program and environment problems, and any
  unnamed cause, turn it off at once.
- A switched-off switch never re-arms itself. After 10 consecutive verified
  items, a person's approver key signs a re-arm over the off event, its
  cause and those items; the log records it, and a gate honours it only with
  the key pinned. A forged, tampered or early re-arm is ignored.
- The gate's count of unapproved delivered saves now uses the certifying
  policy, as its own save check does. A step the certified qualification
  classified as not saving made every approved, verified item a problem.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…arm_to_sign, rearm_evidence)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
abrichr added a commit that referenced this pull request Oct 25, 2026
- The re-arm card shows Flow's re-arm request: why the switch went off,
  when, and the 10 items saved and checked in a row since, named. Its id
  covers the request, so a re-arm signs only what the person saw.
- Answering it is one Touch ID over Flow's own bytes (rearm_to_sign); Flow
  verifies against the approvers registry, never a key Desktop supplies.
  Desktop's switch view honours a re-arm by a registered approver.
- Feature-detected: the card appears only with a Flow that has signed
  re-arm (#1517). Checked against #1517's real switch log.
- A locked Mac no longer reads as having no Secure Enclave: the check isn't
  remembered, and the refusal says to unlock the Mac.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
abrichr and others added 3 commits October 24, 2026 20:21
…s by default

The run gate reads the approver keys registered in the JIT key directory's
principal registry (hex, as jit register --role approver writes them), so a
re-arm signed with Desktop's approver key counts at run time with no extra
configuration. pinned_approver_keys still names keys explicitly. An
unreadable registry, or one with no approvers, gives no keys, and a
switched-off switch stays off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- An explicit switch-off from the gate, R2 or the engine is never classed
  down to an input problem.
- A duplicate stop counts as an input problem only from the duplicate
  check's structured outcome (StepResult.duplicate_check == 'candidate',
  set by the record-channel check, #1516), never from the error text. An
  unreadable list or channel turns the switch off (record_check_lost); a
  duplicate stop that doesn't say is a program problem.
- A re-arm's approver must be its key's registered name.
- Only a re-arm that verifies ends an off period, so a forged event can't
  hide a switch-off or reset the count.
- Trusting the registered approvers by default is documented as deliberate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…uplicate candidates

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
abrichr added a commit that referenced this pull request Oct 25, 2026
StepResult.duplicate_check is set whenever no_duplicate_candidate fails:
exact or close (a record matched) and candidate (a listed row matched) are
about the item; unreadable (the record channel or the screen list could not
answer, or a record-pinned check had no channel) is about the program. None
otherwise and then omitted, so existing reports serialize unchanged. #1517
classes the problems_only switch from it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the pinned root

#1486 moved approvers out of principals.json into a signed roster. The
gate's default approver keys are now the roster entries the run's pinned
root vouches for (approver_keys_b64, pinned_root), imported by name so this
works before and after #1486 merges; with no roster, no pinned root, or a
roster that doesn't verify, no re-arm counts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Base automatically changed from fix/relabel-repair-duplicate-list to main October 25, 2026 05:41
@abrichr
abrichr enabled auto-merge October 25, 2026 05:41
@abrichr
abrichr merged commit eb1e285 into main Oct 25, 2026
40 checks passed
@abrichr
abrichr deleted the fix/switch-problem-classes branch October 25, 2026 07:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant