Repository navigation
fix(flow): the census follows the target's teach-time role on every rung - #1557
Conversation
#1504 exempted a teach-time form field or control from the candidate census only when the structured tier verified it. On OpenEMR, step_016 (the "Contact" section button, recorded with structural role "button") is verified by its pixel crop. The OCR census found no band to count, a save follows, and the census turned the verified match into identity_uncounted. The closed-loop scroll before it read that as "anchor not in view" and spent its whole budget, so the production program could not be admitted (2 of 2 attempts). - identity_census.teach_time_role: "row" when the recorder saw a record row (structured_row True), "control" when it saw none (structured_row False) or the structural locator names a form-field or control role (textbox, combobox, checkbox, radio, spinbutton, switch, slider, button, tab, menuitem), otherwise "unknown". - The census treats a "control" as not_a_row on every rung. "row" and "unknown" are still counted, so a link, cell or list item, or a target with no recorded role, still halts when it can't be counted. - Closed-loop scroll: when the next target is in view but the census refuses it, the scroll stops and says why (safety halt, no input), instead of scrolling until the budget runs out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Real-app check, run by the production-lifecycle session: the OpenEMR representative admission case is VERIFIED with the census on (exact_record read-back, 0 model calls, 108 s engine). It was tested at 76ad93efa: measure/production-lifecycle-final (origin/main 6dc0be4 + #1516 + harness) merged with this branch's 39daa43. Before the fix, the same case halted at step_015 in 2 of 2 attempts. Only the representative case ran, not the 5 fault cases. |
Review fix for #1557. The teach-time control role now only replaces a cannot_count verdict; an OCR census that can count still decides. A recorded structured identity with no teach-time census (a backend without one, a legacy bundle) is unknown, not a control. Without this, a per-row button (role button) with OCR look-alike rows was clicked, and a table-row Edit button taught and replayed on a backend without a census signed the swapped look-alike chart. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
AI review (coordinator) Verdict: this reopens the wrong-row bug on two paths. I turned off auto-merge. A fix with regression tests is on branch How teach time decides
|
| role | base (merge-base) | this PR |
|---|---|---|
| none | halts, identical_candidates |
halts |
| button / menuitem / tab | halts, identical_candidates |
clicks, census_status=not_a_row |
Hole 2: a structured row identity with no teach-time census becomes a control
Real headless Chromium. The page has two identical <tr> rows, each with <button id="edit-N">Edit</button>, where the id follows the row's position. I taught "Edit on chart 1001, then Sign" and replayed with the rows swapped, using a backend without a census (_CannotCountBackend, which stands in for native and remote surfaces) for both teach and replay.
- Teach recorded
structured_identity='John Smith Follow-up Dr Lee'(that is row text),structured_row=Noneand rolebutton, soteach_time_rolereturnedcontrol. - Base: the step halts with
identity_uncounted. - This PR:
success=True, statusOpened chart 1002,Signed chart 1002,census_status=not_a_row. That is the wrong record.
What still holds
- On Playwright, a
<tr>with a button inside getsstructured_row=True, so the role isrowand the step halts (identical_candidates). This PR doesn't change that. - The role+name structural locator refuses div rows whose buttons all have the same name ("Open" x3, candidate_count=3). The census isn't involved there.
Not caused by this PR, but open
Div rows whose buttons have positional ids (#open-0) sign the wrong chart on base and on this PR. No identity is armed for that step, and the DOM census can't see div rows, so #1504 never covered them. This needs its own issue: count repeated sibling structures, not only tr/li/role=row.
Fix (on the branch)
--- identity_census.py teach_time_role
if recorded_row is False:
return "control"
+ if getattr(template, "structured", None):
+ # A structured identity was recorded but nothing counted its rows (a
+ # backend without a census, a legacy bundle): it may be a row's text.
+ return "unknown"
--- replayer.py _apply_candidate_census
if check.mode == "structured":
decision = self._structured_census(step, resolution.point)
- if decision.verdict == "cannot_count" and teach_role == "control":
- decision = census_mod.CensusDecision("not_a_row")
- elif teach_role == "control":
- decision = census_mod.CensusDecision("not_a_row")
else:
decision = self._ocr_census(step, resolution, before_png, params, workflow)
+ if decision.verdict == "cannot_count" and teach_role == "control":
+ # A census that can count always decides.
+ decision = census_mod.CensusDecision("not_a_row")The fix follows one rule: a control role can only replace cannot_count, and a census that can count always decides. The OpenEMR step_016 case still passes. Its pixel-only _ocr_census returns cannot_count (no band), and the role then turns that into not_a_row. All 13 of this PR's tests pass with the fix.
Tests on the branch
New tests:
test_a_control_role_never_skips_an_ocr_census_that_can_count(button, menuitem, tab, checkbox)test_a_recorded_structured_identity_with_no_teach_census_is_not_a_controltest_realism_browser.py::test_a_row_button_taught_without_a_census_halts_after_a_swap(real Chromium)
Results:
- On this PR's head, all 6 new tests fail.
- With the fix,
test_census_teach_role.py,test_realism_browser.pyandtest_identity_census.pypass, and ruff is clean.
If you merge the branch into this PR, re-enable auto-merge.
…nsus that can count Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in #1504, #1557 and the reviewer's control-role fix. In _apply_candidate_census, the pixel census still runs when a pixel or VLM identity has no OCR band; a teach-time control role then replaces only a cannot_count. A census that can count, pixel included, always decides. The #1557 tests for the no-census branch now make the pixel census unavailable (crop-only steps can otherwise be counted by pixels). A new test checks that the pixel census decides for a control: the crop found once verifies, found twice halts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fixes the admission block in the production lifecycle run (
packages/flow/benchmark/lifecycle/results/production-2026-10-25-final-blocked/README.md, commit 84ce8bd onmeasure/production-lifecycle-final).Before
#1504 skips the candidate census for a target that teach time recorded as a form field or control, but only when the structured tier verified the step. On OpenEMR, step_016 is the "Contact" section button (structural role
button), and its pixel crop verifies it. The OCR census had no band to count, and a save follows, so the census turned the verified match intoidentity_uncounted. The closed-loop scroll before it (step_015) read that as "anchor not in view" and spent its whole budget. The representative case halted in 2 of 2 attempts, so the production program couldn't be admitted. It admitted at 38fc21a.After
identity_census.teach_time_role(step)says what teach time recorded:rowwhen the recorder saw a record row (structured_rowtrue);controlwhen it saw none (structured_rowfalse), or when the structural locator names a form-field or control role (textbox, searchbox, combobox, checkbox, radio, spinbutton, switch, slider, button, tab, menuitem);unknownotherwise.controlas not a row on every rung (structured, template, OCR, pixel), so a verified identity stands.rowandunknownare still counted, so a pixel-resolved link, cell, list item or row, or a target with no recorded role, still halts when it can't be counted. A teach-time row wins over a control role (a row's own button is still counted).Tests
tests/test_census_teach_role.py:census_status="not_a_row").identity_uncounted.identical_candidates.Locally (1 worker): ruff, mypy (including strict), and 13 test files, 376 passed.
I haven't rerun OpenEMR here; that's the lifecycle run's job.
🤖 Generated with Claude Code