Skip to content

L08-05 — Calibrate task and stratum abstention #65

Open

Description

@mberrys

Outcome

Measure error, utility, confidence, and support per question family, per head (operation vs target), and per relevant print/customer stratum. Map these to preregistered thresholds that decide when the operation head may yield anything other than ABSTAIN.

Parent and boundary

Parent: #9. Owner: proposed Decision module. This issue owns one bounded slice of Bounded Decision Worker; consume accepted upstream contracts and keep PDF/verdict/publication authority with Core.

Acceptance

  • Thresholds are preregistered per task, head, and stratum. Unsupported slices abstain with a reason.
  • Calibration covers the selected head's probability, not just the reported confidence scalar. Target-head reliability is measured only on cases where its operation was selected.
  • Repeated no-progress decisions within a session (jev: three unchanged steps → BLOCKED) abstain rather than retry.
  • Failure case: one universal probability, an argmax that clears validation, or incomplete context cannot claim safety.

Required proof

Calibration reliability, selective-risk, and OOD test packet, split by head and stratum. Record the exact source SHA, test command or hosted run, fixture/artifact identity, and any skipped or unavailable lane. Run the repository's mapped proof and check-change gate for a PR.

Handoff

Record changed files, observed result, remaining risk, and a 1–3 sentence anti-slop review. Completion means acceptance evidence, not merely a merged implementation.

Sources

Approved master roadmap · North Star: jev-ultrafast@1231850a; see the translation table in #9.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions