fix: parse provider JSON after unmatched prose

[goutamadwant]

Fixes #237.

Provider responses containing a lone opening brace in leading prose previously hid an otherwise valid structured result. The shared extractor now retains balanced-object priority and nested-draft exclusion while recovering valid JSON beneath an unmatched prose brace. The provider adapters still validate the recovered object against their schemas.

Added regression coverage, provider documentation, and an Unreleased entry crediting @goutamadwant and @notlakshitha.

Validation on head fb3a8c0:

• Both reported inputs return null on main and the expected objects with this change.
• Independent P0–P2 review: no actionable findings.
• Hetzner Crabbox: frozen install, typecheck, lint, formatting, coverage (970 passed, 2 skipped), build, and package smoke passed.
• GitHub CI is green, including Windows execution and Node 22/24 runtime checks: https://github.com/openclaw/clawpatch/actions/runs/39314280709.

Shipped CLI/provider proof also passed on the same built package. A deterministic executable fixture writes Plan: wrap keys in { braces. followed by a schema-valid review object through Codex's normal --output-last-message path. The old parser and the fixed parser consume the same response:

baseline CLI review exit=8
error: codex provider produced unparseable JSON output

fixed CLI review exit=0
reviewed: 1
findings: 0
jobs: 1

This exercises the built CLI, provider adapter, shared extractor, schema validation, and workflow. The provider is a synthetic executable fixture; no live provider behavior or credentials are claimed.

[clawsweeper]

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

[clawsweeper]

Codex review: needs real behavior proof before merge.

What this changes

The PR recovers valid provider JSON following unmatched braces in leading prose.

Example: The fallback extractor receives Plan: wrap keys in { braces.\n{"ok":true}.

• Before: Returns null despite the valid JSON object.
• After: Returns the object {"ok":true}.

Review scores

Measure	Result	What it means
Overall readiness	🧂 unranked krab (1/6)	The focused patch has no identified correctness defect, but test-only evidence does not satisfy the external-PR behavior-proof gate.
Proof confidence	🧂 unranked krab (1/6)	Real behavior proof is necessary before merge. See Before merge.
Patch quality	🐚 platinum hermit (4/6)	No actionable review findings were identified.

Product

Kind: Bug fix · Worth it: Yes · Fix scope: Complete
User problem: Review or repair can fail with malformed-output even when the provider returned valid structured JSON after prose containing an unmatched brace.
Reason: Recovering the reported responses extends the established tolerant-extraction contract and benefits every adapter using the shared helper.

Merge readiness

⛔ Blocked before merge - 1 item remains

This PR repairs a current provider parsing defect and preserves the existing recovery contract; no actionable code defect was found.

Priority: P2
Reviewed head: 20008175b096dc883ad865994727d53dbf8ade9e

Before merge

• Add real behavior proof - The supplied evidence exercises extractJson through deterministic tests and reports passing checks, but does not demonstrate recovery through the shipped CLI and provider adapter; provide terminal output or a terminal screenshot of that scenario, redacting credentials and non-public endpoints. After adding proof, update the PR body, then request a new review with @clawsweeper re-review. The PR author or someone with repository write access can request it.

Findings

None.

Tests

• Missing end-to-end proof: No supplied evidence shows a shipped review or fix command successfully consuming a schema-valid provider result after unmatched leading prose.

Agent review details

How this fits together

Provider adapters pass captured response text through the shared JSON extractor, then validate the extracted value before returning review or repair results.

flowchart LR
  A[CLI command] --> B[Provider adapter]
  B --> C[Captured response text]
  C --> D[Shared JSON extractor]
  D --> E[Schema validation]
  E --> F[Workflow result]

Loading

Technical review

Best possible solution:

Keep recovery in the shared extractor while preserving strict parsing, balanced-candidate priority, nested-draft exclusion, and downstream schema validation.

Do we have a high-confidence way to reproduce the issue?

Both reported inputs deterministically reach current main's unmatched-brace termination branch; the PR reports failing-before and passing-after regressions, but no target code was executed during this read-only review.

Is this the best way to solve the issue?

The shared-owner repair addresses the reported failure while retaining historical candidate-selection safeguards, without adding configuration or a parallel parsing path.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against 610a26db8437.

Provenance checked

• Shared fallback candidate selection keeps the original intent (429cc9c: Skip balanced malformed preambles while excluding their nested draft objects; earlier commits extracted the helper and connected Codex to tolerant parsing.)

Testing

Proof path: unit tests only.

Security

None.

Evidence

What I checked:

• Current main still contains the defect: An unmatched opening brace leaves depth nonzero and terminates extraction without examining later objects. (src/provider-json.ts:49, 610a26db8437)
• Introduced repair and focused coverage: The pinned diff retains strict and fenced parsing, prioritizes balanced top-level candidates, and recovers valid children of unmatched braces; tests cover both reported inputs and preserve nested-draft exclusion. (src/provider-json.ts:14, 20008175b096)
• Existing parsing intent: Commit 429cc9c added retries after balanced malformed candidates and explicitly tested that nested draft JSON is skipped. (src/provider.test.ts:200, 429cc9cbb9fd)
• Production failure boundary: Pi passes stdout to extractJson and throws malformed-output on null; Codex and several other adapters also consume the shared extractor. (src/providers/pi.ts:113, 610a26db8437)
• Linked report and supplied proof: The concrete inputs in extractJson gives up when provider output has an unbalanced "{" before the JSON object #237 match the repair; the complete PR body reports deterministic tests and checks, while its comments supply no shipped-command demonstration. (20008175b096)
• History coverage: History identifies steipete's helper extraction, Codex fallback integration, and malformed-preamble repair; local historical blob reads encountered missing objects, and REST commit patches supplied the extraction and integration details. (src/provider-json.ts:3, 611103ab59d5)

Labels

Label changes:

• add P2: A specific provider response pattern interrupts review and repair despite usable JSON; no urgent regression is established.
• add rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🐚 platinum hermit.
• add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Label justifications:

• P2: A specific provider response pattern interrupts review and repair despite usable JSON; no urgent regression is established.
• rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🐚 platinum hermit.
• status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

• Supply a redacted shipped-command trace demonstrating successful recovery of a schema-valid result after unmatched leading prose.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only. Repair and merge execution lanes are retired.

Reviewed October 18, 2026, 11:44 PM ET / October 19, 2026, 03:44 UTC.