Reconsider REVIEW_FINDING_SEVERITY_THRESHOLD="high" override in .fullsend/customized/harness/review.yaml #583

What happened

On PR #582, a change to .github/workflows/preview.yaml introduced a fork-origin guard on the deploy job but not on prepare, so fork-originated workflow runs would still download and unzip untrusted artifacts before the guard applied. CodeRabbit caught this on the first commit d94dbad (review comment, 2026-10-07T16:41:39Z). The fix landed in commit f4a98b3 ("Apply the repository-origin check before downloading and unpacking preview artifacts").

The fullsend review agent ran on all three commits (runs 37653239666, 37656093643, 37658698303) using fullsend-ai/agents@ce2eedd and the repo-local harness override at .fullsend/customized/harness/review.yaml, which sets REVIEW_FINDING_SEVERITY_THRESHOLD: "high" for both the runner and the sandbox. The only finding posted across all three runs was a [protected-path] medium — the fork-guard bug was not reported.

What could go better

The upstream review agent's sub-agent pipeline (correctness, security, intent-coherence, style-conventions) is explicitly designed to inspect GitHub Actions workflows for exactly this kind of permission/guard regression. The most parsimonious explanation for the miss is that any sub-agent output below high is dropped by the post-review severity filter before it reaches the comment. CodeRabbit itself scored the fork-guard finding as "Trivial / Low value"; a natural fullsend grading for the same issue would plausibly land at medium and be filtered out.

Uncertainty. The protected-path finding posted on PR #582 was itself a medium and did reach the comment, which suggests the filter may have a category exemption — so the hypothesis above is not watertight. It is also possible the correctness sub-agent did not produce the finding at all, or produced it at low. Either way, the net effect for this repo is the same: on a security-sensitive diff to .github/, the review surfaced zero actionable correctness or security findings while a competing bot surfaced one that the human acted on. A high-only threshold on governance paths is almost certainly tighter than this repo intends.

Proposed change

In .fullsend/customized/harness/review.yaml, lower REVIEW_FINDING_SEVERITY_THRESHOLD to medium — at least for diffs that touch .github/, CODEOWNERS, or scripts/. Options to evaluate, in order of preference:

1. Drop the override entirely and inherit the upstream default (low) to see which findings surface. Downgrade to medium after one week if comment noise is too high.
2. If keeping the override, change both the runner and sandbox values to medium so the correctness and security sub-agents' typical outputs survive the filter.
3. If the filter does support per-category exemption, add an exemption for correctness and security findings regardless of severity.

Before merging the change, verify the actual behavior of scripts/post-review.sh severity filter in fullsend-ai/agents@ce2eedd to confirm whether the protected-path medium made it through because of a category exemption or because the filter is not applied the way the upstream script implies.

Validation criteria

After the change, over the next 10 PRs that touch .github/, scripts/, or CODEOWNERS:

• The fullsend review posts at least one non-protected-path finding on PRs where competing tooling (CodeRabbit, Qodo) finds one — OR the review comment states explicitly that correctness/security sub-agents ran and found nothing additional.
• No increase in false-positive volume above 1 finding per PR on average (judged by maintainer response: ignored vs. acted-on).
• If the lower threshold produces comment-noise regressions, revert and document the specific noise category in a follow-up issue so an upstream fix can target it.

Generated by retro agent from #582