Skip to content

docs(agents): agent-neutral pre-PR review skill and Review record - #767

Open
vishal-bala wants to merge 3 commits into
mainfrom
feat/pre-pr-review-skill
Open

vishal-bala wants to merge 3 commits into
mainfrom
feat/pre-pr-review-skill

Conversation

@vishal-bala

@vishal-bala vishal-bala commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

Many PRs to this repository have already been through several rounds of agentic review before a human sees them, but none of that review is visible on the PR. The approver can't tell what was checked, so they either repeat the review or put it off. This PR makes that earlier review part of the PR. Agents of any vendor follow a shared pre-PR review procedure, and the result is recorded in the PR description as a short Review record, which the approver checks rather than redoes. A human still approves every merge; the aim is to make that approval quick.

Changes

One guidance file for every agent

AGENTS.md now holds the project guidance that used to live in CLAUDE.md, unchanged, plus three new sections:

  • the pre-PR review;
  • the Redis documentation server;
  • how to respond to review comments.

CLAUDE.md is reduced to @AGENTS.md, so Claude Code reads the same file as Codex, Cursor, Copilot and others. The protected tokenizer line moved byte for byte.

The pre-pr-review skill

.agents/skills/pre-pr-review/ follows the open Agent Skills format: a SKILL.md with name, description and compatibility front matter, plus one reference file per lens. It assumes only git, make and pytest. Subagents are used if the agent has them; otherwise the passes run in sequence. An agent that doesn't load skills reaches the files through AGENTS.md.

The procedure has six steps:

  1. Scope the diff.
  2. Choose lenses from its size, type and paths, and record why each was run or skipped.
  3. Run one pass per lens.
  4. Run a separate challenger pass, which keeps only findings that score 7 or more out of 10 and are tied to a file, a line and a concrete failure.
  5. Fix the survivors in scope.
  6. Write the Review record.

The six lenses are correctness (always), security, design, tests, docs, and Redis behaviour. The triggers come from what review has actually caught in this repository. Bugbot's findings are mostly correctness and security, and it rarely flags vacuous or redundant tests or stale docs. So the tests and docs lenses cover the gap, and they treat deleting a test or a line as a valid finding.

The Redis behaviour lens checks command syntax, defaults, limits and reply shapes against the official documentation through the public, read-only Redis Docs MCP at https://redis.io/mcp. Each of its findings has to cite the page it rests on.

The Review record names the reviewed commit and the agent and model that ran the review. It lists the lenses run and skipped, what was fixed, what was rejected and why, the commands run, the docs consulted, and what was not checked.

Bugbot guide, PR template and contributor docs

  • .cursor/BUGBOT.md points Bugbot at what CI can't check: sync/async drift, query values leaving their clause, MCP authorisation, destructive operations, dependency floors, cluster behaviour and docs drift. It also tells Bugbot to treat the Review record as a claim to verify.
  • .gitignore now ignores .cursor/* except BUGBOT.md, so the guide can be tracked.
  • .github/pull_request_template.md adds Motivation, Changes, Release Notes and Review record sections.
  • CONTRIBUTING.md explains what a Review record contains, and gives human reviewers a short routine for reading one alongside Bugbot's summary.

Notes

Nothing here is enforced. The record is guidance in the template and contributor docs, not a required check, and who may approve is unchanged. That keeps the cost of trying it low. If the record turns out not to help reviewers, it can be dropped without touching CI or branch rules.

Next Steps

  1. After a week of use, review how the records and the lens choices worked on real PRs, and adjust the triggers and lens files.
  2. Decide whether to keep the Copilot review ruleset alongside Bugbot.

Note

Low Risk
Documentation and contributor/agent workflow only; no changes to RedisVL application code, CI enforcement, or branch protection.

Overview
Introduces a shared pre-PR review workflow so agent-assisted work is visible to human approvers via a Review record in the PR description, without changing library runtime code or merge gates.

AGENTS.md becomes the single agent guide (moved from CLAUDE.md, which now only @AGENTS.md), with new sections on running pre-PR review, using the Redis Docs MCP, and handling Bugbot/reviewer comments. The .agents/skills/pre-pr-review/ Agent Skill defines a six-step process (scope → lens selection → per-lens review → challenger scoring → fixes → record) with six reference lenses and a record template.

.cursor/BUGBOT.md aligns automated review with CI gaps; .gitignore ignores .cursor/* but keeps BUGBOT.md. A new .github/pull_request_template.md adds Motivation, Changes, Release Notes, and Review record. CONTRIBUTING.md documents how to write and read Review records and adds a checklist item for maintainer/agent PRs.

Reviewed by Cursor Bugbot for commit 221aaf9. Bugbot is set up for automated code reviews on this repo. Configure here.

Move the project guidance from CLAUDE.md into AGENTS.md so that every
coding agent reads the same instructions; CLAUDE.md now only imports it.

Add a pre-pr-review skill in the Agent Skills format under
.agents/skills/. It chooses review lenses per diff (correctness always;
security, design, tests and docs when the diff calls for them), runs a
challenger pass that keeps only findings tied to a line and a concrete
failure, and writes a Review record into the PR description so the
approver can see what was reviewed, fixed, rejected and not checked.

Add a Bugbot guide scoped to what CI cannot check, and a PR template
with a Review record section.
Describe what the Review record in a PR description contains, how to
produce it with the pre-PR review in AGENTS.md, and a short routine for
human reviewers: check the reviewed commit matches the head, read the
record alongside the Bugbot summary and open threads, and spot-check
one fix and the "Not checked" items before approving.
Add a sixth pre-PR review lens that checks the change's assumptions
about Redis (command syntax, defaults, limits, reply shapes and
version-specific behaviour) against the official documentation rather
than model memory. It recommends the public, read-only Redis Docs MCP
at https://redis.io/mcp, falls back to redis.io/docs, and requires each
finding to cite the page it rests on.

Recommend the same server in AGENTS.md for everyday Redis questions,
and add a "Docs consulted" line to the Review record.
@vishal-bala vishal-bala added the auto:documentation Changes only affect the documentation label Oct 15, 2026
@vishal-bala
vishal-bala marked this pull request as ready for review October 15, 2026 16:07
@vishal-bala
vishal-bala requested review from justin-cechmanek, limjoobin and nkanu17 and removed request for nkanu17 October 15, 2026 16:09
@vishal-bala
vishal-bala added this pull request to stack #769 October 16, 2026 10:25

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto:documentation Changes only affect the documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant