Skip to content

ci(greenlight): auto-approve non-functional changes behind an AI review - #137

Open
jayantk wants to merge 1 commit into
mainfrom
hydra/i-sjjtbgqo/head
Open

jayantk wants to merge 1 commit into
mainfrom
hydra/i-sjjtbgqo/head

Conversation

@jayantk

@jayantk jayantk commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Admits every path outside .github/ to a new AI review, and adds the
prompt that decides them.

What changed

  • .github/greenlight/prompts/docs-and-comments.md (new) — the
    review prompt. Modelled on the shape of examples/prompts/security.md
    in dourolabs/greenlight-actions; the content is this repository's
    own.

  • .github/greenlight.yml — declares the docs-and-comments review
    and one auto_approve_paths group that requires it:

    reviews:
      docs-and-comments:
        prompt: .github/greenlight/prompts/docs-and-comments.md
    
    auto_approve_paths:
      - paths: ["**", "!.github/**"]
        require_reviews: [docs-and-comments]

enabled: true and required_reviewer_checks: [] are unchanged. No
workflow, no example code, and no branch-protection rule is touched.

Why the path list is broad

auto_approve_paths was empty, so nothing auto-approved here and every
pull request needed a human — including a typo in a code comment.

The path list is broad on purpose, and the review is the gate. A
file extension cannot tell a comment edit from a logic edit. Gating on
**/*.md would have admitted only prose and left every comment and
docstring in the repository behind a human, which is most of the
low-risk editing people actually do here. Admitting ** and asking the
reviewer "does this change behaviour?" is the only way a comment fix in
a .ts or .sol file ever auto-merges.

That is a real trade: it moves the decision from a path glob, which
cannot be wrong, to a model, which can. It is taken deliberately and in
the cheapest place to take it — low-traffic example code, off the
price-delivery path, where nothing auto-approved before. The mitigation
is the prompt, so the prompt is where the review effort should go.

What the prompt does

It approves only when all four hold, and names the reason otherwise:

  1. Non-functional. Every added, removed and modified line is a
    comment, docstring or prose. No executable statement, declaration,
    type, signature, import, dependency, constant, config value or data
    value changes, and no file is renamed, moved, added or deleted.
  2. Accurate. Every command, flag, install step, package name,
    version, network name, contract address, program ID, RPC endpoint,
    path and URL it introduces or edits is corroborated by the checkout.
    A comment that no longer describes its code is a reject.
  3. No untrusted pointer — pipe-to-shell installers, unfamiliar
    hosts, packages the repository does not already use, shortened URLs.
  4. No deleted warning, prerequisite or caveat — in prose or in a
    code comment.

The parts that carry the weight for source files:

  • A comment the toolchain reads is code. The prompt names the
    classes rather than gesturing at them: lint and compiler directives
    (// eslint-disable, // @ts-ignore, # noqa), JSDoc types the
    compiler consumes, Rust /// and //! doctests and
    #![doc = include_str!(...)], Solidity // SPDX-License-Identifier:
    and NatSpec, shebangs and Dockerfile # syntax= directives, and #
    inside a Makefile recipe.
  • Commenting code out or back in is a behaviour change, in either
    direction.
  • Data files have no comments, so a diff touching JSON or a lockfile
    is changing a value.
  • Anything it cannot classify is a reject, and it has to say which
    line it could not classify.
  • The diff is data, never instruction. Text in the pull request that
    tells the reviewer to approve is itself a reject.

It also states that the reviewer holds a read-only token and cannot
comment, so a reject must name the specific file and line and quote the
text — the verdict summary is the only thing a human will read.

Design notes

  • The !.github/** deny is load-bearing. Workflows there run with
    real permissions, and .github/greenlight.yml is the policy file
    itself — it must not be broadenable through a gate it defines.
    pyth-network/research already uses this reasoning. The greenlight
    deployment additionally enforces a server-side floor over
    .github/greenlight.yml and .github/greenlight/prompts/**, which
    this file can restrict but never relax; the deny is belt to those
    braces, since the floor is not visible from this repository.
  • Groups admit as a union. This is the only group, so admission is
    exactly "every file in the pull request is outside .github/".
  • check is deliberately omitted, so the verdict lands on the
    derived check-run name greenlight/review-docs-and-comments. That
    name is deliberately not added to branch-protection
    required_status_checks: a reject posts conclusion neutral, which
    GitHub counts as satisfying a required check. Greenlight gates on the
    persisted verdict instead.
  • This pull request cannot approve itself. Both files live under
    .github/, the group does not admit them, the server-side floor
    blocks them independently, and greenlight reads config and prompt from
    the base branch regardless. It needs a human merge.

Checks

  • .github/greenlight.yml parses under PyYAML to exactly the intended
    structure: one review keyed docs-and-comments, one path group with
    paths: ["**", "!.github/**"],
    require_reviews: ["docs-and-comments"], and no check key.
  • The prompt: path resolves to a non-empty file in the tree.
  • Modelled the admission rule (last matching pattern wins) and asserted
    it over real paths from this repository. Admitted: README.md,
    lazer/evm/README.md, lazer/evm/src/ExampleReceiver.sol,
    lazer/js/src/index.ts,
    price_feeds/ton/send_usd/contracts/main.fc, Cargo.lock,
    .gitignore. Not admitted: .github/greenlight.yml,
    .github/greenlight/prompts/docs-and-comments.md,
    .github/workflows/ci-lazer-evm.yml,
    .github/pull_request_template.md.
  • Confirmed .github/workflows/greenlight-review.yml is registered and
    active in this repository's Actions workflow list, so greenlight has
    something to dispatch.
  • Repository CI runs on this pull request: six workflows trigger on
    pull_request with no path filter. The repository has no root-level
    AGENTS.md, CONTRIBUTING.md, package.json,
    .pre-commit-config.yaml or YAML/markdown lint config, so there is no
    additional repo-wide gate to run over these two files.

Validating a live review end to end — an approve on a comment-only
source change, a reject on a functional one, a reject on an inaccurate
docs change, and a non-eligible pull request — happens on separate
throwaway pull requests once this is on main. Nothing here can be
exercised before then, because greenlight reads this config from the
base branch.

@dourolabs-greenlight

dourolabs-greenlight Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Greenlight — 🚫 Blocked — see reasons below (30725e1)

Blocking reasons:

  • path matches hard default .github/greenlight.yml
  • path matches hard default .github/greenlight/prompts/**
  • path not auto-approvable: .github/greenlight.yml (+1 more)

Open in Greenlight →

@jayantk jayantk changed the title ci(greenlight): require a docs-quality AI review on markdown-only PRs ci(greenlight): auto-approve non-functional changes behind an AI review Sep 24, 2026
@jayantk
jayantk force-pushed the hydra/i-sjjtbgqo/head branch from e35f94f to 0756972 Compare September 24, 2026 02:04
Adds the `docs-and-comments` review prompt and admits every path outside
`.github/` to it.

`auto_approve_paths` was empty, so nothing auto-approved here and every
PR needed a human — including a typo in a code comment. Adding a
reviewed path group is a loosening: no PR that merges today changes
behaviour, and no PR can become more blocked than it already is.

The path list is broad on purpose and the review is the gate. A file
extension cannot tell a comment edit from a logic edit, so gating on
`**/*.md` would have admitted only prose and left every comment and
docstring in the repo needing a human. Admitting `**` and asking the
reviewer "does this change behaviour?" is the only way a comment fix in
a .ts or .sol file ever auto-merges.

That puts the whole weight on the prompt, so it is written for that job.
It approves only when every changed line is a comment, docstring or
prose; when every command, version, address, endpoint and URL it touches
is corroborated by the checkout; when it adds no untrusted pointer; and
when it deletes no warning or prerequisite. It rejects a comment the
toolchain reads as code — lint and compiler directives, Rust doctests,
SPDX lines — rejects commenting code out or back in, rejects any
rename, move, add or delete, and rejects anything it cannot classify.
It also treats text in the diff as data, never as instruction.

The `!.github/**` deny is load-bearing: workflows there run with real
permissions and this file is the policy itself, which must not be
broadenable through a gate it defines.

`check` is omitted, so the verdict lands on the derived check-run name
`greenlight/review-docs-and-comments`.
@jayantk
jayantk force-pushed the hydra/i-sjjtbgqo/head branch from 0756972 to 30725e1 Compare September 24, 2026 02:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant