docs(cli): Generate and check docs that mirror CLI behavior by sentry-junior[bot] · Pull Request #1626 · getsentry/toolkit

sentry-junior · 2026-10-23T12:32:44Z

#1541 fixed most of the CLI docs drift found in getsentry/cli#1644, but the remaining gaps were all the same kind of problem: hand-written docs that copy facts out of the code. Fixing each one by hand would just bring the drift back later, so this PR makes the docs come from the code, and adds CI checks for what can't be generated.

Generated from code (via generate-docs-sections.ts markers, checked by check:docs-sections):

Destructive commands in the agent Safety Rules. A new destructive flag on buildCommand drives this list, and buildDeleteCommand sets it automatically. issue merge and trial start set it explicitly. The list grows from 2 commands to 10, and the generated skill picks it up. It now also covers cli uninstall, integration delete and issue unlink, which refactor(cli): Generate the SDK with the TypeScript AST instead of strings #1644's hand-made list missed.
Detected agents in agentic-usage. These come from a new AGENT_DISPLAY_NAMES map. A unit test fails if detectAgent() can return an agent with no display name. This adds Antigravity, which the docs were missing.
Target syntax table in the commands overview and CONTRIBUTING.md. It comes from TARGET_FORMS, and a test runs every documented example through parseOrgProjectArg(). The test also fails if a parse type has no documented form.
Skill file layout in plugins/README.md. Generation fails if a listed path no longer exists.

New checks (added to ci:policy):

check:doc-examples checks about 1,400 sentry … invocations against the live route tree. It covers fenced shell blocks and inline code in hand-written docs, plus command --help examples. An example fails if the command doesn't exist (aliases count), if it uses a flag the command doesn't define, or if it passes a bare my-org where the CLI looks up projects first. The last rule found ambiguous examples in the dashboard help, alert metrics create, project list and agent-conversation list, which now use my-org/.
check:installer-docs checks the install scripts:
- Every SENTRY_* variable or flag an install script reads must appear in its own help text.
- Everything in either installer's help must appear in the installation guide.
- It caught SENTRY_CLI_NO_TELEMETRY, which the bash installer and the guide didn't mention.
check:stale-refs now flags pnpm run cli -- <args>. pnpm passes the literal -- through, so pnpm run cli -- --help fails with "Command not found: --help".

Dev setup: pnpm run cli and generate:docs now create missing src/generated/ files on a fresh checkout through script/ensure-generated.mjs. That removes the generate:schema bootstrap step instead of documenting it. Before this, pnpm run cli and pnpm run test:unit failed on a fresh clone.

Docs-only fixes:

The bare-name row in the sentry init target table now matches resolveTarget: existing project, then org, then a new project.
The --keep-config help text now uses the XDG config dir constant instead of ~/.sentry.
The package README tells Homebrew and package-manager users to run sentry cli setup for agent skills; package managers also skip completions.

Not ported from #1644: its claim that Homebrew runs cli setup automatically. The formula in .craft.yml doesn't run it, and its caveats tell users to run it themselves.

via BYK.

--

View Junior Session in Sentry

Lists that copied code (destructive commands, detected agents, target syntax, skill file layout) are now generated from the code. New checks fail CI when a `sentry ...` example names a missing command or flag, passes a bare org slug as a project-first target, or when an installer option is missing from its help text or the install guide. `pnpm run cli` and `generate:docs` now create missing build-time files, and check:stale-refs flags `pnpm run cli -- <args>`, which pnpm forwards as a literal `--`. Co-Authored-By: Claude Opus <noreply@anthropic.com> Co-Authored-By: BYK <burak.kaya@sentry.io>

vercel · 2026-10-23T12:32:51Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
sentry-local	Skipped		Oct 23, 2026 9:29pm UTC

BYK

Great work!

Address review feedback on the docs drift guards: - Type agent tables by `KnownAgent` (keys of AGENT_DISPLAY_NAMES), so a detectable agent without a display name is a compile error. Drop the map-structure test. - Key TARGET_FORMS by parse result type so TypeScript enforces that every type is documented, and have the generator parse each example and fail on a mismatch. Drop target-syntax.test.ts. - Check only structured `docs.examples`, not examples hand-written into `fullDescription`. - Scan the install scripts in check:env-coverage so installer variables must be in ENV_VAR_REGISTRY (and therefore configuration.md). Add the missing SENTRY_NO_MODIFY_PATH. check:installer-docs now covers only installer help text and flags. - Replace the pnpm-specific `pnpm run cli --` lint with a dev-only preload that drops a leading `--`, so the dev scripts work with any package manager. - Add check:destructive: derive SDK DELETE operations from @sentry/api, follow calls, imports, and barrel re-exports through src/, and fail when a command that sends a DELETE is not marked destructive. Co-Authored-By: BYK <burak.kaya@sentry.io>

@latest

…1594) Refs #1564. This is the first slice: the mechanism plus five commands. A `[<org>/][<project>]` positional now owns both parsing and resolution. The command picks a parser from `src/lib/target-positional.ts`, and that parser comes with its resolver: ```ts parse: orgOptionalTarget, // resolveOrgOptionalTarget parse: projectBoundTarget, // resolveProjectBoundTarget parse: orgProjectTarget(resolveViewTargets), // command-specific resolver ``` `func` gets a `LazyPositional` with three members: `target.parsed` (the `ParsedOrgProject`), `target.resolved` (a promise of the resolver's result), and `target.raw` (what the user typed): ```ts async *func(this: SentryContext, flags: ListFlags, target: OrgOptionalTarget) { const resolved = await target.resolved; ``` `func` never calls `parseOrgProjectArg` or a resolver, never picks which resolver to use, and needs no default for an omitted target. An omitted optional target arrives as auto-detect. `project view` moves its target switch into `resolveViewTargets` and reads `target.parsed.type`. - **No command names to pass.** Resolvers read the command path from `this.commandPrefix`, which Stricli's `forCommand` sets and the SDK invoker now sets from its command path. The `COMMAND_NAME` constants these commands used are gone. A context without `commandPrefix` makes resolution throw rather than print a hint with a blank command name; test mock contexts set it. - **Destructive commands guard automatically.** `buildDeleteCommand` calls `markDestructive` on its lazy positionals, and the target resolver then runs `requireExplicitTarget`. `project delete` uses plain `projectBoundTarget`. Once #1626's `destructive` flag lands, the call can move into `buildCommand` so `issue merge` and `trial start` get it too. - **Lint rule.** `sentry-cli/require-target-resolution` fails a command function that takes `OrgOptionalTarget`, `ProjectBoundTarget` or `OrgProjectTarget<R>` but never reads `.resolved`, since skipping it skips the resolver and the destructive guard. **When things run.** Both steps are lazy (`LazyPositional` in `src/lib/positional-parse.ts`, a plain class with memoized getters). The parse runs on the first `parsed` read and the resolver on the first `resolved` read; each runs once. Both happen inside `func`, so the order matches main: - The auth guard and rc URL check run first. A bad target with no login still reports "Not authenticated" (exit 10). - Parse errors keep their `CliError` semantics. `sentry project view @latest` exits 21 with the `ValidationError`, not Stricli's `Failed to parse …` (exit -4). A parse error also rejects `resolved`. - `parseOrgProjectArg`'s URL side effect still runs after those checks. For a pasted Sentry URL it calls `applySentryUrlContext`, which can switch `SENTRY_URL`/`SENTRY_HOST`, save that host as the default, or throw `HostScopeError`. Running it at Stricli scan time would have let the auth guard and rc URL check run against the host taken from the URL. Resolution stays a promise because resolvers need auth, config, DSN detection, and API calls. Shared code that handles any positional (`buildListCommand`'s plural-alias interception, `help` recovery, `setArgsContext`) reads the raw input with `rawPositional(value)`, an `instanceof LazyPositional` check. Stricli calls parsers with the command context as `this`; the resolver gets that context plus `commandName`. **Callers that skip Stricli.** The typed SDK (`buildInvoker`) and tests that call `command.loader()` pass raw strings. `buildCommand` runs `applyLazyPositionals` on them, and on omitted optional positionals, so every caller keeps passing what the user typed and `func` always gets the same shape. When plural-alias interception drops the target, `buildListCommand` runs it again so the dropped target becomes auto-detect. `sdk.project.view({ orgProject: "@latest" })` is rejected instead of silently auto-detecting. The SDK invoker also sets `directInvoke`, which turns off `help` positional recovery: a failing SDK call whose target is `help` throws its error, as on main, instead of returning help text. **Shell completion is unchanged.** Completion kinds come from a tag on the parser (`withCompletionTarget`), shared with `parseOrgTarget` and `parseOrgProjectTarget`. The generated targets file matches main. Migrated: `trace list`, `replay list`, `feedback list`, `project view`, `project delete`. Not in this PR: - `LIST_TARGET_POSITIONAL` / `buildOrgListCommand`. #1575 (`splitListTargetAndFilter`) reads the raw target and splits it before parsing (for example `acme/web-*`). These should move after it lands, using `target.raw`. - Issue and dashboard positionals. #1572 adds `parseIssueTarget` / `parseDashboardTarget` in the same files. When one of the two PRs rebases, its parsers should switch from the `PARSER_TARGETS` map to `withCompletionTarget`. - `[<org>/<project>/]<id>` positionals (`trace view`, `span list`, `event view`, `log view`, …). Stricli parses each argument on its own, but what each one means depends on how many were given. Those keep parsing in `func`. - Release `org/version` positionals. They already share `resolveReleaseTarget`.  via **BYK**.    -- [View Junior Session](https://junior-prod.sentry.dev/conversations/slack%3AC010884QHLG%3A1792667769.623489) [[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AC010884QHLG%3A1792667769.623489/?project=4510944073809921)  --------- Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: Burak Yigit Kaya <burak.kaya@sentry.io>

BYK

Looks good. Let's make sure this is compatible with #1594 which is just merged and then we're ready to go.

…ards

Follow-up to #1594: `buildCommand` now calls `markDestructive` when `destructive: true`, instead of `buildDeleteCommand` doing it, so the `destructive` flag is the single source for the Safety Rules list and the auto-detect guard. `issue merge` and `trial start` get the guard if they move to lazy target positionals. check:doc-examples now recognizes target positionals by their `org/project` completion kind, which covers the new lazy target parsers as well as `parseOrgProjectTarget`.

sentry-junior · 2026-10-23T20:21:14Z

Merged main (with #1594) into the branch in 18467ae. Git merged it without conflicts, and d1fb35f makes the two work together:

buildCommand now calls markDestructive whenever destructive: true, and buildDeleteCommand no longer calls it itself (the move feat(cli): Parse and resolve org/project targets in their positional #1594 anticipated). The destructive flag is now the single source for both the Safety Rules list and the auto-detect guard. issue merge and trial start get the guard once they use lazy target positionals. A new test in positional-parse.test.ts checks this, and it fails if the call is removed.
check:doc-examples now recognizes target positionals by their org/project completion kind, so it covers orgOptionalTarget, projectBoundTarget and orgProjectTarget(...) as well as parseOrgProjectTarget.
check:destructive still finds all 6 DELETE commands, including project delete on projectBoundTarget.

Passing: ci:policy, oxlint on packages/cli/src and script, and 11 test files including positional-parse, command, mutate-command, project/delete, sdk-invoke, issue merge and trial start. tsc shows only the 3 existing src/lib/mcp.ts errors from @sentry/mcp-server not being built in my sandbox.

Resolve conflicts with #1633, which moved the remaining hand-written examples into `docs.examples`: take main's version of the dashboard commands and the alert/dashboard fragments, then regenerate the skill references. check:doc-examples now covers the migrated examples and flagged: - `dashboard revisions` and `dashboard restore` examples that pass a bare `my-org` (a project lookup); changed to `my-org/` - examples that start with `sudo` or `NAME=value`, which the check misread as the command name; it now skips those prefixes Co-Authored-By: BYK <burak.kaya@sentry.io>

#1633 added the `no-regex-markdown-parsing` lint rule. Replace the fenced block regex with the shared `markdownCodeBlocks` / `blankCodeBlocks` helpers in script/markdown.ts, which use the same lexer as the docs renderer.

@Form

Closes #1637. Stacked on #1626 (it replaces `TARGET_FORMS` from that PR), so the base is `byk/cli-docs-drift-guards`. The `<org>/<project>` target syntax is now one peggy grammar, `packages/cli/grammar/org-project-target.pegjs`. Both the parser and the docs table come from it. - **Parser.** `parseOrgProjectArg` keeps its signature. It still keeps the first non-blank line, auto-detects on empty input, and reads Sentry URLs itself. Everything else goes through the generated parser. Slug and `@` checks run through `options` hooks (`validateResourceId`, `rejectAtSelector`), so error messages stay the same. `parseSlashOrgProject`, `parseExplicitProjectList`, and their helpers are deleted. The backtracking pitfall from the spike is documented in the grammar, and the existing `@latest` test guards it: token rules only capture text, and validation runs in the winning alternative. - **Docs.** Each `target` alternative has `// @Form <syntax> — <meaning>` lines above it, and `generate-docs-sections.ts` builds the table from them. Generation fails if an alternative has no `@form` or `@invalid` line. The check uses peggy's grammar AST, so a new form can't ship undocumented. The table now also lists the forms `TARGET_FORMS` was missing: Sentry URLs, `/<name>`, `<org>/<Project Name>`, and `<Project Name>`. Omitted targets and URLs never reach the grammar, so they are annotated in the grammar header. - **Generator.** `packages/toolkit-core/script/generate-parsers.ts` moved to `scripts/generate-parsers.mjs` (`pnpm run generate:parsers`). One list of grammar specs covers both packages, and `check:generated` and the pre-commit hook use it. The CLI output goes to `packages/cli/src/generated/org-project-target-grammar.{js,d.ts}` and is committed (`.gitignore` exception). It is about 24 KB unminified. peggy moves from a toolkit-core devDependency to the root and the CLI. The lockfile only changes the importer entries. Migration check, not committed: I compared the new parser with a copy of the old one on 139,954 inputs. They were every string up to 4 characters over `a b / , space @ - _ ? # \n 1 % . A \t`, the 53 literals in `arg-parsing.test.ts`, and some extra edge cases, each with `multi` on and off. There were 0 differences in results, error classes, messages, or fields. While doing this I found and fixed one real bug, which needed end-anchored token rules. Without them, PEG prefix matching turned `A #` into a `SyntaxError`. Not in this PR: `parseIssueArg` (step 4 of the issue), and removing `normalizeSlug` / `normalized`. `parseIssueArg` and `trace-target.ts` still use the latter. `originalSlug` is still needed: it is how resolution tells a display name from a slug.  via **BYK**.    -- [View Junior Session in Sentry](https://sentry.sentry.io/explore/conversations/local%3Aweb%3Ac5ce32bb458d46e900d4be2e/?project=4510944073809921)  --------- Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: Claude Opus <noreply@anthropic.com> Co-authored-by: BYK <burak.kaya@sentry.io>

github-actions Bot added the risk: medium PR risk score: medium label Oct 23, 2026

vercel Bot deployed to Preview October 23, 2026 12:34

View deployment

BYK requested changes Oct 23, 2026

View reviewed changes

vercel Bot temporarily deployed to Preview October 23, 2026 14:01 Inactive

BYK mentioned this pull request Oct 23, 2026

feat(cli): Parse and resolve org/project targets in their positional #1594

Merged

BYK reviewed Oct 23, 2026

View reviewed changes

sentry-junior Bot added 2 commits October 23, 2026 20:20

Merge remote-tracking branch 'origin/main' into byk/cli-docs-drift-gu…

18467ae

…ards

vercel Bot deployed to Preview October 23, 2026 20:23

View deployment

sentry-junior Bot mentioned this pull request Oct 23, 2026

ref(cli): Define the org/project target syntax as a PEG grammar #1637

Closed

sentry-junior Bot mentioned this pull request Oct 23, 2026

ref(cli): Define the org/project target syntax as a PEG grammar #1638

Merged

vercel Bot deployed to Preview October 23, 2026 21:24

View deployment

ref(cli): Parse doc code blocks with marked in check:doc-examples

4a49a82

#1633 added the `no-regex-markdown-parsing` lint rule. Replace the fenced block regex with the shared `markdownCodeBlocks` / `blankCodeBlocks` helpers in script/markdown.ts, which use the same lexer as the docs renderer.

vercel Bot temporarily deployed to Preview October 23, 2026 21:29 Inactive

BYK approved these changes Oct 23, 2026

View reviewed changes

BYK merged commit 203fbbe into main Oct 23, 2026
40 checks passed

BYK deleted the byk/cli-docs-drift-guards branch October 23, 2026 21:47

Navigation Menu

Repository navigation

docs(cli): Generate and check docs that mirror CLI behavior - #1626

docs(cli): Generate and check docs that mirror CLI behavior#1626
BYK merged 6 commits into
mainfrom
byk/cli-docs-drift-guards

sentry-junior Bot commented Oct 23, 2026

Uh oh!

vercel Bot commented Oct 23, 2026 •

edited

Loading

Uh oh!

BYK left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

BYK left a comment

Uh oh!

sentry-junior Bot commented Oct 23, 2026

Uh oh!

Uh oh!

This branch was previously deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

sentry-junior Bot commented Oct 23, 2026

Uh oh!

vercel Bot commented Oct 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

BYK left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

BYK left a comment

Choose a reason for hiding this comment

Uh oh!

sentry-junior Bot commented Oct 23, 2026

Uh oh!

Uh oh!

This branch was previously deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

vercel Bot commented Oct 23, 2026 •

edited

Loading