docs: Surface factory definition validation and schema; fix .md code blocks

[hongyi-chen]

Summary

Customers editing factory definitions couldn't tell what files and keys are valid, how to check their YAML before it applies, or what tooling helps with that. The warp/factory-config pull request check and the JSON Schema existed but were buried at the bottom of the page; they now appear near the top alongside the other validation paths — the validation endpoint, the validator script, the Warp Agent's factory-files skill, and the Factory MCP tools. The Factory MCP page now documents its two schema/validation tools directly instead of only listing them in a table.

This PR also fixes the generated .md pages (docs.warp.dev/<page>.md): code blocks were rendering as a single unlabeled line, which made the Markdown version of every page unreadable for agents.

Changes

Content files are under src/content/docs/factories/.

factory-as-code.mdx

• Intro: state that the YAML is plain (anchors, aliases, tags, and duplicate keys are rejected), link the JSON Schema, and say up front that GitHub-backed definitions get the warp/factory-config check on every pull request and that teams can make it required.
• New ## Validate a definition section:
  • ### Pull request checks - what the check validates, when it passes immediately, the (directory) suffix for a subdirectory definition, and how to require it in branch protection.
  • ### Validate locally or in CI - the validation endpoint, the validate_factory_files.py script and its exit codes, the examples repo's CI workflow, and what a clean result does and does not prove.
  • ### Validate with a coding agent - the Warp Agent's built-in factory-files skill, the Factory MCP tools, and the shell fallback.
• New ## JSON Schema section: registry, bundle, and per-document URLs; the document-per-file-kind list; and a # yaml-language-server: $schema= example for editor completion.
• Removed the two sections at the bottom that these replace, and added ## Related pages.
• Updated the frontmatter description.

factory-mcp.mdx

• Added "Edit a factory's definition" to the use-case list and an example prompt.
• New ## Author and validate factory definitions section documenting get_factory_file_schema and validate_factory_files, with a cross-link to the validation section.
• Sharpened the two tool-reference rows and added a Related pages link.

integrations/github.mdx

• Corrected a claim: a pull request that doesn't touch the factory directory still gets the check and passes immediately, so requiring it doesn't block unrelated work. Linked the new validation section.

factory-skills.mdx, how-factories-work.mdx

• Updated the inbound anchor to the renamed #pull-request-checks section.

src/integrations/docs-markdown-integration.js (+ test)

• Fixed the generated .md pages: the converter read a code block's full text content instead of its per-line structure, so every block collapsed to one unlabeled line. It now rebuilds each block line by line, carries the language onto the fence, and preserves a block's title="...". Added unit tests (node --test src/integrations/docs-markdown-integration.test.js).

Content design plan

Audience and JTBD: An engineer (or a coding agent) with a factory definition open in their own repository, mid-edit, who needs to know which keys are allowed and whether what they wrote will apply.

Problem: The page answered "what keys exist" at length but hid the safety net (pull request check) and the exhaustive source of truth (JSON Schema) at the bottom, and never mentioned the validation endpoint, the validator script, the Warp skill, or the Factory MCP tools. The Markdown version of the page lost all code block formatting.

Goals:

• The reader learns in the first screen that GitHub-backed definitions are validated on every pull request and can be gated by a required status check.
• The reader can find the JSON Schema, and the way to validate a tree before pushing, from the table of contents.
• Someone using Warp or another coding agent knows the preferred way to edit and validate a definition programmatically.

Purpose and value: Customers reported these exact questions. Answering them on the page they already have open removes support back-and-forth and stops invalid definitions from reaching a production branch.

Content type: Reference (existing page), with a short task-oriented validation section. No new page.

Skill and template: draft_reference conventions; existing page updated in place.

High-impact scenarios:

• Covers: GitHub-backed definitions and the check; validating from a shell, CI, an editor, the Warp Agent, or any MCP-connected agent; the schema URLs.
• Excludes: GitLab-hosted definitions (the docs state GitLab is not yet supported as a definition host); per-key schema details beyond what the reference sections already carry.

Unverified claims

None. Every endpoint, tool name, check name, diagnostic code, and behavior claim was checked against warp-server source (files listed below) and the live API:

• GET /api/v1/factory-files/schemas, .../v1alpha1, and .../v1alpha1/factory.schema.json fetched unauthenticated; factory.schema.json compiled standalone with Ajv (draft 2020-12), auto-resolving common.schema.json by URL, and accepted a valid factory.yaml while rejecting an unknown key.
• POST /api/v1/factory-files/validate called unauthenticated with valid and invalid trees; the invalid tree returned FF_UNKNOWN_FIELD and FF_INVALID_REFERENCE with path, line, and column, and a tree with anchors, aliases, and a duplicate key returned FF_ANCHOR, FF_ALIAS, and FF_DUPLICATE_KEY. A leading # yaml-language-server: $schema=... comment validates clean.
• The factory-files skill being always bundled for the Warp Agent in the GUI, TUI, and cloud agents: warp/app/src/ai/skills/bundled_tests.rs.
• One claim relies on third-party tooling rather than Warp code: that the VS Code YAML extension reads a # yaml-language-server: $schema=<url> first-line comment. That is the extension's documented modeline feature and the use the server's per-document endpoint was built for (see the comment on GetFactoryFileSchemaDocumentHandler), but it was not exercised in an editor here.

Documentation risk

Risk: engineering-review-required
Rationale: Adds validation endpoint, schema URL, check-name, and MCP tool claims; every claim was verified against warp-server source and the live API, but the change adds commands and configuration examples.
Source files consulted: warp-server@9f9b411b (logic/factoryfile/prcheck/service.go; logic/factory_files_registration.go; logic/factoryfile/validation/contract.go; logic/factoryfile/diagnostic.go; router/handlers/public_api/factory_file_schemas.go and factory_file_validation.go; router/handlers/public_api/factory_mcp/{server,handle_validate_factory_files,handle_get_factory_file_schema}.go), warp@c12e2130 (resources/bundled/skills/factory-files/SKILL.md; app/src/ai/skills/bundled_tests.rs)
Engineering review status: pending
Docs override: none

Rework changes

Addressed all five review findings from rework cycle 1:

• [IMPORTANT] JSON Schema overstatement (factory-as-code.mdx:13) - Implemented. Replaced "The same rules are published as a JSON Schema" with wording that limits the schema claim to accepted files and keys, and points to the validation paths for parser/cross-file checks.
• [IMPORTANT] Pull request checks front-loads mechanics (factory-as-code.mdx:39) - Implemented. Cut the subsection to what an admin decides and does (the check runs, reports problems, passes unchanged definitions, can be required), and moved the head-commit/dry-run/create-update-delete/subdirectory-naming detail to the GitHub integration reference (integrations/github.mdx), which already documents GitHub-specific check behavior.
• [IMPORTANT] Dense coding-agent bullets (factory-as-code.mdx:80) - Implemented. Rewrote all three bullets to one action + one outcome each, cutting the repeated schema/endpoint/tool mechanics and linking to the adjacent local-validation and Factory MCP sections that own that detail.
• [IMPORTANT] Factory MCP section duplicates the canonical page (factory-mcp.mdx:153) - Implemented. Cut the section to a one-line description of each tool plus the existing practical prompt, removing the diagnostics field inventory, state-dependent-check list, and cross-file rationale, and linking to "Validate a definition" for the rest.
• [CRITICAL] No visual proof of the new rendered documentation - Implemented. Ran the local docs dev server and captured screenshots of the rendered "Validate a definition" and "JSON Schema" sections, including the right-hand on-page table of contents confirming the new headings render and navigate correctly. Attached below and on the ticket.

Also checked the CodeQL note (incomplete string escaping in docs-markdown-integration.js's title-attribute path): the fix (title.replace → JSON.stringify) already landed in a prior commit on this branch (267d1ca2), and the alert's most recent scan instance is marked fixed at that commit - no further code change needed.

Computer-use screenshots (2)

Full "Validate a definition" section showing H2 heading and all three H3 subsections (Pull request checks, Validate locally or in CI, Validate with a coding agent) together with the right-hand "ON THIS PAGE" table of contents listing all headings including the new ones.

"JSON Schema" section showing the H2 heading, intro paragraph, bulleted list of three schema endpoint URLs with descriptions, additional paragraph about per-file-kind schema documents, and a code block (factory.yaml) demonstrating the yaml-language-server comment; the "Directory structure" section begins below, and the right-hand "ON THIS PAGE" TOC is visible.

Co-Authored-By: Warp agent@warp.dev

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
docs	Ready	Preview	Sep 10, 2026 10:17pm UTC

[hongyi-chen]

This PR was generated with Warp.

Comment @warp-agent on this PR to send it follow-up work.

 

[warp-factories]

Your Warp account is not a member of any team with access to this repository.

[warp-agent-staging]

Your Warp account is not a member of any team with access to this repository.

[warp-for-oss]

@hongyi-chen

I'm starting a first review of this pull request.

You can view the conversation on Warp.

I completed the review and no human review was requested for this pull request.

Comment /warp-agent-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

[warp-for-oss]

Overview

This PR moves factory definition validation and JSON Schema guidance earlier in the factory docs, adds Factory MCP validation-tool coverage, updates related factory links, and fixes generated Markdown code block conversion with targeted tests.

Concerns

• factory-as-code.mdx overstates JSON Schema coverage in the intro by saying it publishes "the same rules" after listing YAML parser restrictions. Schema consumers still need the parser or validation endpoint for YAML-only restrictions and cross-file checks.
• Security review: no security-specific findings were identified in the changed docs or Markdown conversion code.
• Spec review: no approved or repository spec context was provided, so there was no implementation spec drift to validate.
• Comment/test audit: added code comments explain converter behavior, and the new tests cover distinct converter paths without violating repository conventions.

Verdict

Found: 0 critical, 1 important, 0 suggestions

Request changes

Comment /warp-agent-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

[github-actions]

Review summary

The independent agent completed its review for this commit.

Findings

• src/content/docs/factories/factory-mcp.mdx:149 — [SUGGESTION] The page already carries 3 callouts (:::note at line 16, :::caution at line 109, :::caution at line 139), 1 over the linted budget of 2 (AGENTS.md → Callouts and hints; check_compression_contract.py reports '3 callouts exceed the linted budget of 2'). Requested change: not introduced by this diff, but since the PR substantially edits this page, fold one existing callout into body prose while editing.

Verdict

Approve with nits

[github-actions]

Review summary

The independent agent completed its review for this commit.

Findings

• src/content/docs/factories/factory-as-code.mdx:38 — [SUGGESTION] The claim that a subdirectory definition gets warp/factory-config (factory) appended to the check name isn't itemized in the PR's verification notes the way other claims are; requested resolution: cite the specific prcheck/service.go code path or confirm the exact rendered format before merging.
• src/content/docs/factories/factory-as-code.mdx:61-67 — [SUGGESTION] The example validation-endpoint JSON response omits the validation_scope object that the live API actually returns alongside diagnostics/state_dependent_checks_not_run; requested resolution: add validation_scope to the example so it's fully representative.
• src/content/docs/factories/integrations/github.mdx:171 — [SUGGESTION] The new literal string 'it passes with "No factory configuration changes"' is an unverified verbatim UI/API message not itemized in the Unverified claims section; requested resolution: cite the exact source location for this string or soften to a paraphrase if the wording isn't guaranteed stable.

Verdict

Approve

[warp-agent-staging]

Overview

The validation paths, JSON Schema URLs, and Factory MCP operations remain covered, and the Markdown converter change is intact with passing targeted tests.

Concerns

• JSON Schema is still described as publishing the same rules as the YAML parser, which is not correct for parser-only and cross-file validation.
• The requested editorial rework remains incomplete: the new validation and MCP sections retain dense internal mechanics and duplicate detail rather than focusing on the user action and linking for depth.
• The PR has no current visual proof of the new rendered documentation flow.

Verdict

Found: 1 critical, 4 important, 0 suggestions, 0 nits

Request changes