Repository navigation
docs(ai): say what the docs assistant can't do and where to run the Celopedia install - #2359
Conversation
GigaHierz
left a comment
There was a problem hiding this comment.
head eb22ee3 · setup gate matched API head (no STALE); branch is 7 commits behind main, 1 ahead, merge-tree clean, main did not touch either changed file; mint broken-links --check-redirects: success; check-orphans: none; gh pr checks: all pass (Mintlify Deployment skipping); mint dev render on desktop and 412x915: both pages load, 0 console errors, no horizontal overflow, anchors present
Verdict: REQUEST-CHANGES
Tier: rendered doc change with factual claims about a live system, full pass. Findings are all wording precision; no claim about the assistant is false and nothing is security-sensitive.
Findings (severity-ordered)
1. Medium · build-on-celo/build-with-ai/celopedia.mdx:28 and :35 · "in your terminal" does not say where, and one command does not install for Claude Code
Defect: the page says to run npx skills add celo-org/celopedia-skills "in your terminal" and then lists Claude Code as supported, but the skills tool installs project-scoped by default (./<agent>/skills/, global only with -g), and the Celopedia README says Claude Code only discovers .claude/skills/ or ~/.claude/skills/, so the bare command alone "is not enough" there (it needs a symlink or a manual copy).
Failure scenario: a reader opens any terminal (home directory, or a new empty folder), pastes the command, sees success, restarts Claude Code as the Note advises, and the skill never appears. The page the PR set out to clarify still leaves the "where" half unanswered, and the restart advice cannot fix the Claude Code case.
Fix shape: line 28, "Run this in a terminal, from the root of the project you want to use it in:"; add one sentence that Claude Code needs an extra step described in the repository README. Do not copy the symlink recipe in; keep one fact in one place.
2. Low · build-on-celo/build-with-ai/celopedia.mdx:35 · "Pasting it into a chat ... won't install it" is broader than what is true
Defect: the code only supports the narrower claim that the Ask AI assistant cannot run it (it has only the docs MCP tools, route.ts tool filter with submit_feedback denylisted, no shell). A coding-agent chat such as Claude Code or Codex has a shell and will run a pasted command if asked.
Failure scenario: a reader using a coding agent reads "pasting it into a chat won't install it" and does not try the one path that works for them.
Fix shape: "The command has to run in a terminal. The Ask AI assistant on these docs can't run it."
3. Low · build-on-celo/build-with-ai/use-docs-with-ai.mdx:206 · pronoun now points at the wrong thing
Defect: the new paragraph (line 204) ends on the link to the Celo MCP Server, so the next paragraph's "It searches the same content as the MCP server above" reads as if "It" is the Celo MCP Server, and "the MCP server above" is ambiguous between the docs MCP server and the Celo MCP Server linked one line earlier.
Failure scenario: a reader concludes the live-chain-data server searches the docs, the opposite of the point of the paragraph before.
Fix shape: line 206 "The assistant searches the same content as the docs MCP server above, ..." (or move the limits paragraph after it).
4. Low · build-on-celo/build-with-ai/use-docs-with-ai.mdx:247 vs :33 and :204 · feedback tool is an unlisted limit
Defect: the page lists submit_feedback as a docs MCP tool and ends with "Ask your assistant to use the submit_feedback tool", but the in-page assistant denylists it (route.ts TOOL_DENYLIST), and the new paragraph names the in-page assistant as a product while its limits list omits this. The text is pre-existing, but the PR makes "your assistant" ambiguous.
Failure scenario: a reader asks the Ask AI assistant to submit feedback, and it cannot.
Fix shape: line 247 "Ask your coding assistant (with the docs MCP server connected) to use ...". Do not document the denylist or its reason.
Verified good
Each limitation checked against celo-org/docs-ai-assistant main:
- "can only search these docs":
app/api/chat/route.tsbuilds tools fromcreateMCPClientonhttps://docs.celo.org/mcpminussubmit_feedback; the system prompt says "using ONLY the documentation tools". Accurate (the filesystem query tool reads pages, so "search" slightly understates). - "refuses questions unrelated to Celo": last system-prompt rule, plus
lib/topic-gate.tsreturning a refusal before retrieval. Caveat: the gate fails open on a classifier error (catch block), so "refuses" is intent plus a best-effort gate; the page does not promise more. - "can't see your wallet or account": the POST body reads only
messagesandescalate; no client or wallet context is sent. True. - "can't send transactions, install skills or run commands": no tool beyond the docs MCP tools is registered. True.
- "doesn't read live chain data": same tool set. True.
- "Celo Docs Assistant" name and Ask AI bottom-right:
public/widget.jsline 194 (panel title), line 191 (launcher "Ask AI"),.launcher { bottom: 20px; right: 20px }. - "Try a more thorough answer" re-runs on a stronger model (existing text):
escalateselects the escalation tier. - Celopedia command:
npx skills add celo-org/celopedia-skillsis the first install line in the celopedia-skills README and appears in the agent-skills README; the skills tool's own docs givenpx skills add <owner/repo>as the basic usage, run in a terminal. Real documented usage; project-scoped unless-g(finding 1). - Not published, correctly: rate limit, input cap, 12-message window, tool names the assistant uses, denylist, topic-gate behaviour and fail-open. Nothing on the changed pages helps an attacker beyond what the widget already shows; consistent with docs-ai-assistant issue #3 and docs#2302. Documenting the 12-message memory or input cap for users is optional; if added, keep the numbers out.
- AGENTS.md: headings sentence case and unchanged, callout is a
Note, no promotional wording or product comparison added, links root-relative and/build-on-celo/build-with-ai/mcp/celo-mcpexists,## Relatedpresent, no new anchors added (#ask-a-question-in-the-docsand#install-celopediaresolve in the rendered pages). - PR title and body contain no private-source references; refs are public issues.
What the PR got right
- Targets a real gap (repeated unanswered questions) with the smallest possible change: two files, five lines.
- Every limit it states is backed by a line of code, and it avoids publishing numbers or internals.
- Pointing to the Celo MCP Server for live chain data gives the reader the next step instead of a dead end.
- Merges cleanly; CI green.
Pre-existing, not counted against this PR
use-docs-with-ai.mdxdescription ends with a period and is a long multi-clause sentence (AGENTS.md section 3 wants one sentence, no trailing period); the first paragraph does not name its reader. Candidates for the broader migration.
Merge-order hazards
None. Neither file changed on main since the merge base (the 7 main commits touch ANALYTICS.md, assistant.js, docs.json, wallets, operators, x402 and others). Squash merge; only the "branch up to date" gate applies.
eb22ee3 to
584a330
Compare
|
All four are in 584a330, rebased onto current main.
|
GigaHierz
left a comment
There was a problem hiding this comment.
head 584a330 · API headRefOid == refs/pull/2359/head (matched after re-fetch); 2 commits on merge-base 92b8cc6, 4 behind origin/main (a12466b), git merge-tree clean; on the merge into main: mint broken-links --check-redirects "no broken links found", check-orphans "No orphan pages found"; CI "Check for broken links" pass; mint dev on 3422 at 1280 and 412x915: both pages render, zero console errors after warm-up, no horizontal overflow, anchors present
Verdict: REQUEST-CHANGES
Status of earlier findings
- MEDIUM, "in your terminal" does not say where / bare command does not install for Claude Code: FIXED. celopedia.mdx:28 now reads "Run this in a terminal, from the root of the project you want to use it in:", and the Note says "Claude Code needs an extra step after this command, described in the repository README". Verified at source: the celopedia-skills README has a
### Claude Codeheading under Install that saysnpx skills addalone is not enough (symlink or copy under.claude/skills/), so the anchor resolves; the skills CLI README lists project scope (./<agent>/skills/) as the default and-gfor global, so "from the root of the project" is correct. The symlink recipe is not copied in (one fact, one place). - LOW, "pasting it into a chat won't install it" too broad: FIXED. Now "The command has to run in a terminal. The Ask AI assistant on these docs can't run it." That is the claim the assistant code supports (docs MCP tools only, no shell), and it no longer says anything about coding agents.
- LOW, pronoun at use-docs-with-ai.mdx now points at the Celo MCP Server: FIXED. The paragraph at :206 now begins "The assistant searches the same content as the docs MCP server above", so neither "It" nor "the MCP server" can bind to the Celo MCP Server linked in :204.
- LOW,
submit_feedbacktold to "your assistant": FIXED. :247 now reads "Ask your coding assistant (with the docs MCP server connected) to use thesubmit_feedbacktool". The tool is still listed in the docs MCP table at :33, which is accurate for the MCP server, and the denylist is not published.
New findings
- LOW · PR body, paragraph 1 · The body says "four people pasted
npx skills add celo-org/celopedia-skillsinto the chat", and the three linked weekly issues do not reproduce that number: the exact command appears 3 times (#2320 once, #2324 twice) and a looser count of install-request variants is 5 (#2324 four rows, #2320 one row), with none in #2343.
Failure scenario: the body is the squash-commit record and the only place the motivating evidence lives; a later reader re-counting from the issues gets a different number and stops trusting the other claims in it.
Fix shape: edit the PR body to the number you can reproduce (for example "three people pasted the exact command, two more asked it to install the skill") or cite only the issue numbers. No commit needed. Clears the finding.
Verified good
Each limitation re-read at current celo-org/docs-ai-assistant main (last commit 2026-09-02, widget changes only):
- "can only search these docs": chat route builds its tool set from the docs MCP endpoint at https://docs.celo.org/mcp, minus
submit_feedback(TOOL_DENYLIST), and the system prompt says to answer using only the documentation tools. True. - "refuses questions unrelated to Celo": last system-prompt rule, plus the topic gate. The gate is best-effort, but the page states intent, not a guarantee. True as worded.
- "can't see your wallet or account, send transactions, install skills or run commands, doesn't read live chain data": the request body reads only
messagesandescalate; no client or wallet context; no tool other than the docs MCP tools is registered. True. - "Celo Docs Assistant" and the bottom-right "Ask AI" launcher: public/widget.js:194 (panel title), :191 (launcher label), :29 (
bottom: 20px; right: 20px). - Celo MCP Server link target /build-on-celo/build-with-ai/mcp/celo-mcp exists and its title is "Celo MCP Server"; the page's own sentence at :38 already makes the same split (docs vs live chain data).
npx skills add celo-org/celopedia-skillsis the install line in the celopedia-skills README; it is the only occurrence ofskills addin the repo's mdx, so no other page needs the same fix.- Nothing published that helps an attacker: no rate limit, input cap, message window, tool names beyond the existing docs MCP table, denylist or gate behaviour.
- AGENTS.md: headings unchanged and sentence case, callouts are
Notecomponents, no promotional wording or product comparison added, links root-relative without extension,## Related/closing structure untouched,#install-celopediaand#ask-a-question-in-the-docsids exist in the rendered pages. - Other claims in the body: "the widget calls itself Celo Docs Assistant" true; refs are open public issues (#2320, #2324, #2343), and
RefsnotClosesis right for weekly-gap issues.
What the PR got right
Every one of the four earlier findings was fixed in a single follow-up commit with the wording I suggested or better; the author also checked both READMEs before writing the Claude Code sentence and kept the symlink recipe in the README rather than duplicating it. The PR is two files and a handful of lines, and each stated limit traces to a line of code.
Merge-order hazards
- None against main: merges cleanly into a12466b, and the 4 main commits since the merge base do not touch either file.
- #2368 (the Build tab merge) moves
use-docs-with-ai.mdxtobuild/agents/use-docs-with-ai.mdx(it appears in that PR's file list), and presumablycelopedia.mdxas well. Whichever merges second conflicts or needs a path re-apply: if this PR lands first, #2368 carries the new text across the rename (git handles it as a rename in most cases); if #2368 lands first, this PR must be rebased onto the new paths. The/build-on-celo/build-with-ai/mcp/celo-mcplink added here will also need the newbuild/path then. - Pre-existing, not counted: use-docs-with-ai.mdx description ends with a period and runs to several clauses (AGENTS.md section 3), and the first paragraph does not name its reader.
GigaHierz
left a comment
There was a problem hiding this comment.
head 584a330 · fetch matched headRefOid; main 7b8f97c (#2368 layout); merge-tree clean via rename detection; trial merge: broken-links --check-redirects pass, check-orphans pass, CI green; the merged use-docs-with-ai.mdx still links /build-on-celo/build-with-ai/mcp/celo-mcp (redirect only, rendered href observed); mint dev (smoke test) 0 console errors
Verdict: REQUEST-CHANGES
Status of earlier findings
- PR body "four people pasted the install command": FIXED. The body now reads "pasted ... three times (#2320, #2324), with two more requests in #2324 asking it to install the skill". Re-counted from the issue tables (data only): the exact command appears once in #2320 and twice in #2324 (one of those two carries a trailing install request), and two further looser variants appear in #2324. That is 3 exact plus 2 looser, which is what the body says. Its first paragraph also cites #2343 for the "what is it / what can it do" topic, and #2343 does contain such a question.
- Earlier wording findings (terminal location and Claude Code, "pasting into a chat", pronoun at the limits paragraph,
submit_feedbacktold to "your assistant"): still fixed in the merged tree; the text is as in the diff below. - CI: Check for broken links, GitGuardian and StepSecurity pass; Mintlify Deployment skipping. No other review threads or comments are open.
New findings
- LOW ·
build/agents/use-docs-with-ai.mdx:204(the path after #2368; todaybuild-on-celo/build-with-ai/use-docs-with-ai.mdx:204) · The new sentence links the Celo MCP Server as/build-on-celo/build-with-ai/mcp/celo-mcp. On current main that path exists only as adocs.jsonredirect (/build-on-celo/build-with-ai/:slug*to/build/agents/:slug*); the page itself isbuild/agents/mcp/celo-mcp.mdx. Failure scenario: after the PR merges, the link resolves through a redirect,mint broken-links --check-redirectsstill passes so nothing flags it, and it is the only internal/build-on-celo/link left in any.mdxfile; AGENTS.md section 2 asks for links to the final page and the next redirect clean-up would break it. Fix shape: one line, change the href to/build/agents/mcp/celo-mcp(done as part of the rebase below).
Verified good
- Trial merge (
git merge-tree --write-tree origin/main refs/remotes/pr/2359, exit 0): the edits follow the renames tobuild/agents/celopedia.mdxandbuild/agents/use-docs-with-ai.mdx; diff against main is 2 files, +7/-5, no leftovers underbuild-on-celo/. - Each stated limitation re-read at
celo-org/docs-ai-assistantmain (newest commit 2026-09-02, UI-only, nothing changed since the last review):- Only searches the docs:
app/api/chat/route.tsbuilds its tools from the MCP client onhttps://docs.celo.org/mcpminusTOOL_DENYLIST(submit_feedback); system prompt says to answer using ONLY the documentation tools. True. - Refuses questions unrelated to Celo: final system-prompt rule plus a topic gate that fails open on a classifier error. The page states the intent, not a guarantee. True as worded.
- Cannot see the wallet or account, send transactions, install skills, run commands, or read live chain data: the request body carries only
messagesandescalate, and no tool beyond the docs MCP tools is registered. True. - Name and launcher:
public/widget.jspanel title "Celo Docs Assistant" (:194), launcher "Ask AI" (:191), fixed bottom 20px / right 20px (:29). True. - Install line:
npx skills add celo-org/celopedia-skillsis the first install line in the celopedia-skills README, which also has a### Claude Codeheading saying that command alone is not enough there, so the#claude-codeanchor in the Note resolves.
- Only searches the docs:
- Not published, correctly: rate limit, input cap, message window, denylist reason, gate behaviour.
- Rendered
/build/agents/use-docs-with-aion the trial merge withmint devon :3532 (smoke test): "Celo Docs Assistant" present, 0 console errors, no horizontal scroll at 1280. Celopedia page serves the new "Run this in a terminal" line. - Style: sentence-case headings untouched, callout stays a
Note, no promotional or comparative wording, links root-relative without extension (apart from finding 1),## Relateduntouched. - PR title and body contain no private-source references;
Refs(notCloses) is right for weekly-gap issues.
What the PR got right
Two files and a handful of lines, aimed at a repeated unanswered question; every limit it states traces to a line of code; no internals or numbers published; each earlier finding fixed with a commit or a body edit.
Merge-order hazards and rebase work
- Main has the layout change (#2368):
build-on-celo/build-with-ai/{celopedia,use-docs-with-ai}.mdxare nowbuild/agents/{celopedia,use-docs-with-ai}.mdx. There is no textual conflict; git and GitHub follow the rename and the trial merge is clean. - Author's rebase work (one commit): rebase onto current main and change the href on
build/agents/use-docs-with-ai.mdx:204to/build/agents/mcp/celo-mcp; re-runmint broken-links --check-redirectsandgrep -rn 'build-on-celo' --include='*.mdx' .(expect no hits). - No other content hazard: neither page is touched by other PRs reviewed in this batch.
The old /build-on-celo/ path only resolves through a redirect since the Build tab move.
584a330 to
b3f9b5c
Compare
|
Rebased onto current main. Git followed the rename, so both edits now sit in The Celo MCP Server link is fixed in b3f9b5c and now points at
|
GigaHierz
left a comment
There was a problem hiding this comment.
head b3f9b5c · API headRefOid == refs/pull/2359/head (fetched, matched); merge base is current origin/main 7f5d9bb, git merge-tree --write-tree origin/main refs/remotes/pr/2359 exit 0 (no conflict, 3 commits, 2 files); git grep build-on-celo -- "*.mdx" returns nothing; mint broken-links --check-redirects "no broken links found", mint validate "build validation passed", check-orphans.sh "No orphan pages found"; gh pr checks: Check for broken links, GitGuardian, StepSecurity pass; mint dev on :3722 (own server, restarted once because the shared cache first served pre-PR text; then served the PR text): 0 console errors on /build/agents/use-docs-with-ai, no horizontal overflow
Verdict: APPROVE
Status of earlier findings
- LOW, link through
/build-on-celo/build-with-ai/mcp/celo-mcp: FIXED. The sentence atbuild/agents/use-docs-with-ai.mdxnow links/build/agents/mcp/celo-mcp; the served page renders that href, the page exists (build/agents/mcp/celo-mcp.mdx, title "Celo MCP Server", nav entrydocs.json:195). The author's claim holds:git grepfinds nobuild-on-celostring in any.mdx(it remains only indocs.jsonredirect sources). - Earlier wording and body-count findings (terminal location and Claude Code, "docs MCP server" wording,
submit_feedbacktold to a "coding assistant", install-command count in the body): still FIXED in the head diff.
New findings
None.
Verified good
- Each stated assistant limitation, re-read at
celo-org/docs-ai-assistantmain (newest commit 2026-09-02, UI-only; no change since the last review):- Searches only the docs:
app/api/chat/route.tsbuilds its tool set from the MCP client athttps://docs.celo.org/mcpminusTOOL_DENYLIST(submit_feedback); the system prompt says to answer using ONLY the documentation tools. True. - Refuses unrelated questions: system-prompt rule "Refuse questions unrelated to Celo or its documentation", plus a topic gate. Stated as intent, not a guarantee; true as worded.
- Cannot see a wallet, send transactions, install skills, run commands or read live chain data: no tool beyond the docs MCP tools is registered. True. The Celo MCP Server page covers balances, blocks, transactions, gas and governance, so the pointer for live data is right.
- Name and button:
public/widget.jspanel title "Celo Docs Assistant", launcher "Ask AI", fixed bottom-right. True.
- Searches only the docs:
- Celopedia:
npx skills add celo-org/celopedia-skillsis the first install line of the celopedia-skills README, which has a### Claude Codeheading sayingnpx skills addalone is not enough there, so#claude-coderesolves; the served Note rendershttps://github.com/celo-org/celopedia-skills#claude-code. - Nothing security-sensitive is published: no rate limit, input cap, message window, denylist reason or gate behaviour; the
submit_feedbackNote only tells coding assistants with the docs MCP server connected to use the tool, as before. - Style: sentence-case headings untouched, callout stays a
Note, no promotional or comparative wording, links root-relative without extension,## Relateduntouched. Served/build/agents/celopediashows "Run this in a terminal" and the new Note. - PR title and body contain no private-source references;
Refs, notCloses, for the weekly-gap issues.
What the PR got right
Two files and a handful of lines aimed at a repeated unanswered question; every limit it states traces to a line of the assistant's code; no internals or numbers published; each review round was answered with a commit or a body edit.
Merge-order hazards
None. The branch sits on current main and follows the #2368 renames. No other open PR I know of edits build/agents/celopedia.mdx or build/agents/use-docs-with-ai.mdx.
Two topics keep showing up among the docs assistant's unanswered questions (#2320, #2324, #2343). People ask the assistant what it is and what it can do, and people pasted
npx skills add celo-org/celopedia-skillsinto the chat three times (#2320, #2324), with two more requests in #2324 asking it to install the skill. Neither page answered either point. The widget calls itself "Celo Docs Assistant", butuse-docs-with-ai.mdxnever used that name. That page now names the assistant and says what it can't do.celopedia.mdxnow says the install command has to run in a terminal.The limits come from celo-org/docs-ai-assistant. In
app/api/chat/route.ts, the only tools are the ones served byhttps://docs.celo.org/mcp, withsubmit_feedbackdenylisted, and the system prompt says to answer using only those tools and to refuse questions unrelated to Celo.lib/topic-gate.tsadds a classifier for off-topic questions. The widget's title comes frompublic/widget.js.Refs #2320, refs #2324, refs #2343.
mint broken-links --check-redirects,mint validateandscripts/check-orphans.shpass on the rebased head.