Skip to content

docs: align README and docs claims with default behavior; correct telemetry privacy, churn and funnel wording (CLAIM-01..05, TEL-01/06/08 docs) - #1585

Merged
Dumbris merged 15 commits into
mainfrom
docs/assess-claims-and-telemetry-docs
Oct 12, 2026
Merged

Dumbris merged 15 commits into
mainfrom
docs/assess-claims-and-telemetry-docs

Conversation

@Dumbris

@Dumbris Dumbris commented Oct 11, 2026

Copy link
Copy Markdown
Member

Problem

README and docs entry pages promised more than default behavior, quoted unattributed percentages, and the telemetry page contradicted receiver behavior and over-read two fields. Docs-only; no code or schema changes.

Findings and revalidation (on origin/main c9c1ad2)

  • CLAIM-01: README "scans ... before anything leaves the machine" was wrong. Detection is async and flags the activity log only; response redact/block is opt-in, text-only, and does not cover arguments. README, intro, sensitive-data-detection, activity-log and output-sanitisation now say so.
  • CLAIM-02: README/intro called quarantine a TPA blocker. Reworded to withholding unapproved servers/tools and holding changed definitions; added "What quarantine does and does not cover" (detector misses, implementation changes outside the definition hash, trust modes scan/auto, quarantined:false, clean scan is not proof).
  • CLAIM-03: removed the unattributed 99% / 43% accuracy figures. Added a README section with first-party bench conditions. Revalidation found bench/README.md's 17%/43% table is stale: the current go run ./bench/cmd/bench shows the proxy menu larger than the 45-tool baseline (2625 and 1874 vs 1730 tokens). README and bench/README.md now carry the current numbers and say it measures initial menu size only (no latency, workflow or accuracy claim).
  • CLAIM-04: "one function" replaced by "a small set of discovery and execution tools". tools/list checked per mode: retrieve_tools 12, code_execution 7, direct exposes upstream tools plus describe_tool (README notes the direct exception).
  • CLAIM-05: README quick start now follows add -> review fetch -> rescan -> approve -> harmless read -> activity, links the published quick-start, and qualifies the activity log (bounded retention and per-record size cap, not a compliance guarantee).
  • TEL-01 (docs): telemetry.md no longer says IPs are stripped; it states the receiver does not store the raw IP, that older rows may still hold one, and that clearing history is a separate step.
  • TEL-06 (docs): previous_shutdown/last_error_code described as prior-session evidence, not the cause of terminal silence.
  • TEL-08 (docs): funnel guidance separates prevalence from conditional conversion, notes direct routing can activate without retrieve_tools, and asks for denominators and unknown counts.

Local verification

Clean scratch instances (scratch HOME/data, ports 18985-18989, telemetry off). Full transcripts, raw outputs and screenshots under docs/qa/assessment-2026-10-11/evidence/c4/ (not committed).

  • README path executed on a rebuilt binary against a clean instance: add, review, approve, harmless read (Context7 resolve-library-id) succeeded, activity row visible.
  • Gate: calls to a quarantined server return QUARANTINED_SERVER_BLOCKED with a policy_decision row; unapproved tools return TOOL_BLOCKED.
  • Fake canaries: default config forwards argument and response unchanged and flags both in activity; response_action: redact redacts; response_action: block blocks with isError. Redact replaced only the PEM BEGIN line and left the key body (documented as best-effort; a possible separate product defect).
  • Docs site built with npm run build (no broken links), served locally, screenshots taken.
  • go test ./cmd/release-gate/... ./bench/... pass.

Compatibility and rollout dependencies

  • Docs only. The telemetry privacy wording describes the receiver contract from the assessment (raw IP not persisted). The receiver change (smart-mcp-proxy/mcpproxy-telemetry, branch fix/assess-ip-minimization) had no commits yet when this was written, so merge this after that receiver PR is deployed, or reconcile the exact stored fields (geography granularity) with it first.
  • Not changed here: docs/getting-started/quick-start.mdx approve step (security approve can refuse on current main), docs/img/mcpproxy-architecture.jpeg text, and the public website repo (src/pages/telemetry.astro:104,135 still say IPs are stripped; src/pages/docs/features.astro:29 still cites ~99%).
  • Other PRs edit the telemetry.md activation section; this one touches only privacy, churn and funnel text.

🤖 Generated with Claude Code

…emetry privacy, churn and funnel wording (CLAIM-01..05, TEL-01/06/08 docs)
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Deploying mcpproxy-docs with  Cloudflare Pages  Cloudflare Pages

Latest commit: 57b450c
Status: ✅  Deploy successful!
Preview URL: https://156ecc95.mcpproxy-docs.pages.dev
Branch Preview URL: https://docs-assess-claims-and-telem.mcpproxy-docs.pages.dev

View logs

@github-actions

github-actions Bot commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

📦 Build Artifacts

Workflow Run: View Run
Branch: docs/assess-claims-and-telemetry-docs

Available Artifacts

  • archive-darwin-amd64 (31 MB)
  • archive-darwin-arm64 (28 MB)
  • archive-linux-amd64 (19 MB)
  • archive-linux-arm64 (17 MB)
  • archive-windows-amd64 (31 MB)
  • archive-windows-arm64 (27 MB)
  • frontend-dist-pr (0 MB)
  • installer-dmg-darwin-amd64 (27 MB)
  • installer-dmg-darwin-arm64 (24 MB)
  • smart-mcp-proxymcpproxy-goCW1JG4.dockerbuild (0 MB)

How to Download

Option 1: GitHub Web UI (easiest)

  1. Go to the workflow run page linked above
  2. Scroll to the bottom "Artifacts" section
  3. Click on the artifact you want to download

Option 2: GitHub CLI

gh run download 38206388404 --repo smart-mcp-proxy/mcpproxy-go

Note: Artifacts expire in 14 days.

@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@Dumbris
Dumbris merged commit f85ff0b into main Oct 12, 2026
45 of 46 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants