Skip to content

Measured visual checks: counts in telemetry, their own kind on the dashboard (0.3.0-beta.5) - #120

Open
Karanvir1729 wants to merge 11 commits into
mainfrom
visual-telemetry
Open

Karanvir1729 wants to merge 11 commits into
mainfrom
visual-telemetry

Conversation

@Karanvir1729

Copy link
Copy Markdown
Collaborator

The runtime's measured visual checks (scroll locks left on, click blocks, invisible text, ads over content and the rest in src/visual/engine.ts) and their runtime fixes now reach the dashboard from production traffic, and the dashboard shows them. Before this, telemetry sent nothing about them, so a site full of stuck scrolls and frozen clicks looked clean on genclass.dev.

The owner approved the counts-only fields in VISUAL.md section 10 on 2026-10-27. Nothing else is sent: never a selector, element text, colour, URL, size or one event per finding.

Base: this PR contains PR 118

It is built on the live collector's deploy tree, 33df1e35: main merged with PR 118 (copy-prompts), with the conflicts resolved. That tree also has b0710d04, the demo estimate moved into stats(), which is on local main but was not pushed. "Fix them all" exists only in PR 118, and the collector deploy has to keep it. So this diff against main includes PR 118. Merging this PR lands both; merging PR 118 first shrinks this diff to its own three commits.

0.3.0-beta.4 belongs to PR 119 (the devtools pill, for the demo recording), so this targets 0.3.0-beta.5. Once PR 119 merges, main must be merged into this branch. Expect a conflict in the CHANGELOG's top section.

Runtime (packages/runtime)

  • Session event: visual ("off", "local" or "on", the first-load stub's setting) and fix (the app turned runtime fixes on: visual: { on: true, fix }). Guard or heal is still needed, and the same event's effectiveMode says which applies. These two fields are the only first-load bytes.

  • Summary event: once the checks have run, it carries:

    • counts.checks
    • counts.checksByRoute (at most 20 normalised paths)
    • counts.fixes { applied, undone }
    • runtime.visual { passes, ms, partial, tier }

    The checks' lazy chunk adds them (src/visual/bridge.ts withChecks: follow() wraps the telemetry client's push). First load carries no code for them, and a page where the chunk never loads sends summaries exactly as before.

  • Console notice: printed once by the chunk when telemetry is on. The telemetry notice itself is unchanged.

  • Docs: TELEMETRY.md, both READMEs, PRIVACY.md, VISUAL.md section 10 (approved, as built) and the CHANGELOG (Unreleased, 0.3.0-beta.5).

  • Tests: test/telemetry-visual.test.ts (new); test/visual-bridge.test.ts (its stub gains fix). No review tests were edited, and no model-visible text changed (situation-v2.3).

Collector (telemetry-worker/src/dashboard.ts)

  • Exact counters per page load from summaries, through the same high-water marks as detections: checks, check_kind, check_issue (kind TAB page), check_fix (applied, undone), check_loads, check_tier, check_partial and check_ms.
  • Coverage counters from session events: visual_setting and visual_fix.
  • Input filtering: only check-kind names and page paths are kept; junk keys are dropped.
  • No migration. The counters tables hold any metric.
  • stats() additions: series[].checks, checkSeries, previous.checks and totals.checksWithPage.

Dashboard (telemetry-worker/src/pages.ts)

  • Their own kind, never a model finding:
    • a "Measured check" badge;
    • a card per check with what was measured, the pages, the usual fix, and a Copy prompt that measures the page again with genclass_scan;
    • a hatched slot in Fig. 1, in the muted ink rather than a new hue, so the validated palette is unchanged;
    • a hatched segment in the Pages table;
    • the check fixes as their own row in "What GenClass did";
    • their coverage in Health, including how to turn them on when no page load ran them.
  • Counted as problems in the summary, the tiles, the rail, the app status (a check fix still in place counts as fixed live) and "Fix them all".
  • Sample dashboard: shows simulated checks.
  • Demo estimate: the Snow Day estimate now yields to the measured counts as soon as that project gets its first page load with measured checks (a check_loads row). Delete src/demo.ts after the recording.
  • Tests:
    • test/dashboard.test.ts: the measured visual checks.
    • test/pages.test.ts: the measured visual checks. Existing expectations now count problems as detections plus checks, because the sample has checks; two tests about the model alone strip them with noChecks().

Not run here

Nothing ran locally. The Mac only edits files, the train VM is deallocated, and starting it needs the owner's OK. This PR's CI runs tsc, the build, both runtime vitest steps, the worker tests and the first-load size check.

First-load size: /auto had 11 B of headroom under bundle.test.ts (108.589 of 108.6 KB on main). The two session fields will probably go over it. If CI fails there, raising LIMIT_KB by the measured growth needs the owner's OK. The 0.6 KB growth limit in first-load-size.mjs should pass.

The worker has no tsc step and no tsconfig, so its TypeScript is not type-checked anywhere.

For the owner (nothing here is published, tagged or deployed)

  1. Release: after PR 119's v0.3.0-beta.4, release this as v0.3.0-beta.5 through .github/workflows/release.yml. That means a version bump in packages/runtime/package.json and src/version.ts, the CHANGELOG heading, the exact install pins, then the v0.3.0-beta.5 tag. Then a worker PR like telemetry-worker: the newest runtime on npm is 0.3.0-beta.3; the act 3 rehearsal #114 for RUNTIME_ON_NPM and the CDN pins.

  2. Collector deploy: from main once this PR (and with it PR 118's copy-prompts, which is live) is merged: cd telemetry-worker && npx --yes wrangler@4 deploy. Deploying plain main without PR 118 would remove the live Fix them all and the Connect checkboxes. No D1 migration comes with this PR. Still run d1 migrations list genclass-dashboard --remote first.

  3. snowday-predictor.ca (Dizon's Cloudflare account):

    • pin @genclass/runtime@0.3.0-beta.5 in the script tag;
    • keep guard mode and data-visual="fix";
    • set the token to the "Snowday Predictor" project's key, gc_yTryldWvQz305xsmulOAmq. The live tag sends a key starting gc_Ypx0K, which is why that project has 33 page loads and no problems.

    Until it runs beta.5, the site sends no check counts (beta.4 doesn't), and that project's dashboard keeps the demo estimate.

Follow-ups (not in this PR)

  • @genclass/mcp's genclass_analytics production summary does not mention the checks yet.
  • The MCP view's tool table lists 8 tools and leaves out genclass_scan.

🤖 Generated with Claude Code

Karanvir1729 and others added 11 commits October 26, 2026 19:13
An account's project could not be deleted on its own, only with the whole account. DELETE /api/account/projects/<id> { confirm: "delete" } (signed-in browser only, the site's Origin) deletes an owned app with its stats, visits, the agent's reports and log, its integrations and export copies; its key stops counting at once (forgetToken), and its private link and agent link stop working. The dashboard's Project Settings gets a Delete project section (and a Danger zone entry in its menu): the first click asks, the second deletes and opens the Projects page with a notice.

Tests: the API (400 without the confirm, 403 for another Origin or a CLI session, 404 for another account, every table emptied, the key no longer counted, the other app untouched, a claimed app's links stop working) and the page script in happy-dom (asks once, then one DELETE; no section on a private-link dashboard).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n be done

The subtitle said "and deleting it" on private-link dashboards too, which have no Delete project section; it now says so in account mode only. The page test checks both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… type

- Connect dialog: choices above the prompt (Guard, Eager, Badge on the live site, Connect my coding agent to production, Deploy when done). The page runs installPrompt's own source (installPromptSource), so the copied prompt is the server's text for those choices; ticking the agent choice makes the agent link (one makeAgentLink for it and the MCP page).
- installPrompt: a site with no build step is covered by the prompt alone (the script tag with data-mode, data-visual, data-devtools; GENCLASS_CONFIG for what a tag cannot carry; the agent link from the CDN on local hosts); values one response writes together go in one atom; with the agent link, claude mcp add with GENCLASS_DASHBOARD and GENCLASS_REPORT=on in the agent's own settings; deploy with the project's own command when chosen.
- Problems and the overview: Fix them all copies one prompt for every kind of problem and the dev page's checks; every fix prompt has the agent say what to do on the page and wait for the loads itself (sleep 10 between checks, up to 5 minutes), so nobody types done.
- /account/new?name= fills in the project's name, through sign-in (the page check's Fix it with GenClass names the checked site).

Tests: the choices produce exactly installPrompt's text in the page, the agent choice posts once and names the link, the prompt's no-build path and atom step, Fix them all, the name prefill (escaped, carried through /login).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ection 10)

Approved by the owner on 2026-10-27, counts only. The session event gains `visual`
("off" | "local" | "on") and `fix`; once the checks' chunk has run, every summary
event carries counts.checks, counts.checksByRoute (at most 20 normalised paths),
counts.fixes { applied, undone } and runtime.visual { passes, ms, partial, tier }.
The summary counts are added by the chunk (src/visual/bridge.ts withChecks), so
first load grows only by the two session fields. Never a selector, element text,
colour, URL, size or a per-finding event. With telemetry on, the chunk prints one
console notice saying what of the checks is sent.

TELEMETRY.md, both READMEs, PRIVACY.md, VISUAL.md section 10 and the CHANGELOG
updated. Tests: test/telemetry-visual.test.ts. No model-visible text changed
(situation-v2.3).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The collector counts the runtime's visual check counts exactly per page load
(checks, check_kind, check_issue, check_fix, check_loads, check_tier,
check_partial, check_ms from summaries; visual_setting, visual_fix from session
events); no migration. stats() gains series.checks, checkSeries and
previous.checks.

The dashboard shows them as measured checks, never model findings: a card per
check with a "Measured check" badge (what was measured, where, the usual fix, a
Copy prompt that measures the page again with genclass_scan), a hatched slot in
Fig. 1, their pages in the Pages table, their fixes in What GenClass did, their
coverage in Health. They count in the app's status and in Fix them all. The
sample shows simulated checks. The Snow Day demo estimate yields to the measured
counts once its project gets a page load with measured checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
0.3.0-beta.4 is the devtools pill (PR 119), released first for the demo
recording; the telemetry counts follow in the next beta.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants