[miyaontherelay]

The Kind qualification sheet currently lives in machine-specific scripts and receipts. Add make local-acceptance and a tracked runner that emits signed pass/fail evidence, published application pins, source/cluster identity and timings without copying credentials or API bodies into the report.

This first increment implements four API checks: strict TLS/hostname refusal, session search/transcript, a ready model summary within the fixture window, and owner Usage200/member403 with successful member session access as a positive control. Preflight also checks tenant identity, all nine application copy/serve image pins and completed rollout availability.

ACCEPTANCE_SUITE=api runs this foundation. The default full suite fails closed for every remaining qualification adapter; every report says qualificationComplete: false. Ed25519 SSHSIG integrity verification requires an independently trusted public key and is explicitly not external attestation. Desktop/macOS interaction and the remaining automated checks are documented separately.

Validation: make local-acceptance-test passes11 regressions using a real local HTTPS server and real Ed25519 signing. Covers stale/missing summaries, invalid member authentication, redirects, secret-free failures, signature/payload/key tampering, credential modes/symlinks, foreign sites, stale/mixed rollout pins and tenant mismatch. CLI smoke verifies nonzero exit plus a signed failure report when the fixture is missing; Make path quoting and git diff --check pass. Tests are wired into Lint Charts. No ar-lap rollout, live acceptance run or wipe was performed for this PR.

Review fixes: archive any previous receipt before starting a new run, so a signing failure cannot leave a stale pass at the current path. HTTPS fixtures now use exact paths/query values and distinct tenant/session IDs, refusing unknown routes. Both have regressions. Rebased once onto main after the PostgreSQL readiness fix in#248 merged.

[coderabbitai]

Pre-merge checks: 2

Failed checks: 2 inconclusive

Check name	Status	Explanation	Resolution
Title check		This assessment has not completed. The check will be retried on the next pre-merge check run.
Description check		This assessment has not completed. The check will be retried on the next pre-merge check run.

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

[kjgbot]

Source review of exact head 8d9b027: no actionable findings. Independently passed all 9 offline regressions through the shared heavy queue (real local HTTPS and Ed25519 signing), plus additional stale/future-summary refusal checks and a real CLI missing-config smoke: exit 1, signed/verifiable failed receipt, qualificationComplete=false, no path leakage. Both documented shell blocks parse without execution; git diff --check passes. Member Usage denial requires a successful authenticated shared-session read first; redirects are not followed with cookies. Failure output uses fixed codes rather than subprocess/HTTP exception text. Signature verification requires the separately supplied trusted key and rejects replaced keys/payload tampering. Default full suite fails for missing adapters, and API-only success never claims complete qualification. Deployment/template pin checks include dashboard init-copy and rollout availability, with tenant matching and clean tracked source checks. These are source/fixture checks only: no ar-lap, cluster, real account, Desktop or live acceptance operations performed. The public-key signature proves integrity under the trusted development key, not external attestation or live observations.

[greptile-apps]

Confidence Score: 4/5

[Medium impact] The changes affect a bounded local Kind acceptance workflow, not deployed application behavior.

The PR appears safe to merge, with non-blocking improvements needed for stale reports and request-path tests.

Findings

1.  Old pass survives signing failure ▶
2.  Wrong request paths pass tests ▶

Fix with agent prompt

### Issue 1
ci/agent-relay/kind/acceptance/run.py:25-26
After a successful API run, an unreadable key or failed `ssh-keygen` command makes the next run exit 2 but leaves the old signed pass in `acceptance.json`. Someone opening the default report can mistake it for the latest result. Invalidate the current report before starting another run, or move previous reports to a separate history path. Add a regression that starts with a passing report and then fails signing.

### Issue 2
ci/agent-relay/kind/acceptance/test_acceptance.py:57-58
The HTTPS fixture returns a successful session list for every unrecognized path. Its other checks also match only path fragments. A typo in the session route, or a wrong workspace or session ID in a request, can therefore leave these tests green even though a real Kind run would fail. Match the expected paths and query values explicitly, and return 404 for unknown requests.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Adds a signed local Kind acceptance runner with four API checks.

• The local runner checks TLS, session reads, summaries, and Usage access.
• Preflight ties each acceptance run to a ready, pinned Kind deployment.
• The acceptance runner writes signed results and keeps unfinished qualification checks from passing.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Load private fixture config] --> B[Check tenant, pins and readiness]
    B -->|Pass| C[Run four API checks]
    B -->|Fail| D[Record failed API checks]
    C --> E[Add pending failures for full suite]
    D --> E
    E --> F[Sign report with Ed25519 key]
    F --> G[Write report and return result]

Loading

_{Reviews (1) · Last reviewed commit: "feat(kind): add signed acceptance runner..." · Reviewed by Greptile}