Skip to content

chore(tests): consolidate test patterns — shared Vitest base, central @sim/testing mocks, check:test-patterns, lean agent docs - #8309

Merged
waleedlatif1 merged 4 commits into
stagingfrom
chore/test-consolidation
Sep 25, 2026
Merged

waleedlatif1 merged 4 commits into
stagingfrom
chore/test-consolidation

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • One shared Vitest base (vitest.shared.ts): every workspace config extends it; clearMocks, restoreMocks, unstubEnvs, unstubGlobals are on (integration mode keeps per-file fixtures), globals: true removed. ~450 redundant vi.clearAllMocks/restoreAllMocks/unstubAll* hook calls deleted
  • @sim/testing is now the single home for mocks: ~220 central mocks (xMock + xMockFns, full export surface, faithful enums/error classes) and shared helpers (jsonResponse, createRouteContext, createDeferred, flushMicrotasks, collectStream, principal factories). ~2,300 hand-rolled vi.mock factories and ~600 inline principals across ~1,900 test files now use them
  • Global mocks extended so tests stop re-mocking them: @sim/db re-exports every schema table, @trigger.dev/sdk task() returns its config (+ runs.*, idempotencyKeys, AbortTaskRunError), drizzle sql is a spy, getMockLogger(name), stable telemetry spies. schemaMock table names are generated from packages/db/schema.ts (check:schema-mock guards drift)
  • New check:test-patterns audit (in check:audits): fails on a new local factory for a centrally mocked module, a re-mock of a global module, a local copy of a shared helper, a redundant hook, or a __tests__/ dir. Ratchet baseline holds the 145 deliberate exceptions (per-test component stand-ins, real-SQL inspection, …)
  • Naming: tests sit next to their source and are named after it — __tests__//tests/ dirs colocated, 26 topic-split files merged into their source's test
  • Every integration suite goes through the shared TEST_DATABASE_URL/TEST_REDIS_URL contract (IPv6 loopback works, credentialed Redis rejected); the OAuth token integration test no longer leaves a rate-limit bucket behind
  • Folds in the still-applicable v0.9.2: resource dropdown in org view, new library posts, removed unnecessary tests #8303 review feedback: typed condition-handler fixtures, restored MCP same-author secret-retention case, shared DB guard; declined the clearMocks-off claims (Vitest 5 default, now explicit in config) and the DB-name/test-rule suggestions
  • Agent docs made lean and consistent: root CLAUDE.md 484 → ~140 lines, global.md folded in, API rules in one sim-api-contracts.md, apps/sim/AGENTS.md → pointer, contradictions fixed (comments, hooks example, emcn paths, canvas sentences, repo tree). sim-testing.md 365 → 165 lines (mock table → discovery rule). One run command everywhere (bun run --cwd <ws> test); /ship and /cleanup gate tests through test-audit
  • Fixes: generate-docs no longer reads mock descriptions from test files into the integration catalog; path-independent secret-leak checks in knowledge-processing; best-of-three scaling ratio for the chat-content linearity checks

Type of Change

  • Other: test infrastructure, tooling, docs

Testing

  • bun run test (every workspace): apps/sim 34,269 passed; only failure was a pre-existing sim-cli lock-timing flake that passes 3/3 alone
  • bun run test:integration (Docker Postgres + Redis): apps/sim 96 files / 926 tests, packages/db 14 / 95 — all pass
  • bunx turbo run type-check 26/26, bun run lint, bun run check:audits (51), docs-manifest:check, block registry, check:migrations

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Sep 25, 2026 10:17pm UTC

Request Review

@greptile-apps

greptile-apps Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

[Low risk] Agent documentation and test infrastructure consolidation.

The PR should not merge until the feature-flag fixture is made compatible with per-test environment resets.

Findings

  1. P1 Feature flag fixture is removed ▶
  2. P2 Central mocks can evade audit ▶

Summary

The PR centralizes Vitest configuration and reusable test mocks, migrates tests to shared patterns, adds test-pattern and schema-mock audits, and updates test and agent documentation.

  • The shared environment-reset setting needs a corresponding update to a module-scope feature-flag fixture.
  • The new pattern audit does not discover every supported use of a central mock.

Reviews (1) · Last reviewed commit: "chore(tests): consolidate test patterns ..."

Comment thread vitest.shared.ts
Comment thread scripts/check-test-patterns.ts Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 2296 files

Confidence score: 5/5

  • In packages/testing/src/mocks/billing-core.mock.ts, duplicated pricing logic could drift from billing-subscription-utils.mock.ts as pricing changes; reuse a shared implementation to keep the mocks aligned.
Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/testing/src/mocks/billing-core.mock.ts">

<violation number="1" location="packages/testing/src/mocks/billing-core.mock.ts:4">
P3: This pricing implementation duplicates the existing centralized `billing-subscription-utils.mock.ts` helper, so future pricing changes can make the two mocks disagree. Reuse one shared implementation or extract the pure pricing helper instead of maintaining both copies.</violation>
</file>

Note: This PR contains a large number of files. cubic selects up to 200 of the highest-priority eligible files for this review, so some files may not have been reviewed.
Tip: instead of fixing issues one by one fix them all with cubic

Re-trigger cubic

Comment thread .agents/skills/add-managed-cli/SKILL.md
Comment thread .agents/skills/ship/SKILL.md Outdated
Comment thread .agents/skills/migrate-application-operation/SKILL.md Outdated
Comment thread packages/db/testing/test-infrastructure.ts
Comment thread packages/testing/src/mocks/api-server-routes.mock.ts Outdated
Comment thread packages/testing/src/mocks/admission-gate.mock.ts Outdated
Comment thread packages/testing/src/mocks/feature-flags.mock.ts Outdated
Comment thread packages/testing/src/mocks/billing-core.mock.ts Outdated
@waleedlatif1
waleedlatif1 force-pushed the chore/test-consolidation branch from 3dd94af to 0feb98a Compare September 25, 2026 22:15
@waleedlatif1
waleedlatif1 merged commit 4e17dce into staging Sep 25, 2026
39 checks passed
@waleedlatif1
waleedlatif1 deleted the chore/test-consolidation branch September 25, 2026 23:36

This branch was successfully deployed

1 active deployment
Preview — 0feb98a7 Deployed Sep 25, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant