Skip to content

Harden integration data updater against CI-red PRs - #2229

Open
David Pine (IEvangelist) wants to merge 1 commit into
mainfrom
ievangelist-fix-integration-api-generation
Open

David Pine (IEvangelist) wants to merge 1 commit into
mainfrom
ievangelist-fix-integration-api-generation

Conversation

@IEvangelist

Copy link
Copy Markdown
Member

Summary

#2212 failed Frontend validation even though #2049 closed with "All packages passed API generation". API generation was fine; the failures came from gaps between the updater and CI.

Root causes and fixes

  1. Updater-owned suppression list. Fill gaps in What's new in Aspire 17.0 #2177 hand-added two Declared export … entries to api-generation-failures.json. The updater rewrites that file from real generator failures, so it dropped them and the declared-export test failed. Those entries also turned off stale-version checks for both packages.
    • BaseKubernetesResource.Metadata was a resolver bug. ATS projects inherited members onto each derived handle (PersistentVolumeClaim.metadata, KubernetesManifestResource.metadata), but the resolver ignored baseTypeHierarchy. I added a base-hierarchy match that runs only when nothing matches the type directly, so base-type references that resolve today don't become ambiguous.
    • AzureApiManagementResource.Identity is a real upstream gap: it's marked [AspireExport], but ATS doesn't emit it. It now sits in an allowlist inside the test, next to the existing Radius exception.
  2. The updater gate didn't match CI. validate:api-data doesn't run the data-dependent unit tests. The updater now runs bun run test:unit. If tests fail, it still opens or refreshes the PR but leaves off the :octocat: auto-merge label and lists the failing tests in the body. On a later passing run, it puts the label back only if the bot removed it, so a maintainer's hold stays.
  3. Hardcoded upstream version. The standalone-dashboard sample test pinned nightly/dashboard:13.6, but samples.json is synced from upstream, which now uses 17.0. The test now accepts any consistent major.minor pin.
  4. Wrong PR body counts. Get-AreaCounts used git diff HEAD, which skips untracked generated files. chore: Update integration data and GitHub stats (11/20/26) #2212 reported "0 added, 69 removed" when those were version bumps.
  5. Tests changed tracked data. twoslash-types-generator regenerated the tracked aspire.d.ts in place. It now writes to a temp file. The updater also aborts if test:unit changes any file in the paths it stages.

The next updater run rebases #2212 onto main and picks these up.

Third-party links and affiliations

None

Validation

  • vitest run tests/unit/api-reference.vitest.test.ts tests/unit/dashboard-image-references.vitest.test.ts tests/unit/integration-data-updater.vitest.test.ts tests/unit/twoslash-types-generator.vitest.test.ts: passed (107 tests) on main data.
  • Same API-reference and dashboard tests against chore: Update integration data and GitHub stats (11/20/26) #2212's data (chore/update-integration-data): passed (81 tests). CI failed on that data before this change.
  • bun run lint: passed.
  • Full vitest run: 1558 passed, 1 failed (incremental-build, spawn bun ENOENT). It fails the same way on unmodified main on this Windows machine, so it's a local environment issue.
  • New updater fixture cases: unit-test failure withholds auto-merge and lists the failures; a test that changes data aborts publication; area counts include added files.

- Resolve inherited ATS capabilities projected onto derived handles
  (e.g. BaseKubernetesResource.Metadata) as a fallback tier.
- Move known TypeScript export gaps into the test; drop hand-added
  entries from the updater-owned api-generation-failures.json.
- Make the standalone-dashboard sample pin check version-agnostic.
- Count untracked generated files in the PR body area table.
- Run test:unit in the updater; on failure, list failures and withhold
  the auto-merge label; abort if tests rewrite generated data.
- Generate twoslash types to a temp file in tests instead of the
  tracked aspire.d.ts.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 20, 2026 21:58

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The updater can mistake a generic test-failure heading for bot-owned label removal and re-add a label on the next passing run, overriding a maintainer’s manual hold.

1 open issue
What changed in this PR

This PR hardens aspire.dev’s integration-data updater against validation gaps exposed by #2212, aligning its checks more closely with CI.

Changes:

  • Runs unit tests before publication, controls auto-merge labeling, detects test-induced data changes, and corrects generated-file counts.
  • Resolves inherited TypeScript API members and separates known export gaps from generator failures.
  • Makes dashboard image checks version-independent and isolates twoslash test output.
File Description
src/​frontend/​tests/​unit/​twoslash-types-generator.vitest.test.ts Generates declarations in a temporary directory with cleanup.
src/​frontend/​tests/​unit/​integration-data-updater.vitest.test.ts Covers test failures, data mutations, and added-file counts.
src/​frontend/​tests/​unit/​dashboard-image-references.vitest.test.ts Accepts consistent upstream major.minor image pins.
src/​frontend/​tests/​unit/​api-reference.vitest.test.ts Records known missing TypeScript exports in a dedicated allowlist.
src/​frontend/​src/​utils/​api-reference/​typescript-provider.ts Adds inherited-member resolution as a fallback.
src/​frontend/​src/​utils/​api-reference-core.ts Adds base-type hierarchy metadata to TypeScript handles.
src/​frontend/​src/​data/​api-generation-failures.json Removes two manually maintained export-gap entries.
src/​frontend/​scripts/​update-integration-data.ps1 Adds unit-test gating, mutation checks, and accurate file counts.
.github/​workflows/​update-integration-data.yml Uses test results to manage auto-merge labels and report status.

Review effort: Balanced

Comment on lines +189 to +190
elif [ "$AUTO_MERGE" = "true" ] && [ "$has_label" != "true" ] &&
grep -qF -- "$failure_marker" <<< "$previous_body"; then
@aspire-repo-bot

Copy link
Copy Markdown
Contributor

Frontend HTML artifact ready

The latest frontend build uploaded the frontend-dist artifact for PR #2229. Use the VS Code button below to open this PR with GitHub Artifacts Explorer and browse the built HTML locally.

VS Code: Open PR #2229 artifacts

This comment updates automatically when a new frontend build artifact is uploaded.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants