Skip to content

ci: measure the integration splits and a Windows toolchain cache (do not merge) - #5805

Closed
ric-oliv wants to merge 20 commits into
mainfrom
ci/measure-integration-splits-and-toolchain-cache
Closed

ric-oliv wants to merge 20 commits into
mainfrom
ci/measure-integration-splits-and-toolchain-cache

Conversation

@ric-oliv

Copy link
Copy Markdown
Member

Throwaway draft to measure three more changes on top of #5804's state. I'll close it after three runs and fold each change into the PR it belongs to.

  1. The win-x64 and linux-x64 integration tests run in their own jobs, as the win-arm64 ones do since ci: measure the win-arm64 integration job on top of all CI changes (do not merge) #5803. On ci: measure the win-arm64 integration job on top of all CI changes (do not merge) #5803 the win-x64 leg was the slowest check at 26.6 to 28.4 minutes, and linux-x64 ends at about 23. Both runners have 4 cores, which the unit tests keep busy next to the integration tests.
  2. verify-api's Windows job uses the fast drive step, the last Windows job without it.
  3. The Windows jobs share their .NET toolchain through a cache. Every Windows job installs the same SDKs next to the workspace, and the arm64 ones the Android SDK, which their image lacks: 1 to 3.5 minutes per job. The environment action restores both from one cache per architecture and skips the installs on a hit. The workload restore still runs, for the job's own projects. The build legs save the cache on a miss. For this probe they save on the PR, so attempt 1 saves and attempts 2 and 3 restore. The real change would save on main only.

What I'm checking:

#skip-changelog

ric-oliv and others added 20 commits October 25, 2026 00:01
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Taking the app samples out of the platform solution filters meant msbuild.exe no longer built the MAUI sample, or any other Android or Windows app. The Samples job still builds it with the .NET CLI. The MSBuild job finishes well before the slowest build leg, so it builds the sample again, after the solution filter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The win-arm64 runner has 4 cores. With the unit tests in the background next to the integration tests, Pester took 876.6 s on the combined draft, against 684 s with the unit tests in the foreground, and aot.Tests went from 31 s to 2.6 minutes. That made the leg the slowest check, at 30 to 32 minutes.

The macOS MAUI job becomes a matrix with a win-arm64 entry. It builds and packs only the packages the integration tests use, then runs all of them, so the leg keeps the build, the unit tests and the coverage upload.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Without a solution filter, the environment action restores the workloads for every project in Sentry.slnx. That took 184 s in the win-arm64 integration test job, which only builds 5 projects. The action now takes the job's project files, and skips the Android test app's workloads for them, as it does for a solution filter without the test app.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…samples job

The Samples job on win-x64 still installed workloads as MSI packages under Program Files, 163 s, and restored the MAUI sample from the NuGet cache in the user profile, 180 s. It now uses the same drive step as the other Windows jobs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The environment action now requires the solution filters or project files the job builds, and installs only their workloads. Jobs that build everything pass Sentry.slnx. Before, every job without a filter restored the workloads for all of Sentry.slnx, and CodeQL, format-code and verify-api ran a second, unfiltered workload restore after the action. The Playwright job restores the workloads of its test project only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The win-arm64 leg got 14 minutes faster when its integration tests moved to their own job, because its 4 cores couldn't run them next to the unit tests. The win-x64 and linux-x64 runners have 4 cores too, and their legs were the slowest after that, so they get entries in the integration-test matrix as well. The legs keep the build, the unit tests and the unsupported AOT test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…API approval job

It was the last Windows job without the drive step.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every Windows job installs the same .NET SDKs next to the workspace, and the arm64 ones the Android SDK, which their image lacks. That took 1 to 3.5 minutes per job. The environment action now restores both from one cache per architecture, keyed on global.json and the action itself, and skips the installs on a hit. The workload restore still runs, so each job adds whatever its projects need that the cached copy lacks. The build legs save the cache on a miss, because they install the most workloads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.13%. Comparing base (b41b96c) to head (e683a10).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5805   +/-   ##
=======================================
  Coverage   76.13%   76.13%           
=======================================
  Files         528      528           
  Lines       19579    19579           
  Branches     3829     3829           
=======================================
  Hits        14906    14906           
  Misses       3791     3791           
  Partials      882      882           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant