[ric-oliv]

On every build leg, Test and Integration test run one after the other, though the integration tests only need the packages from Create NuGet Packages. In main run 39858273561, win-x64 spent 5.8 minutes in Test and 14 in Integration test, and win-arm64 4.2 and 14.3.

GitHub Actions can now run steps in the background, and this PR uses that on the build legs:

• The job packs first, then starts Test as a background step and runs the integration tests while it runs. It then waits for it with wait-all and uploads the coverage and any failed snapshots.
• The macOS publishing build keeps the current order, because its sparse checkout removes the test outputs and coverage files. It isn't the slowest leg today.
• The step names stay the same, so the step timings in the CI trace data stay comparable.

The first run found a race. Every solution build runs before.Sentry.sln.targets, which calls git submodule update --init --recursive. On linux-x64, dotnet test and dotnet pack started in the same second, and the second git call failed on a submodule's config lock, so the tests failed before they ran. The second commit starts the tests after packing, which takes under a minute.

Measured on the fixed run, comparing the time after Build with the same steps added up one after another:

Leg	After Build, overlapped	Same steps one after another
win-x64	18.6 min	25.3
win-arm64	22.9	28.8
linux-x64	12.3	16.7
linux-arm64	7.4	9.9

• Every leg passed, including the two Alpine container legs, so background steps work there too.
• The first run's win-x64 shows the same saving, 15.1 against 20.6.
• win-arm64 ran on a slow host both times, with Build at 12.4 and 13.5 minutes against a median of 8.8, so its integration tests were slow too. I can't tell yet whether the overlap adds CPU contention, and I'd like a couple more runs before reading much into it.

When a unit test fails

A throwaway draft, #5799, added two failing tests to Sentry.Serilog.Tests. One always fails, and one Verify test has no approved snapshot. On every leg that runs the tests in the background:

• the background Test step failed and the integration tests kept running to the end,
• Wait for the tests then failed the job,
• Upload code coverage was skipped,
• Upload build and test outputs uploaded the .received.txt snapshots, 3 files per leg.

So a failing unit test still fails the leg and still uploads its snapshots, but the leg turns red later: 4.3 minutes after the test failed on linux-arm64, 7.4 on linux-x64, 8.9 on win-x64 and 13.5 on win-arm64. Before, the job stopped right after Test. The workflow's result doesn't arrive later, because it waits for the slowest leg either way. macOS keeps the old order, so it still failed right after Test.

On linux-arm64 the integration tests also failed, because the Native AOT compiler crashed with ILC: error IL1013, a NullReferenceException, while publishing the net9.0 console app. That happened 3 minutes after the background tests had finished, and the same app had published fine 14 seconds earlier, so it looks like a separate flake.

actionlint 1.7.12, the latest release, doesn't know background and wait-all yet, so it flags those two steps. GitHub validates the workflow when the run starts.

#skip-changelog

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.13%. Comparing base (ff0355e) to head (a51cad7).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #5785   +/-   ##
=======================================
  Coverage   76.13%   76.13%           
=======================================
  Files         528      528           
  Lines       19579    19579           
  Branches     3829     3829           
=======================================
  Hits        14906    14906           
  Misses       3791     3791           
  Partials      882      882           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ric-oliv]

Closing this one. Once the integration tests run in their own jobs, in #5809, the unit tests this moves to the background have almost nothing left to overlap with, so it isn't worth the new step syntax and the later failures. Details in #5806.