This document contains all the required information to build, test, and consume MSTest.
To build and test all functionalities of MSTest, we recommend installing Visual Studio 2026 with the following workloads:
.NET desktop developmentUniversal Windows Platform development.NET Core cross-platform development
We recommend the following overall workflow when developing this repository:
- Fork this repository.
- Always work on your fork.
- Always keep your fork up to date.
Before updating your fork, run this command:
git remote add upstream https://github.com/Microsoft/testfx.gitThis will make management of multiple forks and your work easier over time.
We recommend the following commands to update your fork:
git checkout main
git clean -dfx
git fetch upstream
git rebase upstream/main
git pushOr more succinctly:
git checkout main && git clean -xdf && git fetch upstream && git rebase upstream/main && git pushThis will update your fork with the latest from microsoft/testfx on your machine and push those updates to your remote fork.
An experimental native Bazel build is available for the portable .NET 8-11 development graph, selected test suites, and native MTP/TRX package consumers. It does not replace the MSBuild/Arcade shipping build or its complete framework/workload and acceptance matrix.
An experimental BuildXL build can schedule and
cache the existing MSBuild projects on Windows x64. Its default entry point is
TestFx.slnx, preserving project-specific target frameworks except classic UAP,
which requires desktop MSBuild. Separate Debug and Release pipeline legs build
the solution and retain engine counters and project binlogs for later analysis.
After a successful build, each leg also runs the same 13 net10.0 unit suites that
the Bazel dogfood workload runs, from the BuildXL outputs, and retains restore,
build and test timings plus per-suite test-identity hashes so the Bazel and
BuildXL phases can be compared; see
running the matched unit suites.
Outputs are isolated; the Arcade build remains the shipping authority.
The driver enables tracked shared compilation, sizes concurrency to the available
CPUs, and automatically falls back to full input hashing when change journals
are unavailable without modifying local volume settings. It never rewrites the
checked-in XLF files, and -Pack (opt-in, not used by CI) also packs the 33
NuGet packages in the same graph; see
opt-in pack.
CI initializes the
checkout volume's change journal before restoring the action cache between
compatible runs; existing journals are preserved, and initialization failures
fail the job. The Windows CI agent must run elevated.
The BuildXL performance comparison
remains an explicit local experiment for matched cold, changed-input, and no-op
TRX builds. CI runs real solution builds, not repeated benchmark samples.
The easiest and recommended solution is to build the repository with the provided scripts at the repo root.
For Windows:
build.cmdFor Linux and macOS:
./build.shBy default, the script generates a Debug build type, which is not optimized code and includes asserts. As its name suggests, this makes it easier and friendlier to debug the code. If you want to make performance measurements, you ought to build the Release version instead, which doesn't have any asserts and has all code optimizations enabled. Likewise, if you plan on running tests, the Release configuration is more suitable since it's considerably faster than the Debug one. For this, you add the flag -configuration release (or -c release). For example:
For Windows:
build.cmd -configuration releaseFor Linux and macOS:
./build.sh --configuration releaseAnother common flag is -pack which will produce the NuGet packages of MSTest. These packages are required for the acceptance tests (see testing section).
On Windows, -pack also runs eng/verify-nupkgs.ps1, which checks package metadata and compares the files of each MSTest package against its expected shape in eng/package-shapes/<PackageId>.txt. A missing or unexpected file fails the build and is listed in the error. When a package layout change is intentional, regenerate the expected shapes after packing and commit them with your change:
.\eng\verify-nupkgs.ps1 -configuration Debug -updateBaselinesFor more information about all the different options available, supply the argument -help|-h when invoking the build script. On Unix-like systems, non-abbreviated arguments can be passed in with a single - or double hyphen --.
Repository builds use MSBuild's experimental multithreaded mode by default,
including desktop MSBuild on Windows and public/official CI. Windows builds
require Visual Studio 2026 18.12 or newer. To diagnose a build without
multithreading, use build.cmd -mt:0 on Windows or
./build.sh --mt false on Linux/macOS. Direct MSBuild invocations can use
/mt:false to override the inherited MSBUILDENABLEMULTITHREADED default.
Compatibility tests that deliberately opt out and BuildXL's separate native
scheduler retain their existing behavior.
MSTest uses Microsoft common infrastructure called arcade as such all outputs follow this structure:
artifacts
bin
$(MSBuildProjectName)
$(Configuration)
packages
$(Configuration)
Shipping
$(MSBuildProjectName).$(PackageVersion).nupkg
NonShipping
$(MSBuildProjectName).$(PackageVersion).nupkg
Release
PreRelease
TestResults
$(Configuration)
$(MSBuildProjectName)_$(TargetFramework)_$(TestArchitecture).(xml|html|log|error.log)
SymStore
$(Configuration)
$(MSBuildProjectName)
log
$(Configuration)
Build.binlog
tmp
$(Configuration)
obj
$(MSBuildProjectName)
$(Configuration)
toolset
with
| directory | description |
|---|---|
| bin | Build output of each project. |
| obj | Intermediate directory for each project. |
| packages | NuGet packages produced by all projects in the repo. |
| TestResults | Test results produced by test runs. |
| SymStore | Storage for converted Windows PDBs |
| log | Build binary log and other logs. |
| tmp | Temp files generated during build. |
| toolset | Files generated during toolset restore. |
MSTest uses the following 3 kinds of tests:
- Unit tests
- Very fast tests primarily validating individual units.
- Named as
<ProjectUnderTest>.UnitTestswhere<ProjectUnderTest>is the project under test.
- Integration tests
- Slightly slower tests with File system interactions.
- Named either as
<ProjectUnderTest>.IntegrationTestsor as<PackageUnderTest>.Acceptance.IntegrationTestswhere<ProjectUnderTest>is the project under test<PackageUnderTest>is the package under test
- Performance tests
- Focused tests that ensure the performance of specific workflows of the application
The easiest way to run the tests is to call
For Windows:
build.cmd -pack -test -integrationTestFor Linux and macOS:
./build.sh -pack -test -integrationTestNote that -test allows to run the unit tests and -integrationTest allows to run the two kinds of integration tests. Acceptance integration tests require the NuGet packages to have been produced hence the -pack flag.
The Windows, Linux and macOS CI test jobs save existing MTP activities under
artifacts/TestResults/<Configuration>/OpenTelemetry. Download the corresponding
TestResults_<OS>_<configuration or collection kind>_Attempt<job attempt> build
artifact to retrieve them. Failed jobs publish partial traces too. Availability
follows the build's Azure Pipelines retention policy; no guaranteed number of days
is configured in this repository.
The Windows app-model job includes the same directory inside
Windows_App_Model_Diagnostics_Attempt<job attempt>/artifacts/TestResults/<Configuration>/OpenTelemetry.
Local collection is opt-in. After building the desired test host with the
repository scripts, set TESTFX_OTEL_TRACE_DIRECTORY to an absolute directory
before running it. For example, in PowerShell:
$previous = $env:TESTFX_OTEL_TRACE_DIRECTORY
try {
$env:TESTFX_OTEL_TRACE_DIRECTORY = Join-Path $PWD 'artifacts\TestResults\Debug\OpenTelemetry'
& .\artifacts\bin\Microsoft.Testing.Extensions.UnitTests\Debug\net10.0\Microsoft.Testing.Extensions.UnitTests.exe --filter 'FullyQualifiedName~TestFxOpenTelemetryTests'
} finally {
$env:TESTFX_OTEL_TRACE_DIRECTORY = $previous
}OTEL_SDK_DISABLED=true disables collection. Without the directory variable the
existing instrumentation-only dogfooding remains active without writing files.
The helper does not configure or contact an OTLP endpoint. The existing Native
AOT exclusion remains unchanged.
Each uniquely named .jsonl file is a UTF-8 TestFx archive, not an OTLP request:
- The first
resourcerecord declaresformat: "testfx-otel-jsonl",schemaVersion: 1, a unique archiverunId, and resourceattributes. - Each
spanrecord preserves IDs/parents, trace state/flags, source, name/kind, exact UTC start/end,durationTicks(100 ns ticks), status, attributes, events and links. Attribute lists contain{ "key": ..., "value": ... }entries; JSON strings, booleans, numbers and arrays retain their value types. Readers must preserve 64-bit integers. Non-finite floats and unsupported attribute types are represented as invariant strings. - The last
completionrecord containsstatus,spanCountandfailedExports. Requirestatus: "complete"and zero failures before treating a file as a complete export. A crash/forced termination can leave no completion and a partial last line; earlier complete lines remain useful, but are not a full run.
Completion describes archive delivery, not test-session success. For example,
--help and --info produce complete archives with an informative
TestHostBuilder operation (HostType: InformativeCommandLineHost) but no test
results or test.run.exit_code verdict. Do not count those as healthy execution
samples. Likewise, a complete archive can describe a failed or zero-result run.
CI checks completion with eng/verify-test-traces.ps1 before the existing artifact
upload. Incomplete files fail that check on successful jobs and remain diagnostic
warnings on failed/canceled jobs. Empty selection or explicit disablement is not
proof that traces were collected.
Module version/binary MVID, build/SDK/TFM, machine/OS/runtime/CPU, CI/source, logical
execution and supplied retry/job-attempt provenance are retained on the resource.
Do not mix different attempts or use an absent phase duration as zero. Existing test.step.* attributes
are preserved; canonical span elapsed time and reported result durations are not
interchangeable, and fixture intervals may overlap.
The sink omits captured stdout/stderr attributes, but names, parameter values, paths, metadata and exceptions can still contain sensitive information. Treat these files like the other test artifacts; do not upload them to a new destination without reviewing their contents. Synchronous span flushing adds collection overhead and uses no dropping batch queue. See the research, contract and validation record.
Generate an offline cost/coverage snapshot from downloaded test-artifact ZIPs, directories or individual JSONL archives (Python 3.12+, no additional packages):
python .github\scripts\report_test_traces.py `
--input C:\evidence\TestResults_Windows_Release_Attempt1.zip `
--input C:\evidence\TestResults_Linux_Release_Attempt1.zip `
--output C:\evidence\report\test-tracesThe command writes an HTML report, Markdown and JSON with the same filename prefix. It reads trace members in ZIPs without extracting or running artifact code. Failed/incomplete/unsupported archives are named and excluded; the command returns exit 2 when input was rejected or no executing workers were found. Root CTRF reports are selected consistently from an unwrapped artifact or a single common artifact wrapper, in ZIPs and extracted directories; child and merged reports are not included. A damaged trace member does not discard healthy members. Artifact labels remain literal in both Markdown and HTML output.
The snapshot ranks existing fixture invocations and test-occurrence elapsed times, preserves TestContainer step timings, and shows the observation cohorts. Controllers and non-executing operations do not inflate test counts. Host protocol types are shown separately, but either protocol can be used by a test-launched subprocess; compare execution occurrences with the independent root CTRF counts. Shared fixture work and its overlapping interval union are distinct measures, neither CPU time nor time that can be subtracted to obtain native method-body duration.
This is not yet a weekly regression detector. One build can expose expensive setup, but cannot distinguish a regression from machine load. Binary MVIDs keep platform-specific variants separate when their exported TFM labels are identical; those identifiers are not stable cross-build cohort keys. See the first-report research and evidence.
Code coverage is enabled by default for repository tests. Each test project's generated coverage.config includes only its directly referenced production assemblies, excluding shared test-host infrastructure, NuGet package sources, and generated artifacts. Microsoft.Testing.TestInfrastructure is explicitly excluded at both the assembly and source levels.
Compatibility shims in Polyfills directories are excluded from coverage. Ensure.cs and ProcessExtensions.cs remain included because their argument validation and process-lifetime behavior are repository-owned logic worth testing. Coverage exclusions do not change which tests execute.
The repository uses Stryker.NET for mutation testing. The checked-in MutationTesting.slnx provides a focused local run for the server-mode client sources, while the weekly workflow generates isolated solutions for every enabled module in eng/mutation-testing/modules.json. Unsupported targets are explicitly listed with a reason rather than silently omitted. First run .\build.cmd on Windows or ./build.sh on Linux and macOS to provision the repository-local .dotnet SDK. Then install the pinned tool to a dedicated path and run it from the repository root:
On Windows PowerShell:
$strykerVersion = (Get-Content .config/stryker/dotnet-tools.json | ConvertFrom-Json).tools.'dotnet-stryker'.version
$toolCommand = if (Test-Path artifacts/tools/stryker/dotnet-stryker.exe) { "update" } else { "install" }
$previousMutationTesting = $env:MutationTesting
$previousDotnetRoot = $env:DOTNET_ROOT
$previousTelemetryOptOut = $env:DOTNET_CLI_TELEMETRY_OPTOUT
$previousPath = $env:PATH
try {
$env:DOTNET_ROOT = (Resolve-Path .dotnet).Path
$env:DOTNET_CLI_TELEMETRY_OPTOUT = "1"
$env:PATH = "$env:DOTNET_ROOT;$env:PATH"
if ($toolCommand -eq "update") {
& ./.dotnet/dotnet tool update dotnet-stryker --tool-path artifacts/tools/stryker --version $strykerVersion --configfile .config/stryker/NuGet.config --allow-downgrade
}
else {
& ./.dotnet/dotnet tool install dotnet-stryker --tool-path artifacts/tools/stryker --version $strykerVersion --configfile .config/stryker/NuGet.config
}
if ($LASTEXITCODE -ne 0) {
throw "Failed to install the pinned Stryker.NET tool."
}
$env:MutationTesting = "true"
./artifacts/tools/stryker/dotnet-stryker --config-file stryker-config.json --solution MutationTesting.slnx --output artifacts/mutation-testing
if ($LASTEXITCODE -ne 0) {
throw "Stryker.NET mutation testing failed."
}
}
finally {
$env:MutationTesting = $previousMutationTesting
$env:DOTNET_ROOT = $previousDotnetRoot
$env:DOTNET_CLI_TELEMETRY_OPTOUT = $previousTelemetryOptOut
$env:PATH = $previousPath
}On Linux and macOS:
(
set -e
export DOTNET_ROOT="$PWD/.dotnet"
export DOTNET_CLI_TELEMETRY_OPTOUT=1
export PATH="$DOTNET_ROOT:$PATH"
stryker_version=$(sed -n 's/^[[:space:]]*"version":[[:space:]]*"\([^"]*\)".*/\1/p' .config/stryker/dotnet-tools.json)
test -n "$stryker_version"
if [ -x artifacts/tools/stryker/dotnet-stryker ]; then
"$DOTNET_ROOT/dotnet" tool update dotnet-stryker --tool-path artifacts/tools/stryker --version "$stryker_version" --configfile .config/stryker/NuGet.config --allow-downgrade
else
"$DOTNET_ROOT/dotnet" tool install dotnet-stryker --tool-path artifacts/tools/stryker --version "$stryker_version" --configfile .config/stryker/NuGet.config
fi
MutationTesting=true ./artifacts/tools/stryker/dotnet-stryker --config-file stryker-config.json --solution MutationTesting.slnx --output artifacts/mutation-testing
)The opt-in property runs unit-test projects on net10.0 and selects Arcade's open strong-name key for mutated assemblies and their friend assemblies because Stryker's in-memory compiler cannot complete Microsoft delay signing. Native MSTest hosts also consume the adapter/framework from source in this mode; a shipped Microsoft-key adapter cannot access the open-key friend declarations in the locally built platform and would fail before test discovery.
The HTML and JSON reports are written to artifacts/mutation-testing. The mutation testing workflow runs weekly and can be started manually. It reads the enabled modules from eng/mutation-testing/modules.json, runs each production/test-project mapping on an isolated runner, and uploads its detailed Stryker report. Every Microsoft.Testing.Extensions.* project must have an enabled module or an explicit excludedProjects entry with a reason, so newly added extensions cannot silently skip mutation testing. The Windows-only PackagedApp controller and build-only PackagedApp.MSBuild package are excluded from the Ubuntu/net10.0 Stryker shards, not from their acceptance tests.
Generated shard solutions must be passed to Arcade's --projects option using an absolute path; relative paths resolve under Arcade's toolset directory, not the repository root. Shards retain module failure metadata and build-console logs even when the build fails before Stryker starts. An aggregate job verifies that every enabled module used the same source revision, computes the weighted enabled-module score from the mutant counts, reports onboarding coverage and exclusion reasons against the known mutation targets, and uploads the mutation-testing-full-report artifact with a Markdown summary, an HTML index, aggregate JSON, logs, and all module reports. If any expected module is missing or failed, the aggregate is explicitly marked partial and the workflow fails after publishing the available diagnostics.
The Mutation Test Improver remains scoped to the ServerMode client report. Its trusted pre-agent step resolves the upstream run and downloads the binary artifact before the sandbox's text-only CLI proxy starts. It stages validated report data and run identity under /tmp/gh-aw/mutation-input; missing/expired reports and download/validation failures still reach monthly reporting as unavailable evidence, never a fresh zero score. Failed aggregates are usable only when their ServerMode module metadata proves success at the upstream source commit.
If you are working with Visual Studio, we recommend opening it through the open-vs.cmd script at the repo root. This script will set all the required environment variables required so that Visual Studio picks up the locally downloaded version of the .NET SDK. If you prefer to use your machine-wide configuration, you can open Visual Studio directly.
Inside Visual Studio, all projects can be built normally. All but acceptance tests can be tested directly from Visual Studio. The acceptance tests will always use the version of the NuGet packages produced in the artifacts/packages/shipping folder so if you have made some changes and run these tests, it's likely that the changes will not be applied.
Do not use IsImplicitlyDefined="true" on PackageReference items in the MSTest.Sdk .targets files. The package would be defined twice, which can produce NU1009 warnings that are commonly treated as errors.
Do not use VersionOverride on those PackageReference items. Although it can override a version under Central Package Management (CPM), it is forbidden when CentralPackageVersionOverrideEnabled is false and causes NU1013.
Instead, split version specification based on CPM:
- When
ManagePackageVersionsCentrallyis nottrue, setVersiondirectly on thePackageReference. - When
ManagePackageVersionsCentrallyistrue, leave thePackageReferenceunversioned and add a matchingPackageVersionitem.
This supports both values of CentralPackageVersionOverrideEnabled without producing NU1009.
MSTest.Sdk implicitly imports Microsoft.NET.Sdk. To combine it with another base SDK, such as Microsoft.NET.Sdk.Web, import both SDKs manually and list the other SDK first:
<Project>
<Import Project="Sdk.props" Sdk="Microsoft.NET.Sdk.Web" />
<Import Project="Sdk.props" Sdk="MSTest.Sdk" />
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
</PropertyGroup>
<Import Project="Sdk.targets" Sdk="MSTest.Sdk" />
<Import Project="Sdk.targets" Sdk="Microsoft.NET.Sdk.Web" />
</Project>Because an <Import> cannot specify the SDK version in the same way as <Project Sdk="MSTest.Sdk/x.y.z">, pin it in global.json:
{
"msbuild-sdks": {
"MSTest.Sdk": "x.y.z"
}
}Sdk.props and Sdk.targets guard their Microsoft.NET.Sdk imports with _MSTestSdkImportsMicrosoftNETSdk. MSTest.Sdk imports the base SDK only when another SDK has not already set UsingMicrosoftNETSdk, avoiding MSB4011 duplicate-import warnings.
Windows command-line builds require Visual Studio 2026 18.12 or newer for
multithreaded desktop MSBuild, as specified by tools.vs.version in
global.json. The repository uses the XML-based TestFx.slnx solution format,
which these versions also support. The command-line multithreading default does
not change Visual Studio's in-process IDE build scheduling.