PSMutant

Mutation testing for PowerShell. The metric that proves a test would catch a bug, not merely run the line.

Line/branch coverage only tells you a line executed. PSMutant injects small faults ("mutants") into your scripts — flip -eq to -ne, $true to $false, 5 to 6, drop a -not — and checks whether your Pester suite fails. The percentage it catches is your mutation score: the share of injected bugs your tests would actually stop.

It's built on PowerShell's own AST (no Python, no external parser — StrykerJS is JS-only), and every mutant runs inside a throwaway sandbox, so your source is never modified, even if a run is interrupted.

To our knowledge, PSMutant is the first mutation-testing module on the PowerShell Gallery.

Install

Install-Module PSMutant -Scope CurrentUser

Requires PowerShell 7.0+ and Pester 5.2+. Both floors are executed by a CI gate rather than declared -- one leg per supported minor of each. On PowerShell 7.0 to 7.3 use Pester 5.x: Pester 6 fails there with Unable to find type [PesterConfiguration], which is Pester's constraint rather than this module's.

Windows PowerShell 5.1 is not supported, and cannot be. The module's own source uses the ternary operator, which 5.1 cannot parse, so it fails to load rather than running degraded. And the operator set mutates ternaries by naming the 7.0 syntax-tree type for them, which does not exist before 7.0.

PSMutant does not import a Pester for you, and does not declare one as a required module. It runs under whichever Pester >= 5.2.0 your session already has, and only imports one if none is loaded at all. That is what makes the >= 5 promise real: a manifest dependency is a minimum that PowerShell satisfies with the newest version installed, so declaring one would mean choosing your Pester before you got a say. The trade is that Install-Module PSMutant will not install Pester for you -- Install-Module Pester -MinimumVersion 5.2.0 if you need it, which is also what the error message says.

Use

Create a config pointing at the pure modules you want to test and the Pester files that cover them (see examples/psmutant.config.json):

{
  "mutate": ["src/MyModule.Transform.ps1"],
  "tests":  { "src/MyModule.Transform.ps1": ["tests/Transform.Tests.ps1"] },
  "coveredLinesOnly": true,
  "thresholds": { "high": 85, "low": 70, "break": null },
  "reportPath": "reports/ps-mutation.json"
}

Run it from your repo root:

Import-Module PSMutant
$result = Invoke-PSMutation -ConfigFile ./psmutant.config.json
"$($result.Score)% ($($result.Killed)/$($result.Total))"
exit $result.ExitCode        # 0 unless thresholds.break is unmet, or a declaration went stale
                            # $result.FailureReason says which: None | StaleEquivalents | BelowThreshold

Survivors are printed with file:line and the exact source→mutant change — each is a missing assertion, an equivalent mutant (a change that can't alter behaviour), or dead code.

Gating a monorepo, one package at a time

Get-ChildItem ./packages -Directory |
    Invoke-PSMutation -ConfigFile ./psmutant.config.json -Quiet |
    Where-Object ExitCode -ne 0

-ConfigFile binds from the pipeline by value and by property name, and -SourceRoot by property name with FullName aliased — so a list of configs, or a list of directories, each becomes an independent run with its own sandbox, baseline and report. They are processed as they arrive rather than collected first, so twenty packages give the first verdict in the time the first run takes. One result object per config; fold them if you want a single answer:

$worst = $configs | Invoke-PSMutation -Quiet | Sort-Object ExitCode -Descending | Select-Object -First 1

PSPath is deliberately not aliased. It is provider-qualified (Microsoft.PowerShell.Core\FileSystem::/x), and every path in the config resolves against -SourceRoot.

-SourceRoot must be a directory. Piping files binds -ConfigFile by value and -SourceRoot from the same object's FullName, which would point the root at the config file — so that is refused by name rather than surfacing later as a puzzling sandbox error.

Gating a pull request on what it changed

$changed = git diff --name-only origin/main...HEAD
$result  = Invoke-PSMutation -ConfigFile ./psmutant.config.json -ChangedFile $changed
exit $result.ExitCode

mutate is intersected with what changed, and everything follows: only those files are enumerated, hashed and answered for. It answers the question a reviewer actually has — are the lines this PR introduced tested well enough? A whole-repo score cannot answer that, and a whole-repo score is what makes people turn the gate off.

You compute the diff. There is deliberately no -ChangedSince <ref>: a diff is not a fact this module can work out. It needs a base, and every way that goes wrong goes wrong in your environment — a shallow clone where the ref was never fetched, a detached HEAD, a merge base that is not the one the reviewer sees. Resolving it here would turn those into a mutation tool refusing to run, several layers from the shell where they can be fixed.

An empty list is refused, because git diff against a ref that was never fetched prints nothing and exits 0 — and read as a pass, that is a green gate that measured no mutants at all. A list that holds files, none of which are in mutate, is a different situation: an ordinary documentation change, which passes and says so.

The score is real — every mutant in those files was evaluated — but it is not the project's. So the report goes to <report>.changed.json and never the project's file, mode is "Changed", and changedFiles sits beside the score in both the document and the result object. The schema requires it there: a percentage over part of a tree that does not say which part is the number this module exists to stop people quoting.

It cannot be combined with -RecheckFrom, -UpdateBaseline or -MergeIntoBaseline — folding a scoped run's survivors into a whole-project baseline would record "no survivors" for every file the run never looked at. -ListOnly is allowed, and previewing what a PR would mutate is the cheapest use either has.

Restricting mutants to changed lines is not implemented. It needs hunk offsets, which have the same problem -ChangedSince has and no agreed shape yet.

In CI: survivors on the pull request

Two ways to put a survivor in front of a reviewer instead of in job-log scrollback. Both list survivors only, never killed mutants, and both are worth having: annotations need no setup, and the SARIF log keeps a history.

Annotations, automatically. Under GitHub Actions (GITHUB_ACTIONS=true) each survivor is also printed as a ::warning file=...,line=...:: workflow command, which GitHub shows on the diff. Under Azure Pipelines (TF_BUILD=True) it is a ##vso[task.logissue type=warning;sourcepath=...;linenumber=...] command, which appears in the build summary linked to the line. Neither is printed anywhere else, so a developer's console stays clean. -Quiet does not silence them: it exists to keep a CI log short, and CI is where a survivor most needs to be seen.

A SARIF log, when you ask for one. Set sarifPath and the run also writes a SARIF 2.1.0 log of its survivors, which code-scanning services turn into alerts that open, persist and close across runs:

{ "reportPath": "reports/ps-mutation.json", "sarifPath": "reports/ps-mutation.sarif" }

• One rule per operator the run applied (PSMutant/BinaryOperator, ...), so a team can suppress one operator's findings without the others.
• warning level: a survivor is a gap in the tests rather than a defect in the code.
• A declared equivalent produces no alert. The config already argued it, and the report still lists it.
• Each alert is fingerprinted by file:function:description, the same address an equivalence declaration uses, and never by mutant id or line. Ids renumber and lines move whenever anything above them changes, and an alert keyed on either would close and reopen on unrelated edits. Identical mutations in one function are numbered #1, #2 so neither swallows the other.
• run.properties carries the score, killed, survived and total, so the log can be read on its own.
• A run with no survivors writes a log with no results. That is what closes the alerts it used to have.

Which runs write one. A full run writes sarifPath. A -ChangedFile run writes <name>.changed.sarif beside it, never over it: uploaded as the project's log, a few files' survivors would close every alert in every other file, because a code-scanning service reads a missing result as a fixed one. A -RecheckFrom run, a -ListOnly preview and an interrupted run write none, for the same reason.

The log carries no category (automationDetails), deliberately. On GitHub a category in the file overrides the one the upload step names, so a fixed one would make two PSMutant uploads in one repository replace each other. Name it in the pipeline:

# GitHub Actions
- uses: github/codeql-action/upload-sarif@v3
  if: always()
  with:
    sarif_file: reports/ps-mutation.sarif
    category: mutation

# Azure Pipelines, with GitHub Advanced Security for Azure DevOps
- task: AdvancedSecurity-Publish@1
  condition: succeededOrFailed()
  inputs:
    SarifsInputDirectory: '$(Build.SourcesDirectory)/reports'
    Category: 'mutation'

Without Advanced Security, install the SARIF SAST Scans Tab extension and publish the directory holding the log as a pipeline artifact named exactly CodeAnalysisLogs. Each run then gets a Scans tab. That view is per run: there is no alert history, and nothing closes when a survivor is killed.

Either way the upload step needs if: always() / condition: succeededOrFailed(). A run that fails its break threshold is exactly the run with findings, and a step that runs only on success uploads them on every build except the ones that have any.

examples/azure-pipelines.yml is a complete Azure pipeline. It covers a full run on main, a -ChangedFile run on pull requests (which needs fetchDepth: 0, because Azure Pipelines checks out shallow and detached by default), and both ways of publishing the log. It uploads to Advanced Security from full runs only: a pull request's log covers the files it changed, and its findings already reach the PR as build-summary warnings.

Browsing every mutant: the mutation testing report

Set mutationTestingReportPath and a run also writes every mutant it ran in the mutation-testing-elements report format, the open format Stryker's tools read. Each file carries its source, and each mutant its operator, its exact start and end, its replacement and its verdict, so a viewer can draw it in place:

{ "reportPath": "reports/ps-mutation.json", "mutationTestingReportPath": "reports/mutation-testing.json" }

PSMutant writes the data and ships no viewer. Open the file with Stryker's <mutation-test-report-app> HTML component, or upload it to the Stryker dashboard.

• Statuses map one to one: Killed, Survived, TimedOut as Timeout, CompileError and RuntimeError. A declared equivalent is Ignored, with its argument as the reason; both tools leave it out of the score.
• Mutants on lines no test ran (coveredLinesOnly) are not listed. The format would count them as NoCoverage, undetected, and show a lower score than the one the run gated on; the JSON report counts them.
• Which runs write one is the same as for SARIF: a full run writes the path, a -ChangedFile run a .changed sibling, and a recheck, preview or interrupted run none.

Seeing what a config would mutate, before it costs you minutes

Invoke-PSMutation -ConfigFile ./psmutant.config.json -ListOnly

Prints the mutant set — per file, per operator, and how many candidates survive the coveredLinesOnly filter — and stops. Nothing is evaluated, no report is written, no score is produced. It is the answer to "what would this actually do?", which is the question you have exactly when you are least sure: adding a file to mutate, changing operators, or wondering why a file scores 100%.

That last one is the reason it exists. A file that produces no candidates scores a vacuous 100% — it is listed in mutate, hashed into the report, contributes 0 of 0, and in a blended score is invisible. Two files in a real repository were in that state and it took a dedicated investigation to notice. -ListOnly names them, along with the files whose candidates the coverage filter removed entirely. Different faults with different fixes, so they are listed apart: one is a test to write, the other is a file that does not belong in mutate.

     34 src/Transform.ps1 -> 31 covered
     19     ConditionForcing -> 18
     11     BinaryOperator -> 10
      4     ReturnValue -> 3
      0 src/Constants.ps1

  31 mutant(s) over 2 file(s) would be evaluated.
  1 file(s) produced NO candidate, so each scores a vacuous 100%: src/Constants.ps1

The result object carries both sets by name — FilesWithNoCandidate and FilesEmptiedByCoverage — so a repository that considers either one a mistake can fail its own build on it. The module does not: a file with nothing to mutate is not always an error, and a preview that evaluated nothing has no business issuing a verdict. ExitCode is always 0.

Cost: one baseline suite run, because coveredLinesOnly is part of what would actually be mutated and a preview that skipped it would answer a different question than the run does. Never the mutants x suite that a run pays.

Rechecking survivors while you write assertions

Killing survivors is an edit-run-edit loop, and re-running the mutants you already killed is most of the wait. -RecheckFrom runs only the mutants a previous report recorded as survivors, minus any you have declared equivalent — the config already states in writing that no test can kill those, so re-evaluating them is guaranteed-wasted work.

A recheck report can seed the next recheck, so the loop gets shorter every round rather than starting over from the full set:

Invoke-PSMutation -ConfigFile ./c.json                                        # full: 127 mutants
# ...add assertions...
Invoke-PSMutation -ConfigFile ./c.json -RecheckFrom ./reports/ps-mutation.json          # 56
# ...add more, killing 20...
Invoke-PSMutation -ConfigFile ./c.json -RecheckFrom ./reports/ps-mutation.recheck.json  # 36

Each round overwrites that one *.recheck.json scratch report. The full report is never touched, whichever report you chain from.

Two things it deliberately will not do:

• It produces no score. Killing 6 of 10 survivors is not 60% of anything — the denominator is a filtered set. The result object and the *.recheck.json report carry counts (NowKilled, StillSurviving), never a percentage, and thresholds are not applied. The recheck report is written beside the full one, so a partial run can never overwrite the baseline that CI reads.
• It will not guess. Mutants are matched by (file, id), and the id is an AST-walk position — it only means anything for byte-identical source and the same operator set. Reports therefore record a SHA256 per mutated file plus the operator list, and a recheck whose source or operators moved refuses to run rather than confidently rechecking the wrong mutants.

Finish with a full run before trusting a number or raising thresholds.break. A recheck is sound only for additive test changes: editing or deleting an existing test can revive a mutant that was killed before, and a recheck never evaluates those.

Resuming an interrupted run

Ctrl-C, a cancelled CI job or a killed agent leaves a partial report. -ResumeFrom continues that run instead of starting over:

Invoke-PSMutation -ConfigFile ./c.json -ResumeFrom ./reports/ps-mutation.json

The mutants it already recorded are carried over and only the ones it never reached are evaluated. The result is a complete run and carries a real score — which is what separates this from -RecheckFrom, whose set is filtered and whose number would mean nothing. What it cannot claim is that one run stood behind all of it, so the report says resumed and carriedOverUnverified.

It refuses rather than resuming when the carried-over verdicts might be stale, on exactly the terms -MergeIntoBaseline uses:

• the report must be a partial one — a completed report has nothing left to evaluate, and a recheck report describes a different kind of run;
• it must be numbered against the same source and operator set, because mutant ids are AST-walk positions and an edit renumbers them;
• no mapped test file may have shrunk or disappeared. Adding a test cannot revive a mutant the earlier run killed; editing or deleting one can, and a resume never looks at it again.

Up to one worker-count's worth of mutants may be re-evaluated, because a mutant can finish without being recorded. That is harmless — they simply run again.

Reusing verdicts for files nothing touched

-Incremental reuses the previous report's verdict for every mutate file whose verdicts cannot have changed, and evaluates only the rest:

Invoke-PSMutation -ConfigFile ./c.json -Incremental

A mutant's verdict depends on its file, the tests that run against it, and the other code those tests run. So each mutate file gets a key: a hash over the file, its covering test files, and every other source file its covering suite executed. Which files those are is measured, once per covering suite, with Pester's coverage over every script in the sandbox, and recorded in the report beside the key. A file whose key is unchanged since the previous report reuses its rows; every other file is evaluated, and its key recorded for the next run. The first -Incremental run has nothing to reuse: it evaluates everything and records the keys.

The executed files are the point. Replayed over this repo's last 60 merges, a key on the mutate file and its tests alone would have reused the verdicts for about 78% of the mutated code (weighted by line count), and missed every change to a helper the tests also run. Keyed on all of src/ instead, only 35-40% would have been reused. The files the suite actually executed are the ones that can change a verdict.

Measured on PSComplexity's own config, 1424 mutants, 3 workers, every run's verdicts identical to a plain run's:

run	wall	reused
plain	401s	--
first -Incremental: nothing to reuse, dependencies measured	479s	0
nothing changed	24s	1424
one source file touched	161s	1065

Touching that one file re-ran it and the four files whose covering suites execute it, and reused the other ten. The first run costs about a fifth more, for measuring each suite's dependencies; a run that re-evaluates a file measures its suite again.

The previous report is the one at reportPath, and it must be a full run's report that recorded incremental data, written by the same PSMutant, Pester and PowerShell minor version, under the same timeoutFactor, timeoutFloorSeconds and recordAllKillers. Anything else is not reused, and the run says why. A file whose candidates no longer match its recorded rows -- you changed operators, say -- is evaluated in full. The result is a complete run with a real score; before it starts it says how many verdicts it reused, under -Quiet too, and the report marks each reused row with "Reused": true and counts them in reused.

What the key cannot see:

• a file a suite reads as data rather than running -- a JSON fixture, a schema;
• an installed module, which is not in the sandbox;
• a file the suite loads but runs nothing from: dot-sourcing a file of functions none of which is called counts no command, so a later edit that adds top-level code to it, or redefines a function the suite does call, is not seen.

After changing one of those, run without -Incremental. It cannot be combined with -RecheckFrom, -ResumeFrom, -MergeIntoBaseline, -ChangedFile or -ListOnly.

Declaring equivalent mutants

Some mutants provably cannot change behaviour — ConvertTo-Json -Depth 6 versus -Depth 7 on data four levels deep, for instance. No test can kill them, and chasing them produces fixtures that assert nothing. Declare them, with a reason:

"equivalents": {
  "src/Report.ps1:55:6 -> 7": "Depth N and N+1 emit identical JSON for any graph shallower than N; this report nests 4 deep at worst."
}

The key is file:line:description, exactly as printed in the survivor list. A declared mutant leaves the denominator, so thresholds.break: 100 becomes a meaningful gate: every catchable fault is caught.

The declaration is a claim that gets checked, not a mute button:

• A blank reason is ignored — the mutant stays in the score.
• If a declared mutant is ever killed, the run fails: the config asserted no test could catch it, and one did.
• If a declared mutant no longer exists, the run fails: the code moved and nobody revisited the claim.
• Both fail regardless of thresholds.break, including report-only mode — a stale declaration is not a low score, it is a false statement inflating whatever score is printed.

The number of exclusions is printed next to the score and stored as declaredEquivalent, so a 100% resting on a dozen declarations can never be mistaken for one that killed everything.

A declaration can also sit in the code, on the line directly above the mutated one:

    # PSMutant-Equivalent: 0 -> 1: -Path is the only positional parameter
    [Parameter(Mandatory, Position = 0)]
    [string[]]$Path,

The comment is <original> -> <mutated>: <reason>, the description exactly as the survivor list prints it, because a line often holds several mutants and the declaration argues about one. It becomes the config key file:line:description for the line below it, so it faces every check above unchanged -- and it moves with the code, where a line-keyed config entry goes stale on any edit above it.

• Directly above means through any comment lines that follow it, with no blank line in between.
• A declaration that cannot become a key is refused before anything is mutated: a blank line below it, code before it on its own line, nothing below it, no -> in it, or no reason.
• One mutant declared both in the code and in the config is refused, naming both keys: two arguments for one mutant drift apart, and one of them silently does nothing. The same goes for a config that declares one mutant under both its function form and its line form.
• Only the mutate files are read for declarations.

How it works

1. Baseline — runs your tests once (must be green) with Pester code coverage over the mutate files, recording which lines actually executed. A statement written over several lines counts on all of them: a line is covered when the innermost command spanning it ran, so the second line of a long condition is judged with the condition, and the body of a script block that never ran stays uncovered even though the pipeline holding it did. A line no command spans at all — a param() default — is never covered, because nothing Pester instruments can say whether it executed.

2. Enumerate — parses each file's AST and collects candidates; only those on covered lines are kept (an uncovered mutant is guaranteed to survive and teaches nothing).

3. Evaluate — copies the source subtrees into a temp sandbox, splices each mutant into the copy, runs the covering tests in a worker, and restores the copy. Tracked source is never touched. The sandbox is created under a name carrying 128 bits of randomness, and a path that already exists is refused rather than cleared: temp is world-writable on a shared machine, and a predictable name is one another local user can occupy first.

   Before the first mutant, each covering suite runs once, unmutated, where the mutants will run: a fresh runspace per worker that has only Pester, not your session's modules, global functions or preferences. A suite green in your session but not there is refused, named with its failing tests, because every mutant would otherwise be scored Killed by that failure. So is one that runs no test there, every test skipped or never discovered: it can say nothing about any mutant it covers.

4. Score — killed / total, written to the JSON report. A mutant that hits the per-mutant timeout counts toward killed, because a mutant that hangs your suite is a fault -- but it is also reported on its own as timedOut, in the JSON and on the summary line. "The suite proved this fault is caught" and "the suite hung and we assumed so" are different claims, and a rising timedOut means your timeout is too tight or a covering suite too slow, not that your tests got better.

   A mutant whose covering tests "passed" without running a single test is a RuntimeError, not a survivor: it ended their discovery -- an exit reached while a test file was loading, say in a function a BeforeDiscovery calls for its cases -- so nothing was asked about it. It is left out of the score and counted as runtimeErrors, and the summary line says how many.

A loop-condition guard drops any candidate inside a while/for/do condition, so a flipped comparison can never spin an infinite loop — which is what makes in-process execution safe and fast.

Mutated code that reaches exit cannot end the run. Reached inside a test, or a BeforeAll, it fails that test and the mutant is killed; reached while tests are being discovered, it is a RuntimeError as above; at the top of a file a test dot-sources, it ends only that file. [Environment]::Exit() is different: it ends the whole PowerShell process, run and all, and nothing running inside the process can stop it. Leave code that calls it out of mutate.

Tests that killed a neighbour run first

Most of a run's time goes to killed mutants, and a killed mutant runs its covering files until the first test fails -- the whole of a slow file, when that test is near the end. But the mutants in one function are mostly killed by the same few tests. So once a mutant is killed, its killing tests are remembered, and each later mutant in the same function, then the same file, first runs up to three of them alone. If one fails, the mutant is killed and nothing else runs. If none does, the whole files run as they always did, so a survivor pays one short extra run. The first pass has its own deadline, by the same rule as each file's -- max(floor, seconds × factor) -- with seconds the slowest of its tests run alone, and never longer than the whole files' deadline. When it runs out, the whole files run; a mutant that hangs the first pass is still judged by them.

Measured on this repo's own runner file, interleaved, two rounds each way with three workers: 307 mutants in about 22% less wall clock (931 and 924 seconds off, 703 and 729 on), every one killed both ways. CPU time fell only about 4%: the saving is mostly time a worker spent waiting on a slow file, not work it did.

A test is only tried first once it has passed alone, against unmutated source. A test that depends on what an earlier test left behind fails alone whatever the code is, and trusted blindly it would kill every mutant it was tried on. Each learned test is run once on its own, on an idle worker between mutants, before any mutant relies on it; one that fails there is never tried first.

So a verdict is the same either way: a test failing against a mutant, having passed alone without it, is a failure the whole files would show too. What can differ is which test the report names in KilledBy, and with several workers it can differ between two runs, since what has been learned depends on which mutants finished first. Set killersFirst to false for the old timing.

Running mutants in parallel

Wall-clock is mutants × suite, and that is the reason mutation testing gets set off and walked away from rather than run in an edit loop. workers evaluates several at once:

{ "workers": 0 }   // this machine: ProcessorCount - 1
{ "workers": 4 }   // exactly four

Capped at 64, which is the framework's number rather than a taste: the scheduler blocks on WaitHandle.WaitAny, and that throws above 64 handles. Clamped rather than refused, because the worker count does not change the answer.

With more than one worker, each worker is a pwsh process of its own (since 0.8.0). Workers used to be runspaces in one process, sharing one GC heap, and they stopped scaling at about four. Measured on one 24-core machine over seven of PSMutant's own covering suites, the same 112 runs spread over warm workers, interleaved:

workers	runspaces, Windows	processes, Windows	runspaces, WSL 2	processes, WSL 2
1	34.9s	35.7s	46.9s	47.5s
4	17.1s	14.3s	17.8s	14.8s
8	15.3s	9.8s	13.6s	9.0s
16	16.6s	7.1s	12.8s	6.4s

One worker is as fast either way, so a single worker still runs in this process and starts without paying for one. A process costs about 1.3 seconds to start with Pester, once per worker and again after every fifty mutants, when a worker is renewed, and more total CPU at high counts -- each has its own JIT and GC.

More workers is still not monotonically better, and the best number depends on your repo — measure yours. These were measured with runspace workers, before 0.8.0, on two consumer repos on the same 24-core machine, each interleaved in both orders:

workers	8 mutate files, 3 covering suites	112 mutate files, 112 covering suites
1	36.2s	102.0s
2	18.9s	57.9s
4	13.9s	40.1s
8	14.6s	33.6s
16	18.4s	39.8s

Both get worse past their own optimum, and the optima differ: four for the first, eight for the second. More distinct covering suites means more independent work and less contention on the same few files, so the wider repo scales further. Process workers move both optima up, by the table above, but not to infinity: a disk and a set of cores are still shared. 0 is a starting point rather than an answer. Try a few numbers on your own repo and keep the one that wins; PSMutant's own gate pins 3.

Each worker gets its own sandbox copy and its own Pester-loaded runspace, in its own process, so nothing is shared but the read-only candidate list. Finished mutants are recorded in candidate order rather than completion order, so the answer does not depend on which worker finished first. Measured against PSComplexity, whole config:

	mutants	wall	score
serial	554	266s	100
workers: 8	554	100s	100

The two reports are identical row for row — same order, same verdicts, same KilledBy — with only the duration fields differing. That is asserted by a test that runs one fixture both ways and compares, because "deterministic regardless of scheduling" is a claim worth checking rather than reasoning about.

It is opt-in, and the default is 1. Your covering suite runs N times concurrently, and file isolation is complete. Two hazards that bit PSMutant's own suite are gone since 0.8.0, because each worker is a process: an environment variable a test writes is written in that worker's own environment, and a temp file named after $PID is named per worker, since each has its own process id. What no process isolates is anything outside it -- a fixed path outside the sandbox, a network port, a database, a registry key -- and a suite that writes one races itself.

The symptom is a verdict, not an error: a flipped value fails an assertion that should have passed, which is scored as a killed mutant. So turning this on by default could move your score for a reason that is not about your tests' quality. Try it, and if the score moves, the two runs disagree about your suite rather than about your code.

A mutant that ends its worker's process is scored Killed, with no test named, and the worker is replaced: a mutant that calls [Environment]::Exit, say, or recurses until the stack overflows. The tests did not pass, and the run goes on. With one worker, which runs in PSMutant's own process, such a mutant ends the run, as it always has. An unmutated covering suite that ends its worker is refused before the first mutant.

The run-wide per-mutant timeout is multiplied by workers. The baseline is measured once, alone, with the machine to itself; N mutants sharing it are slower for reasons that have nothing to do with the fault injected in them — and an overrun is scored Killed, so a budget sized for a solo run turns ordinary contention into kills and the score goes up. A repo adopting parallelism to make the gate affordable would watch its score improve with no way to tell that from having written better tests. The cost of the other direction is that a genuinely non-terminating mutant takes N times longer to cut off, bounded by the run deadline, which is patience rather than a wrong answer.

Each mutant's deadline comes from its own covering test file's time, not the whole suite's. A mutant runs only the tests mapped to its file, so a deadline sized from the whole baseline -- which also pays for coverage instrumentation -- let a mutant looping forever run for minutes on a repo with many test files. Before the first mutant, each covering suite runs once, unmutated, in every worker (the same pass that refuses a suite green only in your session), and that time sets the deadline for the files it covers: max(floor, seconds × factor). It is not multiplied by workers again: every worker ran the suite at the same time, and the slowest copy's time is the one taken, so the contention is already in it. Multiplied again, a hung mutant of a run with 32 workers waited 128 times its suite. The report's durations.perMutantTimeoutSeconds is the largest deadline any mutant got.

Operators

Name	Mutation
BinaryOperator	-eq↔-ne, -gt↔-le, -lt↔-ge, -and↔-or, +↔-, *↔/
BooleanLiteral	$true↔$false
NumberLiteral	N → N+1
NegationRemoval	-not X → X, !X → X
StringLiteral	quoted string → '' (opt-in — high-volume/low-signal)
ConditionalBoundary	-gt↔-ge, -lt↔-le (opt-in — off-by-one at a boundary, which the swaps above cannot produce)
ConditionForcing	an if/elseif condition → $true and → $false (opt-in)
ReturnValue	return <expr> → return $null (opt-in)
ComparisonOperator	-like↔-notlike, -match↔-notmatch, -contains↔-notcontains, -in↔-notin, -is↔-isnot, and the -c/-i spellings of -eq…-le negated as BinaryOperator negates the bare ones (opt-in)
CaseSensitivity	-ceq → -eq, -ieq → -ceq, and likewise for every operator with a case-sensitive spelling, -creplace and -csplit included. Bare → -c is not tried: on a number it changes nothing (opt-in)
BitwiseOperator	-band↔-bor, -bxor → -bor, -shl↔-shr (opt-in)
StringOperatorRemoval	$a -replace 'x','y' → $a, and the same for -split and -join: the transformation skipped. Not a -join that is the whole of a $(...) in a double-quoted string, where removing it changes only the separator (opt-in)
NumberBoundary	an integer N → N-1 and → 0, and a fractional number such as 1.5 → N+1, N-1 and 0, which NumberLiteral never reaches. A value equal to the original or already listed is dropped, so 1 gives only 0 and 0 only -1 (opt-in)
ArithmeticOperator	% → *, +=↔-=, *=↔/=, %= → *=, and ++↔-- on either side of the operand. Not inside a for loop's iterator, where $i++ → $i-- never ends (opt-in)

The last three reach decisions that live in structure rather than in an expression. A phase guard like if ($SyncUsers) { ... }, or a fallback chain of if ($Ref.Value) { return ... }, holds no comparison, no literal and no negation — so every default operator emits nothing and the file scores a vacuous 100% over code nothing has tested. ConditionForcing asks the only question that matters about such a guard: does any test notice which way the decision went?

They are opt-in because switching one on roughly doubles the mutant count and lowers the score, so a repo gating on thresholds.break would go red purely from upgrading. Turn them on deliberately, and expect to move the threshold down before moving it back up. Adding an operator also renumbers mutants, so any existing report can no longer seed a -RecheckFrom run — the operator set is recorded in the report and the mismatch is refused rather than guessed at.

Config reference

The format is defined by schemas/v1/config.schema.json, which ships with the module. Naming it in the config makes the file self-describing -- a mistake surfaces while you are writing the config, rather than several minutes into a run:

{
  "$schema": "https://raw.githubusercontent.com/Fortigi/PSMutant/main/schemas/v1/config.schema.json",
  "mutate": ["src/MyModule.ps1"],
  "tests": { "src/MyModule.ps1": ["tests/MyModule.Tests.ps1"] }
}

The module validates against this same file when it runs, so nothing depends on the config having been checked first — the schema is the format, not a description of it. It refuses rather than warns: a warning in a CI log is indistinguishable from silence, and every mistake in this class makes the gate weaker while the build stays green.

That matters because both of PowerShell's coercions fail open. "timeoutFactor": "four" would otherwise leave the per-mutant timeout empty, and a timeout expiry counts as a kill — a run reporting a number it never measured. "coveredLinesOnly": "no" would mean yes.

An unknown key is still answered with the nearest valid name (Did you mean 'break'?), and a missing mutate or tests with what the key is for, because those are the two answers a schema cannot give.

Key	Meaning
mutate	Files to mutate. Pure / I/O-free logic pays off most. Required. An entry holding * or ? is a pattern: * and ? stay within one path segment, ** spans any number of them, and a pattern also matches everything below what it names (src/**/*.ps1). Each pattern is expanded once at start-up and the files it matched are printed; a pattern matching no file is refused.
tests	Map each mutate file → the Pester file(s) covering it (per-file test scoping). Required. Every key must be a file listed in mutate (or matched by a pattern there); one that is not is refused, because it covers nothing while its tests still join the baseline. A test entry may be a pattern too, read as in mutate. Comments (_-prefixed keys) go at the top level, not inside this map.
testsByConvention	Pair a mutate file that has no tests entry with tests/<Name>.Tests.ps1, when that file exists (default false). An explicit entry always wins, every pair made this way is printed at start-up, and tests may then be {}. A file with neither still runs the whole suite, as before.
operators	Operator classes to inject. Default is the four expression operators; StringLiteral, ConditionalBoundary, ConditionForcing, ReturnValue, ComparisonOperator, CaseSensitivity, BitwiseOperator, StringOperatorRemoval, NumberBoundary and ArithmeticOperator are opt-in.
coveredLinesOnly	Restrict mutants to lines the baseline executed (default true).
killersFirst	Before running a mutant's whole covering files, run only the tests that already killed a mutant in the same function or file (default true). See Tests that killed a neighbour run first. false runs the whole files for every mutant; recordAllKillers turns it off too.
ignoreCalls	Commands whose arguments are never mutated. Default: the logging commands Write-Verbose, Write-Debug, Write-Information, Write-Host and Write-Progress, whose text rarely says anything about your tests. [] mutates inside every call. Every mutant inside such a call is skipped, from any operator, and the number is printed beside the score and written to the report as skippedAsIgnoredCall, with the list as ignoreCalls. Matched by name, ignoring case and a module qualifier. Write-Warning, Write-Error and throw are not in the default, because tests often assert those messages.
sandboxSubtrees	Directories copied into the sandbox (default ["src","tests"]; set to your layout).
workers	How many mutants to evaluate at once, each in its own sandbox copy and its own Pester runspace -- in a pwsh process of its own when there is more than one. Default 1; 0 means this machine (ProcessorCount - 1). See Running mutants in parallel — it is opt-in for a reason.
timeoutFactor / timeoutFloorSeconds	Per-mutant timeout = max(floor, seconds × factor), where seconds is how long the mutant's own covering tests took unmutated, in every worker at once (defaults 4 / 15). A file whose tests were not measured gets max(floor, baseline × factor × workers), from the baseline run alone. A non-terminating mutant is cut off and counted Killed, so the run never hangs.
equivalents	file:function:description → reason, for mutants that provably cannot change behaviour. Excluded from the denominator. The run fails if a declaration is killed, matches nothing, or matches more than one mutant — a declaration argues about one mutant, so matching several is ambiguous rather than a broader claim. description is <original> -> <mutated>, taken from the source. file:line:description is still accepted, and is the only form available for code outside any function; prefer the function form, because a line number moves whenever anything above it is edited.
thresholds.high / thresholds.low	Colour bands for the console score: green at or above high, yellow at or above low, red below (defaults 85 / 70). They affect the printed colour only, never the exit code.
thresholds.break	null = report-only. A number fails the run (ExitCode 1) below it.
reportPath	Where the JSON report is written (relative to -SourceRoot). A -RecheckFrom run writes <name>.recheck.json beside it and never touches this file.
mutationTestingReportPath	Where every mutant is written in the mutation-testing-elements format, for Stryker's HTML viewer or dashboard (relative to -SourceRoot unless rooted). Absent means none. See Browsing every mutant.
sarifPath	Where a SARIF 2.1.0 log of the survivors is written, for GitHub code scanning or Azure DevOps Advanced Security (relative to -SourceRoot unless rooted). Absent means none. A -ChangedFile run writes <name>.changed.sarif; recheck, preview and interrupted runs write none. See In CI.

Unrecognised keys are an error, not a warning. A misspelling used to resolve to null and quietly weaken the run: thresholds.brake left the break gate unable to fail, and a misspelled operator name was dropped while still being recorded in the report as though it had been applied. The run now stops and names the key, with the nearest valid one where there is one. Keys starting with _ are ignored, so you can use them for comments — JSON has none of its own.

Every report also carries how it was produced:

"schemaVersion": 3,
"producedBy": { "module": "PSMutant", "version": "0.7.0" },
"generatedAt": "2026-08-20T14:05:09Z",
"durations": { "baselineSeconds": 12.4, "totalSeconds": 354.1, "perMutantTimeoutSeconds": 50 }

schemaVersion is there so a consumer can branch on a number instead of guessing from which keys are present. It changes when a field changes meaning, disappears, or becomes required — the three ways a document that was valid stops being valid — never when an optional one is added. It is 3 as of 0.7.0: a mutant's Status can now be TimedOut, CompileError or RuntimeError as well as Killed and Survived, which a version 2 reader would reject, and timedOut, compileErrors and runtimeErrors joined the required full-run disclosures. A TimedOut mutant counts as killed, as before. CompileError (the mutated source did not parse, so no test ran -- a PSMutant defect, please report one) and RuntimeError (the run ended without a verdict) are left out of the score. schemas/v2/ and schemas/v1/ still ship, because an archived report says schemaVersion: 2 or 1 and only its own schema can validate it. durations makes a run comparable with the next one — the timeout sits beside the baseline it was derived from, so a suite drifting toward its bound is visible before it crosses.

Beside mutationScore sits totalMutationScore: the same kills over every mutant the operators produced, with the uncovered ones counted as not killed, as Stryker's total is. mutationScore says how well the tests check the code they run; the total says how much of the code they check at all. The summary prints it when the two differ. The thresholds gate on mutationScore.

A full report also discloses what its score does not answer for: skippedAsUncovered, filesWithNoMutants (files whose candidates the coverage filter removed entirely) and filesWithNoCandidate (files no operator matched at all). The last two are the two shapes of a vacuous 100%, kept apart because the fixes differ, and they are what -ListOnly shows you before a run rather than after one.

Reports also record operators and a sourceHashes map (SHA256 per mutated file). Those exist so -RecheckFrom can prove the mutant ids in a report still refer to the same code.

The report format is defined by schemas/v3/report.schema.json, which ships with the module, so a dashboard or a ratchet can validate a report without reading this repo's tests:

Test-Json -Json (Get-Content ./reports/ps-mutation.json -Raw) `
          -Schema (Get-Content ./schemas/v3/report.schema.json -Raw)

It covers both shapes — a full run, and the partial run -RecheckFrom writes, identified by "mode": "Recheck". A recheck may not carry mutationScore at all, so a partial number cannot be mistaken for a real one even by a reader who ignores the note.

Validate the file, not a parsed object: PowerShell's ConvertFrom-Json recognises the ISO-8601 generatedAt and hands back a [datetime], so the string the schema describes is already gone. Extra properties are permitted deliberately — schemaVersion changes when a field changes meaning, disappears or becomes required, never when an optional one is added, so a validating reader keeps working across releases that record more.

What to point it at

Pure, deterministic logic where a subtle fault is a real bug: transforms, validators, classifiers, SQL-fragment builders. Skip entry points that do live I/O on load — a mutant there is unreachable without real infrastructure.

Treat the score like a ratchet: directional. Don't chase 100% — equivalent mutants and untested log strings make the last stretch noise. Raise thresholds.break to lock in gains.

How your tests should load the code under test

Every worker is one PowerShell process serving many mutants, in a runspace that is reused for many of them: this process when there is one worker, a pwsh process of its own for each when there are more. So each mutant's tests must read the source from disk, as it is now:

In a covering test	Works?
. $PSScriptRoot/../src/Thing.ps1 in a BeforeAll (dot-source)	yes
Import-Module $PSScriptRoot/../src/Thing.psm1, with or without -Force	yes -- PSMutant unloads every module loaded from its sandbox before each mutant
code that defines a PowerShell class or enum, loaded either way above	yes -- see below
using module ../src/Thing.psm1	no -- refused. PowerShell keeps the first version it loads for the life of the process, even across runspaces
a mutant inside an Add-Type call	no -- refused. A compiled type cannot be compiled again in the process; only the opt-in StringLiteral operator reaches one

A refused run names the file and line, and says what to change.

Classes and enums need one extra step, which PSMutant takes for you. PowerShell caches a compiled script by its text, and a type literal such as [MyClass] in a cached script stays bound to the first class it saw -- so a test file that never changes would test every mutant against the original class. When a mutate file defines a class or an enum, each covering test file in the sandbox gets one line appended for each mutant, # psmutant mutant <id> ..., and is restored afterwards. Line numbers are unchanged. Two things notice the line: a test that reads or hashes its own file, and a signed test file under an AllSigned execution policy.

Development

The test estate is written against Pester 6.2.0 and CI pins that exact version. That is a contributor requirement only -- the module itself still supports Pester 5.2+, and tools/Test-PSMutantPesterCompatibility.ps1 proves it on every CI run.

Every gate is a committed script, and running it is passing it: each one throws on a fault, so what you run by hand is what CI runs. In the order CI runs them:

Import-Module Pester -RequiredVersion 6.2.0 -Force                 # PESTER_VERSION in pins.env
./tools/Test-PSMutantCiParity.ps1                                  # workflow rules shared with PSComplexity
./tools/Test-PSMutantRelease.ps1                                   # manifest, CHANGELOG and release notes agree
./tools/Invoke-PSMutantAnalyzer.ps1                                # lint: PSSA_PATHS, every severity
Invoke-Pester ./tests                                              # unit tests
./tools/Test-PSMutantOrderIndependence.ps1                         # the suite reversed, plus an environment check
./tools/Test-PSMutantBlockIsolation.ps1                            # every Describe and Context run alone
./tools/Measure-PSMutantCoverage.ps1                               # coverage gate (100%)
Test-PSComplexity ./src -Recurse                                   # complexity gate (needs PSComplexity)
Invoke-PSMutation -ConfigFile ./psmutant.self.config.json -SourceRoot .   # dogfood: PSMutant on itself
./tools/Test-PSMutantPowerShellCompatibility.ps1                   # every supported PowerShell (downloads them)
./tools/Test-PSMutantPesterCompatibility.ps1                       # every supported Pester (needs them installed)

Two more run outside the merge gate: ./tools/Test-PSMutantPackage.ps1 -Path <staged package> when publishing, and ./tools/Test-PSMutantPinFreshness.ps1 weekly, which needs network access.

While developing, ./tools/New-PSMutantScopedConfig.ps1 -Since HEAD -Run mutates only the files you changed -- seconds instead of minutes, and never a substitute for the full run before a PR.

Quality gates (all required on main)

Every one of these runs in the CI test job and blocks the merge on failure:

• Unit tests — the suites under tests/, in their usual order and again reversed.
• Coverage — 100% of src/, measured by tools/Measure-PSMutantCoverage.ps1.
• PSScriptAnalyzer — zero findings at every severity over PSSA_PATHS; rules are excluded by name, with a reason, in PSScriptAnalyzerSettings.psd1.
• Complexity — every unit must stay at or under 15 cyclomatic and 15 cognitive, measured by PSComplexity (Test-PSComplexity) — a faithful cognitive metric, not a bundled approximation.
• Self-mutation — PSMutant mutation-tests itself; the score must stay above the thresholds.break floor in psmutant.self.config.json.
• Compatibility — a real mutation run under every supported Pester and every supported PowerShell.

The two Fortigi modules dogfood each other: PSMutant gates its complexity with PSComplexity, and PSComplexity gates its test quality with PSMutant. Separately, code-scanning.yml uploads PSScriptAnalyzer findings to GitHub code scanning.

License

MIT © Fortigi