You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
bug: --shuffle flag silently ignored after #5438 (sort() in run() overwrites shuffle from loadTests()) #5605
testFiles.sort() was moved from loadTests() to run() to fix worker suite distribution. Good fix — but it introduced a regression for --shuffle users.
The regression
In lib/codecept.js, the execution order is now:
// loadTests() — line ~217if(this.opts.shuffle){this.testFiles=shuffle(this.testFiles)// ✅ shuffled here}// run() — line 293 (added by #5438)this.testFiles.sort()// ❌ always sorts — overwrites the shuffle!// then:mocha.files=this.testFiles// receives sorted, not shuffled, files
sort() in run() is unconditional — it runs regardless of whether --shuffle was specified. This means --shuffle is silently ignored on every run after this commit.
Steps to reproduce
# Run twice with --shuffle — expect different order each time
npx codeceptjs run --shuffle
npx codeceptjs run --shuffle
# Both runs will produce identical alphabetical order — shuffle has no effect
Expected behavior
--shuffle randomizes test execution order. Each run should produce a different sequence.
Actual behavior
Both runs produce identical alphabetical order. The shuffle applied in loadTests() is overwritten by sort() in run().
Suggested fix
Guard the sort with a shuffle check:
// run() — only sort if shuffle is not activeif(!this.opts.shuffle){this.testFiles.sort()}
This preserves alphabetical order for normal runs while respecting shuffle when requested.
Happy to submit a PR if the fix looks right to maintainers.
Activity
fix: guard sort() with shuffle check so --shuffle is not silently ign…