Skip to content

feat(tests): run_all.py --changed runs the checks that read what changed - #2887

Open
jordanella wants to merge 1 commit into
NotRexed:mainfrom
jordanella:feat/run-all-changed
Open

jordanella wants to merge 1 commit into
NotRexed:mainfrom
jordanella:feat/run-all-changed

Conversation

@jordanella

Copy link
Copy Markdown
Contributor

A full tests/run_all.py is about 490 checks and several minutes, and the ones an edit touches had to be picked by hand, which is how a check that cuts a function far from the edit gets missed (AGENTS.md: "Tests read the source"). This is the dev-loop proposal's D1 (#2636).

Change

tests/run_all.py --changed [BASE] takes the files changed since BASE's merge base (origin/main by default), committed or not, plus the untracked ones, and runs:

  • the checks whose text names one of them: by path, by file name standing alone, or as a module they import (import chunks, from chunks import);
  • the changed checks themselves, and the byte-compile check;
  • the C++ unit tests when something under gpu/ changed;
  • every check when the runner itself or a shared file under tests/ changed.

It prints how many files changed and how many checks read them, and that a pull request still needs the whole run. Nothing else in the runner changes.

Example

On this branch (the runner changed): --changed: 2 files changed since origin/main, every check (the runner or a shared test file changed). For a change to nx2/chunks.py alone it picks the chunk, emit and lift-cache checks (under a quarter of all), not the renderer's.

Testing

  • tests/test_run_all_changed.py (new): which checks a changed file picks (by path, by name and not inside a longer name, by import), that a changed check runs itself, that the runner or a shared test file runs everything, that an unnamed file runs none, and the files from git (committed, uncommitted and untracked) in a scratch repository.
  • run_all.py --changed origin/main on this branch, Windows: every check, as above: 423 passed, 21 failed, 79 skipped. The 21 fail the same way on main here (this PC's compilers and Linux-only checks).
  • The checks that read the changed files (10): pass, apart from test_spawn_emit, which fails the same way on main here.

See #2636 (D1).

🤖 Generated with Claude Code

A full run is ~490 checks, and the ones an edit touches had to be picked by hand. --changed [BASE] takes the files
changed since BASE's merge base (origin/main by default), committed or not, and the untracked ones, and runs the checks
whose text names one of them, by path, by name or as an imported module, the changed tests themselves, and the
byte-compile check; the C++ unit tests when gpu/ changed, and every check when the runner or a shared test file did. It
says that a pull request still needs the whole run. This is the dev-loop proposal's D1 (NotRexed#2636).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant