Affected test selection — designing the rule that narrows tests together with the rule that falls back to all of them

Definition

Affected test selection runs only the tests that this change affects. In the import graph, which maps how files pull in each other with import, you start from the changed files, follow the edges backward, and run only the tests that reach those files.

Why it matters

If you run the tests for hundreds of files after changing a single line, the same work repeats on every pre-push hook and every PR CI run. When several jobs run in parallel (multiple working folders, multiple agents), the same full test suite shares the CPU at the same time, and the wait gets longer.

Selective runs reduce this cost, but they carry a risk. If the import graph is incomplete, you skip a test that should have run, and broken code gets through. That is why you have to design the "narrowing rule" together with the "fall back to everything when uncertain" rule.

How it works

Collect changed files (against a base point)
   └→ Deleted/renamed files, or a global config change?  ── yes ──→ run all tests
        │ no
        └→ Follow the import graph backward to select tests → run only the selected tests

Take the base point from the situation. PR CI uses the PR's base commit that the platform reports, and a push hook uses the "previous remote SHA of that branch" that Git passes to the hook. For a new branch, Git does not record which branch it was split from, so have a person state it explicitly through an input such as TEST_BASE.

In practice

Use what the tool provides and do not build the graph yourself.

# Vitest: only tests related to changes since the base commit
vitest run --changed "$BASE" --maxWorkers=2 --passWithNoTests

# Jest family
jest --changedSince="$BASE"

// vitest.config.ts — if any of these files change, rerun everything without selection
export default defineConfig({
  test: {
    forceRerunTriggers: [
      "**/package.json", "**/pnpm-lock.yaml", "**/tsconfig*.json",
      "**/vitest.setup.*", "messages/**", "public/**",
    ],
  },
});

• Send deletions and renames to the full run. A file that is gone cannot be a starting point in the graph, so you cannot find the tests that depended on it.
• Block a dirty working tree or a ref that is not the current HEAD (fail closed). This avoids passing while the base and the actual contents disagree.
• Pin the hook itself with regression tests: push range, base for a first push, dirty working tree, missing base, and the deleted-file fallback.
• Decide the worker count by measuring. Adding workers inside one process is different from launching several processes at the same time. Measure it under the real concurrent setup before deciding, and do not generalize to other environments.

Trade-offs

When not to use it

• In a small project where all tests finish in a few seconds, the selection logic only adds complexity.
• Do not use selective runs alone for the final merge or deploy candidate. Run the full suite and E2E separately.
• Do not trust only the static graph in a codebase with many dynamic imports or a lot of code generation.

Common mistakes

• Treating an unknown situation as a pass. It should be "if unsure, run everything", not "if unsure, pass".
• Fixing the base point to main or HEAD~1. It drifts from the actual PR target and from the previous remote SHA.
• Expecting selective runs to be fast when a shared module changes. For such changes, the related tests are close to the whole suite.
• Covering E2E tests that only flake in a parallel environment with sleep. Run only the failed test on its own first to tell whether it is an environment problem (playwright-flaky-vs-failed-triage).
• Claiming "it got faster" without measuring.

Related concepts

• playwright-flaky-vs-failed-triage — how to separate failures that only wobble under parallel execution
• mutation-testing — another way to check whether tests actually catch defects
• test-oracle — whether the selected tests have a reliable source of correct answers