vitest-5-0-0-24-b

Question

How does vitest list find tests without executing them?

Answer key

A correct answer says

`vitest list` collects tests by static parsing by default instead of running files.

The stale answer

`vitest list` executes test files to collect test names.

The change's lesson

Answer

`vitest list` uses the same collection phase as `vitest run` and stops before the run phase. Each test file is still imported in a worker, so top-level code and the `describe` callbacks execute. The `test`/`it` calls only register tasks in the suite tree, so the test bodies and their hooks (`beforeEach` etc.) never run. Vitest then prints the collected task names, or JSON with `--json`, along with filters like `-t`.

Because the files really are imported, side effects in them, in setup files or in their imports still happen. If you only want the matching file paths, `--filesOnly` lists them from the include globs without collecting tests.

I answered this from memory and didn't check the Vitest source or docs, so behaviour may differ slightly between versions.

Answering cost $0.06.

Grade

wrong

The judge's reason: It says `vitest list` imports and executes test files to collect tests, which is the stale behavior; the key says it parses them statically by default.

The judge's log

Session log

What the agent wrote, the tools it called, and what they returned, from docs/bench/2026-10-11-done3/runs/B-project/vitest-5-0-0-24-b.jsonl. Thinking and the session's settings are left out. The raw log

1. Agent

   `vitest list` uses the same collection phase as `vitest run` and stops before the run phase. Each test file is still imported in a worker, so top-level code and the `describe` callbacks execute. The `test`/`it` calls only register tasks in the suite tree, so the test bodies and their hooks (`beforeEach` etc.) never run. Vitest then prints the collected task names, or JSON with `--json`, along with filters like `-t`.
   
   Because the files really are imported, side effects in them, in setup files or in their imports still happen. If you only want the matching file paths, `--filesOnly` lists them from the include globs without collecting tests.
   
   I answered this from memory and didn't check the Vitest source or docs, so behaviour may differ slightly between versions.