vitest-5-0-0-18-b

Question

What does vitest --repeats do?

Answer key

A correct answer says

The `--repeats` CLI option reruns tests a given number of times.

The stale answer

Repeats can only be set per test via the `repeats` test option.

The change's lesson

Answer

`vitest --repeats=<n>` runs every test `n` extra times, whether it passes or fails. The default is `0`. If any one of those runs fails, the whole test is marked as failed, so the flag is mainly for finding flaky tests. A test that sets its own `repeats` option (e.g. `test('x', { repeats: 3 }, ...)`) overrides the global value. `--retry` is different: it only re-runs a test after it fails, and the test passes if any attempt succeeds.

I found the flag documented only on the config page for Vitest's main branch, not in the CLI reference. It may be new, so check that your installed version supports it.

Sources:
- [Vitest config: repeats (main.vitest.dev)](https://main.vitest.dev/config/repeats)
- [docs/config/repeats.md mirror](https://next.tangled.org/vitest.dev/vitest/blob/main/docs/config/repeats.md)

Answering cost $0.10.

Grade

correct

The judge's reason: It says the --repeats CLI flag reruns every test a set number of times and treats the per-test repeats option only as an override, not the sole mechanism.

The judge's log

Session log

What the agent wrote, the tools it called, and what they returned, from docs/bench/2026-10-11-done3/runs/B-project/vitest-5-0-0-18-b.jsonl. Thinking and the session's settings are left out. The raw log

1. Tool call: ToolSearch

   {
     "query": "select:WebSearch",
     "max_results": 1
   }

2. Tool output

   {"type":"tool_reference","tool_name":"WebSearch"}

3. Tool call: WebSearch

   {
     "query": "vitest CLI --repeats option",
     "mode": "standard"
   }

4. Tool output

   Web search results for query: "vitest CLI --repeats option"
   
   Links: [{"title":"main.vitest.dev","url":"https://main.vitest.dev/config/repeats"},{"title":"next.tangled.org","url":"https://next.tangled.org/vitest.dev/vitest/blob/main/docs/config/repeats.md"},{"title":"github.com","url":"https://github.com/vitest-dev/vitest/blob/46dc57156700ebd5d9d99fc4c5e4180614984f27/docs/guide/cli.md"},{"title":"github.com","url":"https://github.com/vitest-dev/vitest/blob/43fa6baa6d10fe4363dc06a1af2da52d5746dd4c/docs/guide/cli.md"},{"title":"github.com","url":"https://github.com/Aslemammad/vitest/blob/5958d75d3c4bb5b2ccfead937ed05657cfb857db/docs/guide/cli.md"},{"title":"Command Line Interface","url":"https://v0.vitest.dev/guide/cli"},{"title":"next.tangled.org","url":"https://next.tangled.org/vitest.dev/vitest/blob/v1/docs/guide/cli.md"},{"title":"next.tangled.org","url":"https://next.tangled.org/vitest.dev/vitest/blob/v0/docs/guide/cli.md"},{"title":"next.tangled.org","url":"https://next.tangled.org/vitest.dev/vitest/blob/v2/docs/guide/cli.md"}]
   
   The `--repeats` flag is documented as a config option with a CLI form, `--repeats=<number>`. Per the official config reference (default 0), it runs each test a set number of times regardless of outcome. A test that sets its own repeats option overrides this value.
   
   The docs also say that a single failing repetition marks the whole test as failed, which makes the flag useful for checking flaky tests. Verifying that tests are stable across multiple runs is its main use, and any failure on any repetition fails the test.
   
   The CLI reference pages I found don't list `--repeats` in their options tables, so the config page is the main source. Search results also included mirror copies of the same docs, which show the same details.
   
   
   REMINDER: You MUST include the sources above in your response to the user using markdown hyperlinks.

5. Agent

   `vitest --repeats=<n>` runs every test `n` extra times, whether it passes or fails. The default is `0`. If any one of those runs fails, the whole test is marked as failed, so the flag is mainly for finding flaky tests. A test that sets its own `repeats` option (e.g. `test('x', { repeats: 3 }, ...)`) overrides the global value. `--retry` is different: it only re-runs a test after it fails, and the test passes if any attempt succeeds.
   
   I found the flag documented only on the config page for Vitest's main branch, not in the CLI reference. It may be new, so check that your installed version supports it.
   
   Sources:
   - [Vitest config: repeats (main.vitest.dev)](https://main.vitest.dev/config/repeats)
   - [docs/config/repeats.md mirror](https://next.tangled.org/vitest.dev/vitest/blob/main/docs/config/repeats.md)