Claude Opus 5.5

vitest-5-0-0-3-b

Claude Code's read-only tools, web search on, ludion sync on

Question

My test passed even though the async assertion was wrong because I didn't await it. Does Vitest catch that?

Answer key

A correct answer says
A test fails if an async assertion (such as expect(...).resolves) is not awaited.
The stale answer
An un-awaited async assertion only prints a warning.

Answer

Yes, partly. I checked the Vitest 5.0.3 installed in this project (`node_modules/vitest/dist/chunks/index.C4tKZ0yW.js:1906`). If you forget to `await` an `expect(...).resolves` or `expect(...).rejects` assertion, the test fails with an error: "Promise returned by `expect(...).resolves...` was not awaited". Forgetting to `await` `expect.poll(...)` or `expect.element(...)` fails the same way (line 7636).

Vitest only catches those built-in async matchers, though. If your assertion sits inside your own async function, a `.then()` callback or a `setTimeout` that you never awaited, Vitest has no idea it's pending. The test finishes before the assertion runs, so it passes. That's probably what happened to you.

Two things help:
- **`expect.assertions(n)` or `expect.hasAssertions()`** at the top of the test fails it if the expected assertions never ran.
- **The `vitest/valid-expect` rule from `eslint-plugin-vitest`** flags a missing `await` before you even run the tests.

Answering cost $0.12.

Grade

correct

The judge's reason: It states that an un-awaited expect(...).resolves/rejects fails the test, and it does not present the warning-only behavior as current.

The judge's log

Session log

What the agent wrote, the tools it called, and what they returned, from docs/bench/2026-10-11-done3/runs/B-project-sync/vitest-5-0-0-3-b.jsonl. Thinking and the session's settings are left out. The raw log

  1. Tool call: Grep

    {
      "pattern": "was not awaited",
      "path": "C:\\Users\\haya0\\AppData\\Local\\Temp\\ludion-bench-s5aj2D\\work\\node_modules",
      "output_mode": "content",
      "head_limit": 10
    }
  2. Tool output

    node_modules\@vitest\browser\dist\locators-BaWxQJd7.js:1:[Omitted long matching line]
    node_modules\@vitest\browser\dist\context.js:15:			const error = new Error(`The call was not awaited. This method is asynchronous and must be awaited; otherwise, the call will not start to avoid unhandled rejections.`);
    node_modules\@vitest\browser\dist\context.js:88:				const error = new Error(`The call was not awaited. This method is asynchronous and must be awaited; otherwise, the call will not start to avoid unhandled rejections.`);
    node_modules\vitest\dist\chunks\index.C4tKZ0yW.js:1906:				const awaitError = /* @__PURE__ */ new Error(`Promise returned by \`${assertion}\` was not awaited. This assertion is asynchronous and must be awaited; otherwise, it is not guaranteed to complete before the test finishes:\n\nawait ${assertion}\n`);
    node_modules\vitest\dist\chunks\index.C4tKZ0yW.js:7636:						throw copyStackTrace(/* @__PURE__ */ new Error(`${assertionString} was not awaited. This assertion is asynchronous and must be awaited; otherwise, it is not executed to avoid unhandled rejections:\n\nawait ${assertionString}\n`), STACK_TRACE_ERROR);
  3. Tool call: Read

    {
      "file_path": "C:\\Users\\haya0\\AppData\\Local\\Temp\\ludion-bench-s5aj2D\\work\\node_modules\\vitest\\dist\\chunks\\index.C4tKZ0yW.js",
      "offset": 1890,
      "limit": 30
    }
  4. Tool output

    1890				if (!test.promises) return;
    1891				const index = test.promises.indexOf(promise);
    1892				if (index !== -1) test.promises.splice(index, 1);
    1893			});
    1894			// record promise
    1895			if (!test.promises) test.promises = [];
    1896			// setup `expect.soft` handler here instead of `wrapAssertion`
    1897			// to avoid double error tracking while keeping non-await promise detection.
    1898			if (isSoft) promise = promise.then(noop, (err) => {
    1899				handleTestError(test, err);
    1900			});
    1901			test.promises.push(promise);
    1902			let resolved = false;
    1903			test.onFinished ??= [];
    1904			test.onFinished.push(() => {
    1905				if (!resolved) {
    1906					const awaitError = /* @__PURE__ */ new Error(`Promise returned by \`${assertion}\` was not awaited. This assertion is asynchronous and must be awaited; otherwise, it is not guaranteed to complete before the test finishes:\n\nawait ${assertion}\n`);
    1907					if (error.stack) awaitError.stack = error.stack.replace(error.message, awaitError.message);
    1908					throw awaitError;
    1909				}
    1910			});
    1911			return {
    1912				then(onFulfilled, onRejected) {
    1913					resolved = true;
    1914					return promise.then(onFulfilled, onRejected);
    1915				},
    1916				catch(onRejected) {
    1917					resolved = true;
    1918					return promise.catch(onRejected);
    1919				},
  5. Tool call: Read

    {
      "file_path": "C:\\Users\\haya0\\AppData\\Local\\Temp\\ludion-bench-s5aj2D\\work\\node_modules\\vitest\\dist\\chunks\\index.C4tKZ0yW.js",
      "offset": 7620,
      "limit": 25
    }
  6. Tool output

    7620							clearTimeout(timerId);
    7621						}
    7622						if (lastError) {
    7623							await onSettled?.({
    7624								assertion,
    7625								status: "fail"
    7626							});
    7627							throwWithCause(lastError, STACK_TRACE_ERROR);
    7628						}
    7629					};
    7630					let awaited = false;
    7631					test.onFinished ??= [];
    7632					test.onFinished.push(() => {
    7633						if (!awaited) {
    7634							const negated = chai.util.flag(assertion, "negate") ? "not." : "";
    7635							const assertionString = `expect.${chai.util.flag(assertion, "_poll.element") ? "element(locator)" : "poll(assertion)"}.${negated}${String(key)}()`;
    7636							throw copyStackTrace(/* @__PURE__ */ new Error(`${assertionString} was not awaited. This assertion is asynchronous and must be awaited; otherwise, it is not executed to avoid unhandled rejections:\n\nawait ${assertionString}\n`), STACK_TRACE_ERROR);
    7637						}
    7638					});
    7639					let resultPromise;
    7640					// lets `expect.element` register the poll with the task deadline
    7641					const wrap = chai.util.flag(assertion, "_poll.wrap");
    7642					const start = () => resultPromise ||= wrap ? wrap(promise(), STACK_TRACE_ERROR) : promise();
    7643					// only .then is enough to check awaited, but we type this as `Promise<void>` in global types
    7644					// so let's follow it
  7. Agent

    Yes, partly. I checked the Vitest 5.0.3 installed in this project (`node_modules/vitest/dist/chunks/index.C4tKZ0yW.js:1906`). If you forget to `await` an `expect(...).resolves` or `expect(...).rejects` assertion, the test fails with an error: "Promise returned by `expect(...).resolves...` was not awaited". Forgetting to `await` `expect.poll(...)` or `expect.element(...)` fails the same way (line 7636).
    
    Vitest only catches those built-in async matchers, though. If your assertion sits inside your own async function, a `.then()` callback or a `setTimeout` that you never awaited, Vitest has no idea it's pending. The test finishes before the assertion runs, so it passes. That's probably what happened to you.
    
    Two things help:
    - **`expect.assertions(n)` or `expect.hasAssertions()`** at the top of the test fails it if the expected assertions never ran.
    - **The `vitest/valid-expect` rule from `eslint-plugin-vitest`** flags a missing `await` before you even run the tests.