Claude Opus 5.5
In a project, with and without ludion sync
+27pointsDifference between Without ludion sync and With ludion sync, in percentage points.
By subject
| Model | Condition | agents | vitest | wrangler | All |
|---|---|---|---|---|---|
| Claude Opus 5.5 | Claude Code's read-only toolsweb search onludion sync off | 11/1595% interval 48% to 89% | 6/1595% interval 20% to 64% | 7/1595% interval 25% to 70% | 24/4595% interval 39% to 67% |
| Claude Opus 5.5 | Claude Code's read-only toolsweb search onludion sync on | 14/1595% interval 70% to 99% | 11/1595% interval 48% to 89% | 11/1595% interval 48% to 89% | 36/4595% interval 66% to 89% |
Questions it didn't get right: Without ludion sync
How can I make wrangler versions upload pick the newest compatibility date automatically?
wranglerwrongwrangler >=4.124.0 changed what `--latest` does. It uses the latest compatibility date supported by this version of Wrangler. Silent: code written for older versions still runs, without an error.Log
Which data-location jurisdictions can I pass when creating a D1 database from the CLI?
wranglerwrongwrangler >=4.121.0 added `--jurisdiction us`. Create a D1 database in the US jurisdiction with wrangler d1 create and --jurisdiction us. Older versions don't have it.Log
Wrangler warned that my bundle is close to the size limit. Is that limit measured compressed or uncompressed, and how big is it?
wranglerwrongno lesson for this changeLog
Where did the wrangler r2 sql and wrangler pipelines commands go?
wranglerwrongwrangler >=4.145.0 renamed `wrangler r2 sql` to `wrangler basin sql`. Previous command paths remain available as hidden compatibility aliases. Silent: code written for older versions still runs, without an error.Log
How can I measure my Worker's startup time and bundle size locally with wrangler?
wranglerwrongno lesson for this changeLog
Is there a wrangler config setting for a Workflow's concurrency limit?
wranglerwrongwrangler >=4.127.0 added `workflows[].concurrency.limit`. The limit is the maximum number of Workflow instances that can run concurrently. Older versions don't have it.Log
What is the connect handler in a Worker, and how do I configure it in wrangler?
wranglerwrongwrangler >=4.125.0 added `connect`. Configure connect with protocol tcp and a port to receive raw socket connections during wrangler dev in the Worker connect(socket, env, ctx) handler. Older versions don't have it.Log
In the JSON for wrangler versions secret bulk, what does a null value do?
wranglerno answerwrangler >=4.148.0 changed what `wrangler versions secret bulk` does. Set a secret value to null in JSON input to remove it from the new Worker version. Silent: code written for older versions still runs, without an error.Log
My queued Agent callback can't see the request that queued it. Is that expected?
agentswrongagents >=0.24.0 changed what `queue()` does. Queued callbacks run from the alarm loop in a fresh invocation, so they no longer see the enqueuing request's connection or request through getCurrentAgent(). Silent: code written for older versions still runs, without an error.Log
What replaces subAgent() for managing dynamic agents?
agentswrongagents >=0.23.0 deprecated `subAgent()`; use `this.dynamicAgents.get()` instead. subAgent(), hasSubAgent(), listSubAgents(), abortSubAgent() and deleteSubAgent() still work, delegate to this.dynamicAgents, and are deprecated in place. Silent: code written for older versions still runs, without an error.Log
What base class does Agent extend in the Cloudflare Agents SDK?
agentswrongagents >=0.22.0 changed what `Agent` does. Agent now directly extends Cloudflare's DurableObject, with the required PartyServer runtime vendored into agents/lifecycle. Silent: code written for older versions still runs, without an error.Log
Is the second argument of a this.schedule() callback the Schedule object?
agentswrongno lesson for this changeLog
Is @vitest/runner still published as its own package?
vitestwrongno lesson for this changeLog
Which browser providers can Vitest browser mode use?
vitestwrongno lesson for this changeLog
How do I check that an element's text contains a phrase in Vitest browser mode?
vitestwrongvitest >=5.0.0 changed what `toHaveTextContent` does. The browser-mode matcher now validates that the text content is exactly equal to the expected string; toMatchTextContent performs a partial match.Log
Which folder does vitest --merge-reports read from by default?
vitestwrongvitest >=5.0.0 changed the default of `--merge-reports`. By default, Vitest stores all blob test results in .vitest/blob/, so they can be later merged using --merge-reports. Silent: code written for older versions still runs, without an error.Log
My getByText('Save') locator no longer matches a 'Save changes' button in Vitest. Why?
vitestwrongno lesson for this changeLog
Do I need extends: true for each inline project to pick up my root Vitest settings?
vitestwrongvitest >=5.0.0 changed the default of `extends`. Projects defined with an inline configuration inherit options from the root-level configuration. Silent: code written for older versions still runs, without an error.Log
Can expect.poll hang my Vitest test forever if the callback never settles?
vitestwrongno lesson for this changeLog
Does Vitest have something like jest-when for argument-based mock results?
vitestwrongvitest >=5.0.0 added `vi.when`. vi.when takes a spy and lets you define argument-specific behaviors. Older versions don't have it.Log
How does vitest list find tests without executing them?
vitestwrongvitest >=5.0.0 changed what `vitest list` does. It parses test files statically instead of running them to collect tests; pass --no-static-parse to run the files instead. Silent: code written for older versions still runs, without an error.Log
Questions it didn't get right: With ludion sync
Wrangler warned that my bundle is close to the size limit. Is that limit measured compressed or uncompressed, and how big is it?
wranglerwrongno lesson for this changeLog
Where did the wrangler r2 sql and wrangler pipelines commands go?
wranglerno answerwrangler >=4.145.0 renamed `wrangler r2 sql` to `wrangler basin sql`. Previous command paths remain available as hidden compatibility aliases. Silent: code written for older versions still runs, without an error.Log
How can I measure my Worker's startup time and bundle size locally with wrangler?
wranglerwrongno lesson for this changeLog
Is automatic resource provisioning in wrangler deploy still in beta?
wranglerno answerits lesson was not publishedLog
Why does native RPC to my Agent fail when I address it with idFromString()?
agentswrongno lesson for this changeLog
Which browser providers can Vitest browser mode use?
vitestwrongno lesson for this changeLog
My getByText('Save') locator no longer matches a 'Save changes' button in Vitest. Why?
vitestwrongno lesson for this changeLog
Can expect.poll hang my Vitest test forever if the callback never settles?
vitestwrongno lesson for this changeLog
Vitest throws when I call vi.mock inside beforeEach. Why?
vitestwrongno lesson for this changeLog
Every answer
| Question | Subject | Without ludion sync | With ludion sync |
|---|---|---|---|
wrangler-4-111-0-0-b | wrangler | correct | correct |
wrangler-4-122-0-2-b | wrangler | correct | correct |
wrangler-4-124-0-4-b | wrangler | wrong | correct |
wrangler-4-121-0-6-b | wrangler | correct | correct |
wrangler-4-121-0-7-b | wrangler | wrong | correct |
wrangler-4-118-0-8-b | wrangler | correct | correct |
wrangler-4-129-1-9-b | wrangler | wrong | wrong |
wrangler-4-141-0-10-b | wrangler | correct | correct |
wrangler-4-145-0-11-b | wrangler | wrong | no answer |
wrangler-4-116-0-16-b | wrangler | wrong | wrong |
wrangler-4-127-0-17-b | wrangler | wrong | correct |
wrangler-4-125-0-19-b | wrangler | wrong | correct |
wrangler-4-117-0-20-b | wrangler | correct | correct |
wrangler-4-113-0-21-b | wrangler | correct | no answer |
wrangler-4-148-0-24-b | wrangler | no answer | correct |
agents-0-27-0-0-b | agents | correct | correct |
agents-0-27-0-2-b | agents | correct | correct |
agents-0-25-0-3-b | agents | correct | wrong |
agents-0-24-0-5-b | agents | wrong | correct |
agents-0-24-0-6-b | agents | correct | correct |
agents-0-23-0-7-b | agents | wrong | correct |
agents-0-22-0-9-b | agents | wrong | correct |
agents-0-22-0-11-b | agents | correct | correct |
agents-0-22-0-12-b | agents | wrong | correct |
agents-0-22-0-13-b | agents | correct | correct |
agents-0-22-0-14-b | agents | correct | correct |
agents-0-21-0-16-b | agents | correct | correct |
agents-0-20-0-18-b | agents | correct | correct |
agents-0-20-0-19-b | agents | correct | correct |
agents-0-19-0-21-b | agents | correct | correct |
vitest-5-0-0-3-b | vitest | correct | correct |
vitest-5-0-0-4-b | vitest | correct | correct |
vitest-5-0-0-5-b | vitest | wrong | correct |
vitest-5-0-0-6-b | vitest | wrong | wrong |
vitest-5-0-0-7-b | vitest | wrong | correct |
vitest-5-0-0-9-b | vitest | wrong | correct |
vitest-5-0-0-12-b | vitest | wrong | wrong |
vitest-5-0-0-13-b | vitest | correct | correct |
vitest-5-0-0-14-b | vitest | wrong | correct |
vitest-5-0-0-15-b | vitest | wrong | wrong |
vitest-5-0-0-16-b | vitest | correct | wrong |
vitest-5-0-0-17-b | vitest | wrong | correct |
vitest-5-0-0-18-b | vitest | correct | correct |
vitest-5-0-0-20-b | vitest | correct | correct |
vitest-5-0-0-24-b | vitest | wrong | correct |
Without tools
Claude Opus 5.5
Claude Opus 5.5 doesn't know 28 of 29 measured lessons; 28 not measured. 95% interval 83% to 99%.
| Model | Condition | agents | vitest | wrangler | All |
|---|---|---|---|---|---|
| Claude Opus 5.5 | No toolsweb search offludion sync off | 1/995% interval 2% to 43% | 0/995% interval 0% to 30% | 0/1195% interval 0% to 26% | 1/2995% interval 1% to 17% |
Lessons it misses without tools
- wrangler >=4.122.0 changed the default of `nodejs_compat`. With a compatibility date of 2026-08-04 or later it is on; to turn it off, remove the positive flags and add both no_nodejs_compat and no_nodejs_compat_v2. Silent: code written for older versions still runs, without an error. wrangler
- wrangler >=4.124.0 changed what `--latest` does. It uses the latest compatibility date supported by this version of Wrangler. Silent: code written for older versions still runs, without an error. wrangler
- wrangler >=4.121.0 added `--jurisdiction us`. Create a D1 database in the US jurisdiction with wrangler d1 create and --jurisdiction us. Older versions don't have it. wrangler
- wrangler >=4.118.0 changed the default of `X_LOCAL_OBSERVABILITY`. wrangler dev now captures request traces and console logs into the Local Explorer Observability tab; set X_LOCAL_OBSERVABILITY=false to opt out. Silent: code written for older versions still runs, without an error. wrangler
- wrangler >=4.145.0 renamed `wrangler r2 sql` to `wrangler basin sql`. Previous command paths remain available as hidden compatibility aliases. Silent: code written for older versions still runs, without an error. wrangler
- wrangler >=4.127.0 added `workflows[].concurrency.limit`. The limit is the maximum number of Workflow instances that can run concurrently. Older versions don't have it. wrangler
- wrangler >=4.125.0 added `connect`. Configure connect with protocol tcp and a port to receive raw socket connections during wrangler dev in the Worker connect(socket, env, ctx) handler. Older versions don't have it. wrangler
- wrangler >=4.117.0 renamed `/cdn-cgi/handler/scheduled` to `/cdn-cgi/local/scheduled`. wrangler dev and the Vite plugin transparently rewrite the old paths to the new ones, meaning you can continue to use the old paths. Silent: code written for older versions still runs, without an error. wrangler
- wrangler >=4.148.0 changed what `wrangler versions secret bulk` does. Set a secret value to null in JSON input to remove it from the new Worker version. Silent: code written for older versions still runs, without an error. wrangler
- agents >=0.27.0 added `npx agents tui`. It is a terminal client for a conversation's Web Channel. Older versions don't have it. agents
- agents >=0.24.0 added `transport`. useAgent speaks the Agent protocol over one of two wires: cf-websocket (the default, a hibernating WebSocket) or the experimental capnweb transport. Older versions don't have it. agents
- agents >=0.23.0 deprecated `subAgent()`; use `this.dynamicAgents.get()` instead. subAgent(), hasSubAgent(), listSubAgents(), abortSubAgent() and deleteSubAgent() still work, delegate to this.dynamicAgents, and are deprecated in place. Silent: code written for older versions still runs, without an error. agents
- agents >=0.22.0 changed what `Agent` does. Agent now directly extends Cloudflare's DurableObject, with the required PartyServer runtime vendored into agents/lifecycle. Silent: code written for older versions still runs, without an error. agents
- agents >=0.22.0 changed the default of `throttle`. useAgentChat now coalesces updates every 50ms; pass throttle: false to render every chunk as it arrives, or a number to change the interval. Silent: code written for older versions still runs, without an error. agents
- agents >=0.22.0 added `Scheduler`. Scheduler, under agents/schedules, is a reusable Lifecycle capability for persistent delayed, dated, cron, and interval callbacks. Older versions don't have it. agents
- agents >=0.20.0 changed what `createMcpHandler` does. A function input to createMcpHandler is always treated as an SDK v2 factory, so concurrent Worker requests receive isolated server instances. Silent: code written for older versions still runs, without an error. agents
- agents >=0.20.0 changed what `MCPClientConnection` does. MCPClientConnection probes for stateless MCP with server/discover, then falls back to the legacy initialize handshake on the same connection when needed. Silent: code written for older versions still runs, without an error. agents
- vitest >=5.0.0 changed what `vitest.config` does. Previously, vitest running from a subdirectory was using a config file from a parent directory; pass the config explicitly with --config. Silent: code written for older versions still runs, without an error. vitest
- vitest >=5.0.0 changed what `toHaveTextContent` does. The browser-mode matcher now validates that the text content is exactly equal to the expected string; toMatchTextContent performs a partial match. vitest
- vitest >=5.0.0 changed the default of `--merge-reports`. By default, Vitest stores all blob test results in .vitest/blob/, so they can be later merged using --merge-reports. Silent: code written for older versions still runs, without an error. vitest
- vitest >=5.0.0 changed what `-t` does. The pattern is matched against the full name: the enclosing suite names and the test name joined with ' > ' (the same string shown in the reporter output). Silent: code written for older versions still runs, without an error. vitest
- vitest >=5.0.0 changed the default of `extends`. Projects defined with an inline configuration inherit options from the root-level configuration. Silent: code written for older versions still runs, without an error. vitest
- vitest >=5.0.0 added `vi.when`. vi.when takes a spy and lets you define argument-specific behaviors. Older versions don't have it. vitest
- vitest >=5.0.0 added `--repeats`. It repeats every test regardless of the result, which is useful for hunting flaky tests. Older versions don't have it. vitest
- vitest >=5.0.0 added `-p`. It is a shorthand for the --project filter. Older versions don't have it. vitest
- vitest >=5.0.0 changed what `vitest list` does. It parses test files statically instead of running them to collect tests; pass --no-static-parse to run the files instead. Silent: code written for older versions still runs, without an error. vitest
- wrangler >=4.121.0 changed what `DO_NOT_TRACK` does. Wrangler disables telemetry when DO_NOT_TRACK=1 is set, regardless of other telemetry settings. Silent: code written for older versions still runs, without an error. wrangler
- wrangler >=4.141.0 changed the default of `durable_objects.code_update_strategy`. When unset, the strategy defaults to deferred with a 5-minute maximum delay; configure mode and max_delay to set it. Silent: code written for older versions still runs, without an error. wrangler