Claude Sonnet 5

Score

62

Task done

81.72%

65–95% 95% CI bootstrap over items · n=93 · measured 2026-08-25

Refusal quality

90.0%

names the gap 77% · offers a route 93% · 10 probes x 3 iterations

Fabrication

0%

0 of 10 probes (30 probe runs) · ≤31%

Dead-air

39.8%

117 of 294 turns

Tool silence

58.8%

117 of 199 tool-call turns

Stalled

22.7%

20 of 88 runs

TTFT p50

1.21s

1.21–1.37s p50–p90 · us-east4 direct

Cost / 1M tok

$10.00