Headline · l1-l2-bench-v1
DevRev Computer vs Claude Code MCP 2
Same Opus model family. Same tasks. Same judge. Matched on the realistic-API server.
Accuracyi
95.4%
vs 67.9% Claude · +28 pts
Tokens per correct answeri
1.7×
fewer than Claude (428K vs 717K)
Cost as data scalesi
+13%
tokens 1×→256x · Claude +65%
Matched comparison (best config, per scale)
| Scale | Computer acc. | Claude acc. | Δ accuracy | Tokens / correct |
|---|---|---|---|---|
| 1x | 96.4% | 69.3% | +27 pts | 424K / 475K |
| 4x | 97.1% | 65.0% | +32 pts | 383K / 667K |
| 16x | 92.1% | 70.0% | +22 pts | 419K / 760K |
| 64x | 96.4% | 68.6% | +28 pts | 430K / 868K |
| 256x | 95.0% | 66.4% | +29 pts | 485K / 817K |
Where the gap is
Claude stays close on simpler lookups but falls behind on multi-hop joins and cross-source synthesis (higher task levels).
Token efficiency (tokens per correct)
Lower is better. Computer answers correctly with far fewer tokens at every scale.
Total tokens vs data scale
Computer stays roughly flat as the dataset grows 1×→256x; Claude's token use climbs.