Leaderboard
148 runs
🏆Best Pass Rate
97.1%
computer · Test4x-dataset-03 · Jul 2, 2026
⚡Fastest Agent
1m 17s
Claude Code MCP 2 · 1x-Dataset · avg per task
🪙Fewest Total Tokens
0
Claude Code MCP 2 · 16x-Dataset
All Runs — Grouped by Agent · Org
Runs grouped by agent and org — click a column header to sort groups and runs within each group
| # | Pass@1↓ | Pass@2↕ | Pass@4↕ | Avg Duration↕ | Avg Turns↕ | Total↕ | Input↕ | Output↕ | Cache Read↕ | Cache Write↕ | Run Date↕ | Model↕ | Notes | Trace |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| computer · Test4x-dataset-0311 runsBest: 97.1% | ||||||||||||||
| 1 | 97.1% | 100.0% | 100.0% | 2m 11s | 10.4 | 52.1M | 152K | 665K | 44.9M | 6.4M | Jul 2, 2026 | — | 📋 | View trace → |
| 2 | 95.0% | 99.8% | 100.0% | 1m 55s | 10.3 | 40.4M | 86K | 546K | 34.7M | 5.1M | Jul 1, 2026 | — | 📋 | View trace → |
| 3 | 93.8% | 99.5% | 100.0% | 1m 52s | 12.2 | 27.9M | 289K | 819K | 19.8M | 7.0M | Jun 22, 2026 | — | 📋 | View trace → |
| 4 | 92.9% | 99.4% | 100.0% | 1m 53s | 10.0 | 32.3M | 73K | 497K | 27.2M | 4.6M | Jul 1, 2026 | — | 📋 | View trace → |
| 5 | 92.3% | 99.5% | 100.0% | 1m 57s | 17.1 | 67.6M | 403K | 586K | 58.9M | 7.8M | Jun 23, 2026 | — | 📋 | View trace → |
| 6 | 90.8% | 98.3% | 100.0% | 1m 59s | 11.3 | 33.3M | 265K | 451K | 28.2M | 4.3M | Jun 25, 2026 | — | 📋 | View trace → |
| 7 | 90.0% | 98.3% | 100.0% | 1m 23s | 16.6 | 49.8M | 448K | 528K | 39.9M | 8.9M | Jun 21, 2026 | — | 📋 | View trace → |
| 8 | 89.2% | 98.8% | 100.0% | 1m 53s | 17.6 | 69.0M | 416K | 605K | 60.2M | 7.8M | Jun 23, 2026 | — | 📋 | View trace → |
| 9 | 89.2% | 98.0% | 100.0% | 3m 41s | 10.4 | 48.4M | 281K | 587K | 41.7M | 5.8M | Jun 24, 2026 | — | 📋 | View trace → |
| 10 | 87.7% | 98.1% | 100.0% | 1m 59s | 8.9 | 32.3M | 145K | 866K | 24.1M | 7.1M | Jun 18, 2026 | — | — | — |
| 11 | 73.1% | 85.6% | 96.4% | 2m 16s | 9.6 | 16.2M | 41K | 388K | 11.8M | 3.9M | Jun 15, 2026 | — | — | View trace → |
| computer · Test1x-dataset-019 runsBest: 96.4% | ||||||||||||||
| 12 | 96.4% | 100.0% | 100.0% | 2m 13s | 11.5 | 57.3M | 101K | 692K | 49.8M | 6.7M | Jul 2, 2026 | — | 📋 | View trace → |
| 13 | 90.8% | 99.5% | 100.0% | 1m 52s | 11.2 | 31.2M | 239K | 446K | 26.4M | 4.1M | Jun 26, 2026 | — | 📋 | View trace → |
| 14 | 90.7% | 99.4% | 100.0% | 1m 51s | 10.1 | 33.2M | 86K | 452K | 28.3M | 4.4M | Jul 2, 2026 | — | 📋 | View trace → |
| 15 | 86.9% | 96.9% | 99.8% | 1m 40s | 18.6 | 58.4M | 526K | 555K | 48.0M | 9.3M | Jun 23, 2026 | — | 📋 | View trace → |
| 16 | 85.4% | 95.9% | 99.5% | 2m 1s | 9.7 | 31.4M | 78K | 854K | 23.7M | 6.8M | Jun 18, 2026 | — | — | — |
| 17 | 83.1% | 95.6% | 99.5% | 1m 60s | 16.5 | 52.6M | 492K | 576K | 42.9M | 8.6M | Jun 23, 2026 | — | 📋 | View trace → |
| 18 | 76.9% | 88.9% | 94.8% | 2m 35s | 18.2 | 71.5M | 409K | 591K | 62.8M | 7.7M | Jun 21, 2026 | — | 📋 | View trace → |
| 19 | 76.1% | 87.5% | 94.0% | 4m 1s | 9.9 | 45.4M | 206K | 564K | 38.9M | 5.7M | Jun 24, 2026 | — | 📋 | View trace → |
| 20 | 74.4% | 88.2% | 92.3% | 2m 26s | 13.4 | 24.8M | 65K | 491K | 19.5M | 4.7M | Jun 15, 2026 | — | — | View trace → |
| computer · Test64x-dataset-033 runsBest: 96.4% | ||||||||||||||
| 21 | 96.4% | 99.8% | 100.0% | 2m 6s | 10.4 | 58.0M | 89K | 654K | 49.5M | 7.8M | Jul 2, 2026 | — | 📋 | View trace → |
| 22 | 94.3% | 99.7% | 100.0% | 1m 26s | 10.7 | 34.7M | 157K | 502K | 29.3M | 4.8M | Jul 1, 2026 | — | 📋 | View trace → |
| 23 | 90.0% | 97.3% | 99.8% | 1m 30s | 11.0 | 34.5M | 129K | 457K | 29.2M | 4.7M | Jun 26, 2026 | — | 📋 | View trace → |
| computer · Test16x-dataset-0611 runsBest: 96.2% | ||||||||||||||
| 24 | 96.2% | 99.7% | 100.0% | 2m 19s | 10.3 | 28.9M | 109K | 794K | 22.1M | 5.9M | Jun 26, 2026 | — | 📋 | View trace → |
| 25 | 94.6% | 99.0% | 100.0% | 1m 54s | 10.4 | 30.6M | 144K | 812K | 23.6M | 6.0M | Jun 22, 2026 | — | 📋 | View trace → |
| 26 | 93.6% | 99.4% | 100.0% | 1m 47s | 9.9 | 32.3M | 115K | 483K | 27.1M | 4.6M | Jul 1, 2026 | — | 📋 | View trace → |
| 27 | 93.1% | 97.3% | 99.5% | 1m 36s | 14.5 | 48.8M | 50K | 530K | 39.0M | 9.2M | Jun 22, 2026 | — | 📋 | View trace → |
| 28 | 92.3% | 98.1% | 99.8% | 1m 32s | 12.6 | 37.6M | 58K | 507K | 32.2M | 4.8M | Jun 22, 2026 | — | 📋 | View trace → |
| 29 | 92.1% | 98.9% | 100.0% | 2m 8s | 10.5 | 54.1M | 112K | 670K | 46.3M | 7.0M | Jul 2, 2026 | — | 📋 | View trace → |
| 30 | 91.5% | 98.6% | 100.0% | 2m 1s | 11.1 | 27.0M | 139K | 828K | 18.2M | 7.8M | Jun 22, 2026 | — | 📋 | View trace → |
| 31 | 87.7% | 96.9% | 99.8% | 2m 25s | 9.9 | 46.6M | 133K | 543K | 39.9M | 6.1M | Jun 24, 2026 | — | 📋 | View trace → |
| 32 | 86.9% | 91.8% | 92.3% | 1m 35s | 11.4 | 38.6M | 124K | 532K | 29.9M | 8.0M | Jun 23, 2026 | — | 📋 | View trace → |
| 33 | 84.6% | 93.8% | 98.5% | 2m 11s | 7.7 | 26.5M | 49K | 894K | 19.8M | 5.8M | Jun 18, 2026 | — | — | — |
| 34 | 79.2% | 84.4% | 87.5% | 2m 18s | 8.2 | 39.0M | 107K | 505K | 33.8M | 4.6M | Jun 24, 2026 | — | 📋 | View trace → |
| computer · OrgFix256x3 runsBest: 95.0% | ||||||||||||||
| 35 | 95.0% | 99.7% | 100.0% | 2m 6s | 10.3 | 64.5M | 171K | 660K | 54.4M | 9.3M | Jul 2, 2026 | — | 📋 | View trace → |
| 36 | 88.6% | 98.1% | 100.0% | 1m 58s | 10.8 | 37.2M | 104K | 476K | 31.6M | 5.0M | Jul 2, 2026 | — | 📋 | View trace → |
| 37 | 87.9% | 93.2% | 95.7% | 1m 55s | 10.3 | 35.2M | 110K | 464K | 29.5M | 5.1M | Jul 1, 2026 | — | 📋 | View trace → |
| computer · OrgFix9.56 runsBest: 85.9% | ||||||||||||||
| 38 | 85.9% | 96.9% | 100.0% | 2m 6s | 10.5 | 15.4M | 85K | 483K | 10.9M | 3.9M | Jun 4, 2026 | — | 📋 | View trace → |
| 39 | 80.8% | 94.9% | 99.5% | 2m 54s | 11.1 | 17.5M | 45K | 472K | 12.8M | 4.2M | Jun 9, 2026 | — | 📋 | View trace → |
| 40 | 71.8% | 84.6% | 91.3% | 2m 48s | 10.9 | 15.5M | 58K | 464K | 11.1M | 3.9M | Jun 4, 2026 | — | 📋 | View trace → |
| 41 | 66.7% | 80.5% | 90.8% | 1m 51s | 7.8 | 11.8M | 30K | 451K | 8.2M | 3.2M | Jun 4, 2026 | — | 📋 | View trace → |
| 42 | 65.4% | 80.0% | 93.8% | 1m 58s | 11.6 | 18.2M | 70K | 503K | 13.7M | 4.0M | Jun 4, 2026 | — | 📋 | View trace → |
| 43 | 60.3% | 77.4% | 88.7% | 1m 54s | 7.6 | 11.3M | 57K | 417K | 7.8M | 3.0M | Jun 3, 2026 | — | — | View trace → |
| computer · Test16x-dataset-023 runsBest: 85.9% | ||||||||||||||
| 44 | 85.9% | 91.8% | 92.3% | 2m 53s | 9.9 | 15.4M | 387K | 463K | 9.9M | 4.6M | Jun 16, 2026 | — | — | — |
| 45 | 70.5% | 81.5% | 89.2% | 2m 17s | 14.1 | 27.6M | 93K | 515K | 21.6M | 5.4M | Jun 15, 2026 | — | — | View trace → |
| 46 | 55.1% | 63.6% | 71.8% | 2m 35s | 12.7 | 23.7M | 58K | 528K | 18.1M | 5.0M | Jun 10, 2026 | — | 📋 | View trace → |
| Claude Code MCP 1 · 1x-Dataset16 runsBest: 83.3% | ||||||||||||||
| 47 | 83.3% | 90.3% | 92.3% | 1m 39s | 19.5 | 16.7M | 48K | 468K | 14.2M | 2.0M | Jun 15, 2026 | Opus 4.6 | 📋 | — |
| 48 | 75.6% | 86.7% | 91.8% | 1m 38s | 17.7 | 16.2M | 42K | 460K | 13.5M | 2.1M | Jun 4, 2026 | Opus 4.6 | 📋 | — |
| 49 | 75.6% | 85.1% | 89.7% | 1m 44s | 17.9 | 16.2M | 47K | 453K | 13.7M | 2.0M | Jun 16, 2026 | Opus 4.6 | 📋 | View trace → |
| 50 | 74.4% | 80.0% | 84.1% | 1m 45s | 18.6 | 16.6M | 43K | 470K | 14.0M | 2.1M | Jun 9, 2026 | Opus 4.6 | — | — |
| 51 | 74.4% | 82.6% | 89.2% | 1m 44s | 18.1 | 16.5M | 48K | 458K | 14.0M | 2.0M | Jun 17, 2026 | Opus 4.6 | 📋 | — |
| 52 | 73.9% | 83.8% | 87.5% | 1m 29s | 12.7 | 29.5M | 523K | 978K | 23.6M | 4.4M | Jun 18, 2026 | Opus 4.8 | 📋 | — |
| 53 | 73.1% | 85.6% | 96.4% | 1m 53s | 21.3 | 17.6M | 48K | 471K | 15.1M | 2.1M | Jun 3, 2026 | Opus 4.6 | 📋 | — |
| 54 | 70.5% | 73.9% | 76.4% | 1m 34s | 17.7 | 16.4M | 42K | 433K | 14.0M | 1.9M | Jun 3, 2026 | Opus 4.6 | 📋 | — |
| 55 | 69.2% | 78.5% | 84.1% | 1m 37s | 19.3 | 16.3M | 43K | 434K | 13.8M | 2.0M | Jun 4, 2026 | Opus 4.6 | 📋 | — |
| 56 | 69.2% | 77.6% | 83.3% | 1m 37s | 17.9 | 26.6M | 80K | 709K | 22.6M | 3.2M | Jun 26, 2026 | Opus 4.6 | — | View trace → |
| 57 | 66.7% | 79.5% | 90.3% | 1m 38s | 17.6 | 17.2M | 44K | 429K | 14.8M | 1.9M | Jun 3, 2026 | Opus 4.6 | 📋 | — |
| 58 | 65.4% | 75.4% | 81.8% | 1m 25s | 12.0 | 27.8M | 524K | 916K | 22.0M | 4.4M | Jun 24, 2026 | Opus 4.8 | 📋 | View trace → |
| 59 | 64.1% | 76.4% | 86.7% | 2m 36s | 16.4 | 19.5M | 48K | 801K | 16.5M | 2.2M | Jun 1, 2026 | Sonnet 4.6 | 📋 | View trace → |
| 60 | 64.1% | 72.3% | 79.5% | 2m 30s | 15.9 | 20.8M | 48K | 785K | 17.8M | 2.1M | Jun 2, 2026 | Sonnet 4.6 | 📋 | — |
| 61 | 43.6% | 55.4% | 63.6% | 2m 41s | 27.6 | 29.5M | 43K | 827K | 25.8M | 2.9M | May 25, 2026 | Sonnet 4.6 | 📋 | View trace → |
| 62 | 43.6% | 52.8% | 63.1% | 2m 47s | 24.4 | 28.3M | 43K | 841K | 25.1M | 2.3M | May 26, 2026 | Sonnet 4.6 | 📋 | View trace → |
| Claude Code MCP 1 · 16x-Dataset10 runsBest: 83.3% | ||||||||||||||
| 63 | 83.3% | 89.2% | 94.9% | 1m 50s | 22.4 | 18.8M | 48K | 507K | 15.5M | 2.8M | Jun 16, 2026 | Opus 4.6 | 📋 | View trace → |
| 64 | 80.0% | 88.9% | 94.1% | 1m 55s | 18.8 | 47.5M | 650K | 1.3M | 38.7M | 6.9M | Jun 18, 2026 | Opus 4.8 | 📋 | — |
| 65 | 78.2% | 89.2% | 94.9% | 1m 43s | 20.4 | 18.9M | 48K | 490K | 15.2M | 3.1M | Jun 17, 2026 | Opus 4.6 | 📋 | — |
| 66 | 76.1% | 85.1% | 92.8% | 1m 42s | 20.7 | 28.1M | 80K | 750K | 22.6M | 4.7M | Jun 26, 2026 | Opus 4.6 | — | View trace → |
| 67 | 74.4% | 85.6% | 89.7% | 1m 45s | 21.5 | 18.7M | 47K | 469K | 15.6M | 2.6M | Jun 16, 2026 | Opus 4.6 | 📋 | View trace → |
| 68 | 73.1% | 83.6% | 89.2% | 1m 51s | 21.3 | 19.1M | 48K | 512K | 15.9M | 2.7M | Jun 15, 2026 | Opus 4.6 | 📋 | — |
| 69 | 70.0% | 80.7% | 87.1% | 1m 51s | 18.3 | 44.6M | 611K | 1.2M | 36.1M | 6.7M | Jun 24, 2026 | Opus 4.8 | 📋 | View trace → |
| 70 | 64.1% | 71.8% | 76.4% | 1m 50s | 21.9 | 18.5M | 43K | 489K | 15.2M | 2.8M | Jun 11, 2026 | Opus 4.6 | 📋 | — |
| 71 | 57.7% | 63.6% | 66.7% | 2m 5s | 22.4 | 18.3M | 43K | 484K | 15.1M | 2.7M | Jun 10, 2026 | Opus 4.6 | 📋 | — |
| 72 | 0.0% | — | — | 4m 43s | 12.5 | 4.7M | 6K | 80K | 4.3M | 315K | Jun 15, 2026 | Opus 4.6 | — | View trace → |
| computer · OrgFix95 runsBest: 82.0% | ||||||||||||||
| 73 | 82.0% | 95.4% | 99.5% | 2m 26s | 12.6 | 19.6M | 36K | 523K | 14.6M | 4.4M | Jun 9, 2026 | — | — | — |
| 74 | 78.2% | 89.2% | 96.9% | 2m 56s | 11.8 | 19.0M | 63K | 513K | 14.5M | 4.0M | Jun 4, 2026 | — | — | — |
| 75 | 76.9% | 90.3% | 96.9% | 2m 21s | 13.1 | 20.1M | 77K | 535K | 15.3M | 4.2M | Jun 4, 2026 | — | — | — |
| 76 | 71.8% | 90.8% | 97.4% | 2m 42s | 12.2 | 18.4M | 71K | 564K | 13.9M | 3.9M | Jun 4, 2026 | — | — | — |
| 77 | 61.5% | 73.9% | 81.5% | 1m 54s | 9.8 | 13.2M | 41K | 523K | 9.5M | 3.2M | Jun 3, 2026 | — | — | — |
| Claude Code MCP 1 · 4x-Dataset8 runsBest: 79.5% | ||||||||||||||
| 78 | 79.5% | 89.2% | 92.3% | 1m 58s | 20.1 | 18.9M | 48K | 512K | 15.5M | 2.8M | Jun 16, 2026 | Opus 4.6 | 📋 | View trace → |
| 79 | 77.7% | 89.4% | 94.8% | 1m 46s | 16.1 | 41.2M | 555K | 1.1M | 34.0M | 5.5M | Jun 18, 2026 | Opus 4.8 | 📋 | — |
| 80 | 75.4% | 84.6% | 89.5% | 1m 47s | 18.9 | 30.3M | 80K | 813K | 24.3M | 5.1M | Jun 26, 2026 | Opus 4.6 | — | View trace → |
| 81 | 74.4% | 86.2% | 89.7% | 1m 50s | 22.2 | 20.0M | 47K | 481K | 16.6M | 2.9M | Jun 17, 2026 | Opus 4.6 | 📋 | — |
| 82 | 73.1% | 82.6% | 84.6% | 1m 52s | 19.4 | 19.1M | 48K | 517K | 15.5M | 3.0M | Jun 15, 2026 | Opus 4.6 | 📋 | — |
| 83 | 70.0% | 79.1% | 86.4% | 1m 40s | 15.9 | 38.0M | 533K | 1.1M | 31.1M | 5.4M | Jun 24, 2026 | Opus 4.8 | 📋 | View trace → |
| 84 | 62.8% | 71.3% | 74.4% | 1m 49s | 20.9 | 18.5M | 43K | 503K | 15.1M | 2.9M | Jun 11, 2026 | Opus 4.6 | 📋 | — |
| 85 | 59.0% | 65.6% | 69.2% | 1m 52s | 19.9 | 18.4M | 43K | 495K | 15.0M | 2.9M | Jun 9, 2026 | Opus 4.6 | 📋 | — |
| claude-code MCP 2 · 1x-Dataset13 runsBest: 78.5% | ||||||||||||||
| 86 | 78.5% | 87.9% | 92.8% | 1m 24s | 12.5 | 40.8M | 473K | 819K | 34.3M | 5.2M | Jun 30, 2026 | Opus 4.8 | — | View trace → |
| 87 | 70.7% | 81.6% | 91.4% | 1m 30s | 13.3 | 46.4M | 512K | 1.0M | 39.0M | 5.8M | Jul 1, 2026 | Opus 4.8 | — | View trace → |
| 88 | 70.5% | 77.4% | 82.0% | 1m 44s | 15.9 | 21.7M | 43K | 442K | 18.8M | 2.4M | Jun 9, 2026 | Opus 4.6 | 📋 | View trace → |
| 89 | 69.3% | 78.7% | 85.7% | 1m 33s | 13.0 | 46.1M | 509K | 1.0M | 38.8M | 5.8M | Jul 1, 2026 | Opus 4.8 | — | View trace → |
| 90 | 63.1% | 73.7% | 79.7% | 1m 17s | 11.2 | 34.8M | 409K | 727K | 29.4M | 4.3M | Jun 29, 2026 | Opus 4.8 | — | View trace → |
| 91 | 61.5% | 72.8% | 79.5% | 1m 25s | 12.8 | 26.1M | 44K | 427K | 22.4M | 3.3M | Jun 15, 2026 | Opus 4.7 | — | View trace → |
| 92 | 61.5% | 78.0% | 89.2% | 1m 50s | 17.0 | 21.7M | 43K | 475K | 18.8M | 2.4M | Jun 15, 2026 | Opus 4.6 | 📋 | View trace → |
| 93 | 60.0% | 72.1% | 81.8% | 1m 53s | 13.0 | 53.5M | 73K | 858K | 47.3M | 5.3M | Jun 23, 2026 | Opus 4.8 | 📋 | View trace → |
| 94 | 57.7% | 71.1% | 78.9% | 1m 44s | 14.5 | 57.6M | 73K | 994K | 50.7M | 5.9M | Jun 18, 2026 | Opus 4.8 | 📋 | View trace → |
| 95 | 56.1% | 66.3% | 75.6% | 1m 38s | 14.4 | 56.2M | 73K | 910K | 50.0M | 5.2M | Jun 25, 2026 | Opus 4.8 | 📋 | View trace → |
| 96 | 55.1% | 68.2% | 79.5% | 2m 36s | 16.5 | 22.5M | 43K | 483K | 19.5M | 2.5M | Jun 16, 2026 | Opus 4.6 | 📋 | View trace → |
| 97 | 52.9% | 60.6% | 67.5% | 1m 47s | 14.3 | 59.9M | 79K | 1.1M | 52.7M | 6.0M | Jul 1, 2026 | Opus 4.8 | 📋 | View trace → |
| 98 | 51.5% | 59.3% | 65.6% | 1m 42s | 13.8 | 55.5M | 73K | 924K | 48.9M | 5.6M | Jun 24, 2026 | Opus 4.8 | 📋 | View trace → |
| computer · OrgFix131 runBest: 78.2% | ||||||||||||||
| 99 | 78.2% | 90.8% | 97.4% | 1m 58s | 11.4 | 19.3M | 48K | 482K | 14.1M | 4.6M | Jun 15, 2026 | — | — | View trace → |
| claude-code MCP 2 · 16x-Dataset13 runsBest: 76.9% | ||||||||||||||
| 100 | 76.9% | 88.4% | 96.6% | 1m 55s | 17.4 | 64.2M | 477K | 1.0M | 57.1M | 5.6M | Jun 29, 2026 | Opus 4.8 | — | View trace → |
| 101 | 72.3% | 81.4% | 87.1% | 1m 50s | 18.6 | 63.8M | 489K | 1.0M | 55.9M | 6.3M | Jun 30, 2026 | Opus 4.8 | — | View trace → |
| 102 | 70.0% | 77.9% | 85.0% | 2m 4s | 18.8 | 74.5M | 540K | 1.3M | 65.7M | 6.9M | Jul 1, 2026 | Opus 4.8 | — | View trace → |
| 103 | 69.2% | 79.1% | 85.1% | 2m 14s | 19.1 | 88.2M | 74K | 1.1M | 80.3M | 6.8M | Jun 24, 2026 | Opus 4.8 | 📋 | View trace → |
| 104 | 66.1% | 74.4% | 76.9% | 1m 58s | 20.7 | 83.3M | 74K | 1.1M | 75.8M | 6.4M | Jun 26, 2026 | Opus 4.8 | 📋 | View trace → |
| 105 | 65.4% | 78.5% | 87.2% | 2m 32s | 21.6 | 33.9M | 250K | 628K | 29.6M | 3.4M | Jun 15, 2026 | Opus 4.6 | 📋 | View trace → |
| 106 | 60.0% | 68.7% | 76.1% | 2m 12s | 19.1 | 85.2M | 74K | 1.2M | 77.3M | 6.7M | Jun 23, 2026 | Opus 4.8 | 📋 | View trace → |
| 107 | 59.2% | 66.1% | 74.3% | 2m 15s | 20.0 | 82.4M | 74K | 1.2M | 74.1M | 7.0M | Jun 18, 2026 | Opus 4.8 | 📋 | View trace → |
| 108 | 59.0% | 71.8% | 82.6% | 2m 13s | 21.2 | 30.6M | 43K | 561K | 27.0M | 3.0M | Jun 16, 2026 | Opus 4.6 | 📋 | View trace → |
| 109 | 52.6% | 64.6% | 78.0% | 2m 17s | 21.4 | 30.7M | 229K | 597K | 26.5M | 3.3M | Jun 11, 2026 | Opus 4.6 | 📋 | View trace → |
| 110 | 47.4% | 60.5% | 71.3% | 2m 9s | 21.4 | 29.2M | 43K | 561K | 25.9M | 2.8M | Jun 9, 2026 | Opus 4.6 | 📋 | View trace → |
| 111 | 25.0% | — | — | 3m 33s | 21.8 | 2.2M | 2K | 47K | 1.9M | 179K | Jun 16, 2026 | Opus 4.6 | — | View trace → |
| 112 | 0.0% | 0.0% | 0.0% | 2m 60s | 1.0 | 0 | 0 | 0 | 0 | 0 | Jun 15, 2026 | Opus 4.6 | — | View trace → |
| computer · Test64x-dataset-012 runsBest: 76.9% | ||||||||||||||
| 113 | 76.9% | 79.8% | 82.0% | 1m 42s | 13.6 | 45.5M | 70K | 535K | 39.4M | 5.5M | Jun 23, 2026 | — | 📋 | View trace → |
| 114 | 76.9% | 81.0% | 83.3% | 1m 41s | 13.8 | 49.4M | 53K | 523K | 43.4M | 5.4M | Jun 23, 2026 | — | 📋 | View trace → |
| claude-code MCP 2 · 4x-Dataset13 runsBest: 74.6% | ||||||||||||||
| 115 | 74.6% | 82.0% | 87.1% | 1m 42s | 15.3 | 52.5M | 469K | 908K | 46.2M | 4.9M | Jun 29, 2026 | Opus 4.8 | — | View trace → |
| 116 | 74.4% | 85.6% | 94.4% | 2m 11s | 20.4 | 28.0M | 42K | 562K | 24.6M | 2.8M | Jun 15, 2026 | Opus 4.6 | 📋 | View trace → |
| 117 | 73.9% | 81.4% | 86.4% | 1m 37s | 15.5 | 55.4M | 486K | 890K | 48.7M | 5.3M | Jun 30, 2026 | Opus 4.8 | — | View trace → |
| 118 | 69.2% | 78.8% | 84.0% | 1m 47s | 17.1 | 71.0M | 74K | 974K | 64.5M | 5.5M | Jun 25, 2026 | Opus 4.8 | 📋 | View trace → |
| 119 | 66.9% | 74.7% | 79.7% | 1m 57s | 17.4 | 72.8M | 74K | 1.0M | 65.5M | 6.2M | Jun 23, 2026 | Opus 4.8 | 📋 | View trace → |
| 120 | 65.4% | 73.2% | 79.2% | 1m 59s | 16.8 | 72.4M | 74K | 1.0M | 65.6M | 5.6M | Jun 24, 2026 | Opus 4.8 | 📋 | View trace → |
| 121 | 65.0% | 72.4% | 79.7% | 1m 40s | 15.7 | 60.7M | 519K | 1.1M | 53.2M | 5.9M | Jul 1, 2026 | Opus 4.8 | — | View trace → |
| 122 | 57.7% | 77.4% | 93.8% | 2m 43s | 18.9 | 26.9M | 43K | 527K | 23.7M | 2.7M | Jun 16, 2026 | Opus 4.6 | 📋 | View trace → |
| 123 | 57.7% | 66.7% | 73.9% | 2m 1s | 17.6 | 75.9M | 73K | 1.1M | 68.7M | 6.0M | Jun 18, 2026 | Opus 4.8 | 📋 | View trace → |
| 124 | 52.6% | 62.1% | 71.3% | 2m 48s | 18.6 | 27.3M | 46K | 521K | 24.1M | 2.6M | Jun 10, 2026 | Opus 4.6 | 📋 | View trace → |
| 125 | 47.4% | 59.0% | 70.8% | 2m 8s | 19.1 | 26.8M | 43K | 547K | 23.5M | 2.7M | Jun 11, 2026 | Opus 4.6 | 📋 | View trace → |
| 126 | 12.8% | 16.9% | 22.1% | 4m 8s | 17.8 | 24.0M | 43K | 388K | 21.4M | 2.2M | Jun 16, 2026 | Opus 4.6 | — | View trace → |
| 127 | 0.0% | — | — | 3m 36s | 20.6 | 2.9M | 4K | 68K | 2.5M | 291K | Jun 16, 2026 | Opus 4.6 | — | View trace → |
| computer · FieldPOC-012 runsBest: 74.4% | ||||||||||||||
| 128 | 74.4% | 90.3% | 96.9% | 2m 29s | 13.8 | 21.1M | 73K | 553K | 16.4M | 4.0M | Jun 2, 2026 | — | — | — |
| 129 | 71.8% | 86.2% | 98.0% | 2m 18s | 11.4 | 16.5M | 69K | 480K | 12.6M | 3.4M | Jun 1, 2026 | — | — | View trace → |
| Claude Code MCP 1 · 64x-Dataset2 runsBest: 73.9% | ||||||||||||||
| 130 | 73.9% | 83.6% | 90.4% | 1m 46s | 20.1 | 28.2M | 80K | 765K | 23.1M | 4.2M | Jun 26, 2026 | Opus 4.6 | — | View trace → |
| 131 | 69.2% | 78.5% | 86.0% | 2m 15s | 21.2 | 64.7M | 736K | 1.5M | 50.6M | 11.8M | Jun 24, 2026 | Opus 4.8 | 📋 | View trace → |
| Claude Code MCP 1 · 256x-Dataset2 runsBest: 73.1% | ||||||||||||||
| 132 | 73.1% | 81.7% | 88.3% | 2m 20s | 20.6 | 76.1M | 793K | 1.6M | 58.5M | 15.2M | Jun 24, 2026 | Opus 4.8 | 📋 | View trace → |
| 133 | 71.5% | 82.4% | 89.2% | 1m 47s | 19.5 | 29.6M | 80K | 759K | 24.1M | 4.7M | Jun 26, 2026 | Opus 4.6 | — | View trace → |
| Claude Code MCP 2 · 64x-Dataset6 runsBest: 72.3% | ||||||||||||||
| 134 | 72.3% | 82.2% | 88.4% | 2m 11s | 21.8 | 75.0M | 581K | 1.2M | 65.6M | 7.6M | Jun 30, 2026 | Opus 4.8 | — | View trace → |
| 135 | 68.6% | 76.3% | 83.6% | 2m 18s | 21.1 | 83.4M | 602K | 1.5M | 73.5M | 7.9M | Jul 1, 2026 | Opus 4.8 | — | View trace → |
| 136 | 68.5% | 82.2% | 90.6% | 1m 54s | 19.4 | 60.4M | 481K | 1.0M | 52.6M | 6.3M | Jun 29, 2026 | Opus 4.8 | — | View trace → |
| 137 | 62.3% | 71.8% | 79.3% | 2m 8s | 23.6 | 87.4M | 75K | 1.2M | 79.6M | 6.5M | Jun 26, 2026 | Opus 4.8 | 📋 | View trace → |
| 138 | 60.8% | 69.4% | 77.3% | 2m 39s | 22.6 | 98.5M | 1.3M | 1.4M | 88.2M | 7.7M | Jun 24, 2026 | Opus 4.8 | 📋 | View trace → |
| 139 | 47.7% | 56.6% | 64.6% | 2m 34s | 22.2 | 91.2M | 754K | 1.3M | 81.5M | 7.6M | Jun 23, 2026 | Opus 4.8 | 📋 | View trace → |
| Claude Code MCP 2 · 256x-Dataset5 runsBest: 72.3% | ||||||||||||||
| 140 | 72.3% | 81.2% | 87.1% | 2m 28s | 17.5 | 54.7M | 420K | 996K | 47.7M | 5.6M | Jun 29, 2026 | Opus 4.8 | — | View trace → |
| 141 | 69.2% | 78.8% | 85.9% | 2m 17s | 24.6 | 75.3M | 560K | 1.3M | 65.5M | 7.9M | Jun 30, 2026 | Opus 4.8 | — | View trace → |
| 142 | 66.4% | 78.4% | 86.2% | 2m 37s | 21.9 | 76.0M | 523K | 1.3M | 66.4M | 7.7M | Jul 1, 2026 | Opus 4.8 | — | View trace → |
| 143 | 59.2% | 68.2% | 74.3% | 3m 6s | 24.4 | 93.9M | 230K | 1.4M | 84.9M | 7.4M | Jun 24, 2026 | Opus 4.8 | 📋 | View trace → |
| 144 | 56.9% | 65.8% | 73.2% | 2m 43s | 27.6 | 92.7M | 226K | 1.4M | 83.4M | 7.6M | Jun 26, 2026 | Opus 4.8 | 📋 | View trace → |
| computer · OrgFix52 runsBest: 62.8% | ||||||||||||||
| 145 | 62.8% | 70.8% | 75.9% | 1m 54s | 5.8 | 11.1M | 59K | 408K | 7.9M | 2.7M | Jun 2, 2026 | — | 📋 | View trace → |
| 146 | 56.4% | 68.2% | 74.4% | 2m 9s | 7.7 | 12.0M | 60K | 448K | 8.8M | 2.7M | Jun 2, 2026 | — | 📋 | View trace → |
| computer · Test4x-dataset-011 runBest: 57.7% | ||||||||||||||
| 147 | 57.7% | 69.2% | 79.0% | 2m 35s | 11.2 | 19.2M | 41K | 490K | 14.0M | 4.6M | Jun 10, 2026 | — | 📋 | — |
| computer · 1MD30jAhtt1 runBest: 53.8% | ||||||||||||||
| 148 | 53.8% | — | — | 2m 53s | 9.9 | 4.0M | 4K | 53K | 3.5M | 431K | Jun 23, 2026 | — | 📋 | View trace → |
Pass Rate Progress Over Time
Pass@1 trend per agent variant — each point is one run, grouped by date