Bars show reported task scores. Whiskers are approximate 95% intervals where standard errors are available. Different reasoning budgets are not equal-cost tests. Missing results are never zero.
DeepSWE 1.1 · 1.1 · reported score · Claude Code · Highest result across two harnesses56.60*Platform unreported Compare resultsTest conditions
Metric
reported score*
Checkpoint version
Not specified*
Harness
Claude Code*
Statistic
Highest result across two harnesses*
Temperature
1.0*
Top p
0.95*
Context length
256K tokens*
FrontierSWE · version unreported · reported score · Claude Code · Dominance recomputed from raw results73.50*Platform unreported Compare resultsTest conditions
QwenQoderBench · version unreported · reported score · Claude Code · Internal · avg@558.40*Platform unreported Compare resultsTest conditions
Metric
reported score*
Checkpoint version
Not specified*
Harness
Claude Code*
Statistic
avg@5*
Timeout
6 hours*
Task set
Internal*
Maximum output
32,768*
Temperature
1.0*
Context length
256K tokens*
QwenReactBench · version unreported · reported score · Claude Code · Internal1,724* EloPlatform unreported Compare resultsTest conditions
Metric
BT/Elo rating*
Checkpoint version
Not specified*
Languages
English and Chinese*
Judge
Multimodal automatic judge*
Task set
Internal*
Harness
Claude Code*
QwenSVGBench · version unreported · reported score · Not specified · Internal1,713* EloPlatform unreported Compare resultsTest conditions
Metric
BT/Elo rating*
Checkpoint version
Not specified*
Languages
English and Chinese*
Judge
Multimodal automatic judge*
Task set
Internal*
Harness
Not specified*
QwenSWEBench · version unreported · reported score · Claude Code · Internal · avg@380.70*Platform unreported Compare resultsTest conditions
Metric
reported score*
Checkpoint version
Not specified*
Harness
Claude Code*
Statistic
avg@3*
Timeout
8 hours*
Task set
Internal*
Maximum output
32,768*
Temperature
1.0*
Context length
256K tokens*
SkillsBench · version unreported · reported score · OpenCode · Public SkillsBench v1.1 · Average of three runs per task70.20*Platform unreported Compare resultsTest conditions
Metric
reported score*
Checkpoint version
Not specified*
Harness
OpenCode*
Tasks
87*
Statistic
Average of three runs per task*
Task set
Public SkillsBench v1.1*
SWE-bench Pro · version unreported · reported score · Claude Code · Refined task set with corrected problematic tasks67.70*Platform unreported Compare resultsTest conditions
Metric
reported score*
Checkpoint version
Not specified*
Harness
Claude Code*
Task set
Refined task set with corrected problematic tasks*