geekradar
Int
Explore GeekRadar

More to explore

Good gear All topics About us Say hello
GeekRadar / The comparison desk
COMPARE.
AI models
anthropic

Claude Opus 4.8

AI model Released

API mode
Chat*
Context window
1,000,000 tokens
Maximum output
128,000 tokens
Open weights
No

Capability & preference

General capability · ECI

A cross-benchmark capability estimate. Index points, not a percentage; intervals show uncertainty.

Claude Opus 4.8 159.6* Claude Opus 4.8 · 95% interval 157.4–162.4

Human preference · Arena

Overall text preferences from head-to-head votes. Not factual accuracy or a task-success rate.

Claude Opus 4.8 1454.0 Claude Opus 4.8 · 95% interval 1450.2–1457.9 · 65,121 votes · 2026-10-28

Model performance

Radar task score: four tests, equal weight. An experimental summary of these tasks, not a universal intelligence rating.

Claude Opus 4.8

Radar task score

65.0* / 100

4/4 core tests · experimental v1

8 benchmark results for this configuration

Scientific reasoning

GPQA Diamond · 25%

Expert-written science questions; strict multiple-choice scoring. Chance accuracy is 25%.

Claude Opus 4.8 91.0*% Approx. 95% interval: 87.3–94.8%

Novel problem solving

Mystery Game Puzzles · 25%

Best-next-move puzzles in an undisclosed game variant. A narrow reasoning test, not general intelligence.

Claude Opus 4.8 36.0*% Approx. 95% interval: 26.5–45.5%

Advanced mathematics

FrontierMath Tiers 1-3 · 25%

Private v2 mathematics problems with Python tools. Earlier versions and Tier 4 are excluded.

Claude Opus 4.8 80.0*% Approx. 95% interval: 75.3–84.7%

Factual answers

SimpleQA Verified · 25%

Correct factual answers under a guessing prompt, graded by a model. Not a hallucination or abstention score.

Claude Opus 4.8 53.0*% Approx. 95% interval: 49.9–56.1%

End-to-end software

MirrorCode · Separate result

Whole-program reimplementation with hidden tests; ML + Private, two languages. Budget: seven days and ten billion tokens per attempt.

Claude Opus 4.8 65.6*% Approx. 95% interval: 44.3–86.8%

Competition mathematics

OTIS Mock AIME 2024-2025 · Separate result

Competition problems with model-extracted answers checked against an exact key. Earlier grading procedures are excluded.

Claude Opus 4.8 98.3*% Approx. 95% interval: 95.6–100.0%

Research mathematics

FrontierMath Tier 4 v2 · Separate result

The hardest private FrontierMath problems, corrected v2 set, with Python tools. Separate from Tiers 1-3 in the core score.

Claude Opus 4.8 56.1*% Approx. 95% interval: 40.7–71.5%

Visual spatial reasoning

Furniture Assembly · Separate result

Assembly-error detection from photos and manuals, with zoom and Python tools. Uses model-assisted grading; not a general vision score.

Claude Opus 4.8 42.5*% Approx. 95% interval: 30.0–55.0%

Bars show reported task scores. Whiskers are approximate 95% intervals where standard errors are available. Different reasoning budgets are not equal-cost tests. Missing results are never zero.

Benchmark results

  • $OneMillion-Bench (expert score) · version unreported · reported score41.80*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*
    Judge
    gemini-3.1-pro-preview*
  • Agents' Last Exam (Pass / Score) · version unreported · Pass27*Platform unreported Compare results
    Test conditions
    Metric
    Pass*
    Checkpoint version
    Not specified*
  • Agents' Last Exam (Pass / Score) · version unreported · Score45.10*Platform unreported Compare results
    Test conditions
    Metric
    Score*
    Checkpoint version
    Not specified*
  • AndroidBench · version unreported · reported score · Public subset · avg@369.80*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*
    Tasks
    95*
    Statistic
    avg@3*
    Task set
    Public subset*
  • Automation-Bench (Pass@1) · version unreported · reported score · Public subset27.20*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*
    Tasks
    600*
    Task set
    Public subset*
  • CoWorkBench · version unreported · reported score · Internal72.30*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*
    Task set
    Internal*
  • DeepSWE 1.1 · 1.1 · reported score · Claude Code or mini-SWE-agent; winning harness unspecified · Highest result across two harnesses59*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*
    Harness
    Claude Code or mini-SWE-agent; winning harness unspecified*
    Statistic
    Highest result across two harnesses*
    Temperature
    1.0*
    Top p
    0.95*
    Context length
    256K tokens*
  • FrontierSWE · version unreported · reported score · Not specified · Dominance recomputed from raw results70*Platform unreported Compare results
    Test conditions
    Metric
    Dominance score*
    Checkpoint version
    Not specified*
    Harness
    Not specified*
    Statistic
    Dominance recomputed from raw results*
    Evaluation date
    2026-08-03*
    Variant
    MEAN@5 submissions*
  • GPQA Diamond · version unreported · reported score92*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*
  • HealthBench · version unreported · reported score52.40*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*
  • HLE · version unreported · reported score · Disabled45.70*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*
    Tool access
    Disabled*
  • HLE w/ tools · version unreported · reported score · Enabled57.90*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*
    Tool access
    Enabled*
  • IFBench · version unreported · reported score62.20*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*
  • JobBench · version unreported · reported score48.40*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*
  • LongBench v2 · 2 · reported score69.10*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*
  • MLS-Bench-Lite · version unreported · reported score · Not specified42.80*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*
    Harness
    Not specified*
  • MRCR v2 256K (8-needle) · 2 · reported score83.20*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*
    Variant
    8-needle*
    Context length
    256K tokens*
  • NL2Repo-Bench · version unreported · reported score · Claude Code · Repository acquisition commands disabled69.40*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*
    Harness
    Claude Code*
    Tool access
    Repository acquisition commands disabled*
  • PaperBench · version unreported · reported score · BasicAgent80.30*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*
    Harness
    BasicAgent*
    Variant
    Code-Dev*
    Judge
    Claude Opus 4.6*
    Runs
    3*
    Timeout
    12 hours per run*
  • PLawBench · version unreported · reported score69.60*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*
    Judge
    gemini-3.1-pro-preview*
  • PRBench-Finance · version unreported · reported score51.90*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*
  • PRBench-Legal · version unreported · reported score52.70*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*
  • QwenQoderBench · version unreported · reported score · Claude Code · Internal · avg@562.70*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*
    Harness
    Claude Code*
    Statistic
    avg@5*
    Timeout
    6 hours*
    Task set
    Internal*
    Maximum output
    32,768*
    Temperature
    1.0*
    Context length
    256K tokens*
  • QwenReactBench · version unreported · reported score · Claude Code · Internal1,694* EloPlatform unreported Compare results
    Test conditions
    Metric
    BT/Elo rating*
    Checkpoint version
    Not specified*
    Languages
    English and Chinese*
    Judge
    Multimodal automatic judge*
    Task set
    Internal*
    Harness
    Claude Code*
  • QwenSVGBench · version unreported · reported score · Not specified · Internal1,648* EloPlatform unreported Compare results
    Test conditions
    Metric
    BT/Elo rating*
    Checkpoint version
    Not specified*
    Languages
    English and Chinese*
    Judge
    Multimodal automatic judge*
    Task set
    Internal*
    Harness
    Not specified*
  • QwenSWEBench · version unreported · reported score · Claude Code · Internal · avg@384*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*
    Harness
    Claude Code*
    Statistic
    avg@3*
    Timeout
    8 hours*
    Task set
    Internal*
    Maximum output
    32,768*
    Temperature
    1.0*
    Context length
    256K tokens*
  • SkillsBench · version unreported · reported score · Claude Code · Public SkillsBench v1.1 · Average of three runs per task65.10*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*
    Harness
    Claude Code*
    Tasks
    87*
    Statistic
    Average of three runs per task*
    Task set
    Public SkillsBench v1.1*
  • Reported score · version unreported · intelligence55.70*Platform unreported Compare results
  • SWE-bench Pro · version unreported · reported score · Claude Code · Refined task set with corrected problematic tasks69.20*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*
    Harness
    Claude Code*
    Task set
    Refined task set with corrected problematic tasks*
    Temperature
    1.0*
    Top p
    0.95*
    Context length
    256K tokens*
  • Terminal Bench 2.1 · 2.1 · reported score · Terminus 2 · Best published result across harnesses84.60*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*
    Harness
    Terminus 2*
    Statistic
    Best published result across harnesses*
  • Toolathlon Verified (Pass@1) · version unreported · reported score76.20*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*
  • WideSearch · version unreported · reported score · Claude Code · Average item F1 over four runs72.90*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*
    Harness
    Claude Code*
    Statistic
    Average item F1 over four runs*
  • WorkSpaceBench · version unreported · reported score66.80*Platform unreported Compare results
    Test conditions
    Metric
    reported score*
    Checkpoint version
    Not specified*

Recorded results

Only the same benchmark, version, workload, scenario and unit share a row. Missing results are not zero. System results include the full submitted hardware and software configuration.

Arena text preference / chinesehuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,506.99 rating

Confidence interval: 1,497.33 – 1,516.64

4361 votes

Category
chinese
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference

1,506.05 rating

Confidence interval: 1,496.39 – 1,515.71

4342 votes

Category
chinese
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference
Arena text preference / codinghuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,490.99 rating

Confidence interval: 1,485.21 – 1,496.77

17403 votes

Category
coding
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference

1,487.47 rating

Confidence interval: 1,481.73 – 1,493.21

18091 votes

Category
coding
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference
Arena text preference / creative_writinghuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,454.56 rating

Confidence interval: 1,448.03 – 1,461.09

12718 votes

Category
creative_writing
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference

1,446.65 rating

Confidence interval: 1,440.16 – 1,453.15

12768 votes

Category
creative_writing
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference
Arena text preference / englishhuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,468.61 rating

Confidence interval: 1,463.63 – 1,473.58

28015 votes

Category
english
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference

1,460.14 rating

Confidence interval: 1,455.17 – 1,465.10

28736 votes

Category
english
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference
Arena text preference / exclude_tieshuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,463.83 rating

Confidence interval: 1,458.81 – 1,468.85

48076 votes

Category
exclude_ties
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference

1,454.52 rating

Confidence interval: 1,449.51 – 1,459.53

48910 votes

Category
exclude_ties
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference
Arena text preference / experthuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,503.17 rating

Confidence interval: 1,495.34 – 1,511.00

7061 votes

Category
expert
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference

1,491.63 rating

Confidence interval: 1,483.93 – 1,499.34

7403 votes

Category
expert
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference
Arena text preference / frenchhuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,479.74 rating

Confidence interval: 1,465.50 – 1,493.99

2185 votes

Category
french
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference

1,477.26 rating

Confidence interval: 1,463.08 – 1,491.44

2236 votes

Category
french
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference
Arena text preference / germanhuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,470.93 rating

Confidence interval: 1,452.25 – 1,489.61

1082 votes

Category
german
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference

1,446.63 rating

Confidence interval: 1,428.09 – 1,465.17

1107 votes

Category
german
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference
Arena text preference / hard_promptshuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,482.22 rating

Confidence interval: 1,477.67 – 1,486.77

42464 votes

Category
hard_prompts
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference

1,474.14 rating

Confidence interval: 1,469.62 – 1,478.66

43328 votes

Category
hard_prompts
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference
Arena text preference / hard_prompts_englishhuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,486.93 rating

Confidence interval: 1,481.28 – 1,492.59

18798 votes

Category
hard_prompts_english
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference

1,475.41 rating

Confidence interval: 1,469.77 – 1,481.05

19488 votes

Category
hard_prompts_english
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference
Arena text preference / industry_business_and_management_and_financial_operationshuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,458.18 rating

Confidence interval: 1,451.85 – 1,464.51

12742 votes

Category
industry_business_and_management_and_financial_operations
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference

1,453.92 rating

Confidence interval: 1,447.63 – 1,460.21

12785 votes

Category
industry_business_and_management_and_financial_operations
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference
Arena text preference / industry_entertainment_and_sports_and_mediahuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,440.52 rating

Confidence interval: 1,434.49 – 1,446.56

16141 votes

Category
industry_entertainment_and_sports_and_media
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference

1,433.69 rating

Confidence interval: 1,427.66 – 1,439.72

15988 votes

Category
industry_entertainment_and_sports_and_media
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference
Arena text preference / industry_legal_and_governmenthuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,476.78 rating

Confidence interval: 1,468.05 – 1,485.51

5508 votes

Category
industry_legal_and_government
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference

1,475.18 rating

Confidence interval: 1,466.40 – 1,483.97

5423 votes

Category
industry_legal_and_government
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference
Arena text preference / industry_life_and_physical_and_social_sciencehuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,477.22 rating

Confidence interval: 1,470.56 – 1,483.87

10434 votes

Category
industry_life_and_physical_and_social_science
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference

1,471.28 rating

Confidence interval: 1,464.69 – 1,477.86

10720 votes

Category
industry_life_and_physical_and_social_science
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference
Arena text preference / industry_mathematicalhuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,496.89 rating

Confidence interval: 1,486.27 – 1,507.50

3485 votes

Category
industry_mathematical
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference

1,481.14 rating

Confidence interval: 1,470.67 – 1,491.60

3626 votes

Category
industry_mathematical
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference
Arena text preference / industry_medicine_and_healthcarehuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,464.16 rating

Confidence interval: 1,454.81 – 1,473.50

4762 votes

Category
industry_medicine_and_healthcare
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference

1,460.18 rating

Confidence interval: 1,450.85 – 1,469.52

4753 votes

Category
industry_medicine_and_healthcare
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference
Arena text preference / industry_software_and_it_serviceshuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,488.00 rating

Confidence interval: 1,482.84 – 1,493.16

25108 votes

Category
industry_software_and_it_services
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference

1,482.83 rating

Confidence interval: 1,477.70 – 1,487.97

25737 votes

Category
industry_software_and_it_services
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference
Arena text preference / industry_writing_and_literature_and_languagehuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,461.69 rating

Confidence interval: 1,455.87 – 1,467.50

17215 votes

Category
industry_writing_and_literature_and_language
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference

1,452.79 rating

Confidence interval: 1,446.97 – 1,458.62

17338 votes

Category
industry_writing_and_literature_and_language
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference
Arena text preference / instruction_followinghuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,476.97 rating

Confidence interval: 1,471.65 – 1,482.29

22619 votes

Category
instruction_following
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference

1,463.00 rating

Confidence interval: 1,457.68 – 1,468.31

23241 votes

Category
instruction_following
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference
Arena text preference / japanesehuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,442.17 rating

Confidence interval: 1,420.46 – 1,463.88

899 votes

Category
japanese
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference

1,440.99 rating

Confidence interval: 1,419.70 – 1,462.27

907 votes

Category
japanese
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference
Arena text preference / koreanhuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,430.54 rating

Confidence interval: 1,411.32 – 1,449.76

1107 votes

Category
korean
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference

1,419.84 rating

Confidence interval: 1,400.73 – 1,438.95

1086 votes

Category
korean
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference
Arena text preference / longer_queryhuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,484.07 rating

Confidence interval: 1,478.95 – 1,489.20

30147 votes

Category
longer_query
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference

1,480.07 rating

Confidence interval: 1,474.96 – 1,485.18

30808 votes

Category
longer_query
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference
Arena text preference / mathhuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,485.30 rating

Confidence interval: 1,474.12 – 1,496.47

2970 votes

Category
math
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference

1,470.51 rating

Confidence interval: 1,459.60 – 1,481.42

3098 votes

Category
math
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference
Arena text preference / multi_turnhuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,475.70 rating

Confidence interval: 1,469.04 – 1,482.37

11189 votes

Category
multi_turn
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference

1,472.37 rating

Confidence interval: 1,465.70 – 1,479.04

11395 votes

Category
multi_turn
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference
Arena text preference / non_englishhuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,450.41 rating

Confidence interval: 1,445.77 – 1,455.05

35972 votes

Category
non_english
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference

1,444.44 rating

Confidence interval: 1,439.80 – 1,449.08

36384 votes

Category
non_english
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference
Arena text preference / overallhuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,461.30 rating

Confidence interval: 1,457.43 – 1,465.16

63994 votes

Category
overall
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference

1,454.03 rating

Confidence interval: 1,450.19 – 1,457.88

65121 votes

Category
overall
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference
Arena text preference / polishhuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,458.38 rating

Confidence interval: 1,440.78 – 1,475.98

1094 votes

Category
polish
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference

1,457.15 rating

Confidence interval: 1,438.93 – 1,475.37

1077 votes

Category
polish
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference
Arena text preference / russianhuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,475.02 rating

Confidence interval: 1,467.33 – 1,482.71

7106 votes

Category
russian
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference

1,466.64 rating

Confidence interval: 1,458.92 – 1,474.35

7064 votes

Category
russian
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference
Arena text preference / spanishhuman preference · 2026-10-28 · rating

Claude Opus 4.8

1,464.77 rating

Confidence interval: 1,450.07 – 1,479.48

1858 votes

Category
spanish
Evaluated configuration
Claude Opus 4.8 (High)
Evaluation type
human preference

1,453.94 rating

Confidence interval: 1,439.54 – 1,468.33

1944 votes

Category
spanish
Evaluated configuration
Claude Opus 4.8
Evaluation type
human preference
Arena text preference / chinesehuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,506.75 rating

Confidence interval: 1,497.13 – 1,516.37

4397 votes

Category
chinese
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference

1,506.50 rating

Confidence interval: 1,496.86 – 1,516.15

4372 votes

Category
chinese
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference
Arena text preference / codinghuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,490.69 rating

Confidence interval: 1,484.90 – 1,496.48

17427 votes

Category
coding
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference

1,487.34 rating

Confidence interval: 1,481.59 – 1,493.09

18069 votes

Category
coding
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference
Arena text preference / creative_writinghuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,454.28 rating

Confidence interval: 1,447.75 – 1,460.81

12797 votes

Category
creative_writing
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference

1,446.31 rating

Confidence interval: 1,439.82 – 1,452.81

12829 votes

Category
creative_writing
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference
Arena text preference / englishhuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,468.60 rating

Confidence interval: 1,463.62 – 1,473.58

28033 votes

Category
english
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference

1,460.11 rating

Confidence interval: 1,455.13 – 1,465.08

28731 votes

Category
english
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference
Arena text preference / exclude_tieshuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,463.52 rating

Confidence interval: 1,458.50 – 1,468.54

48154 votes

Category
exclude_ties
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference

1,454.35 rating

Confidence interval: 1,449.34 – 1,459.36

48949 votes

Category
exclude_ties
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference
Arena text preference / experthuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,502.52 rating

Confidence interval: 1,494.68 – 1,510.36

7067 votes

Category
expert
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference

1,491.73 rating

Confidence interval: 1,484.01 – 1,499.44

7406 votes

Category
expert
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference
Arena text preference / frenchhuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,481.05 rating

Confidence interval: 1,466.75 – 1,495.35

2188 votes

Category
french
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference

1,478.05 rating

Confidence interval: 1,463.81 – 1,492.29

2235 votes

Category
french
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference
Arena text preference / germanhuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,472.37 rating

Confidence interval: 1,453.59 – 1,491.14

1077 votes

Category
german
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference

1,447.61 rating

Confidence interval: 1,429.02 – 1,466.20

1108 votes

Category
german
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference
Arena text preference / hard_promptshuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,481.82 rating

Confidence interval: 1,477.27 – 1,486.37

42534 votes

Category
hard_prompts
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference

1,474.20 rating

Confidence interval: 1,469.67 – 1,478.73

43368 votes

Category
hard_prompts
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference
Arena text preference / hard_prompts_englishhuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,486.74 rating

Confidence interval: 1,481.08 – 1,492.39

18817 votes

Category
hard_prompts_english
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference

1,475.39 rating

Confidence interval: 1,469.74 – 1,481.04

19493 votes

Category
hard_prompts_english
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference
Arena text preference / industry_business_and_management_and_financial_operationshuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,457.76 rating

Confidence interval: 1,451.43 – 1,464.09

12750 votes

Category
industry_business_and_management_and_financial_operations
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference

1,453.98 rating

Confidence interval: 1,447.68 – 1,460.28

12774 votes

Category
industry_business_and_management_and_financial_operations
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference
Arena text preference / industry_entertainment_and_sports_and_mediahuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,440.64 rating

Confidence interval: 1,434.61 – 1,446.68

16188 votes

Category
industry_entertainment_and_sports_and_media
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference

1,433.52 rating

Confidence interval: 1,427.49 – 1,439.56

16015 votes

Category
industry_entertainment_and_sports_and_media
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference
Arena text preference / industry_legal_and_governmenthuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,476.80 rating

Confidence interval: 1,468.08 – 1,485.53

5534 votes

Category
industry_legal_and_government
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference

1,475.27 rating

Confidence interval: 1,466.48 – 1,484.05

5446 votes

Category
industry_legal_and_government
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference
Arena text preference / industry_life_and_physical_and_social_sciencehuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,476.99 rating

Confidence interval: 1,470.33 – 1,483.65

10441 votes

Category
industry_life_and_physical_and_social_science
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference

1,470.92 rating

Confidence interval: 1,464.33 – 1,477.51

10711 votes

Category
industry_life_and_physical_and_social_science
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference
Arena text preference / industry_mathematicalhuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,497.36 rating

Confidence interval: 1,486.71 – 1,508.02

3480 votes

Category
industry_mathematical
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference

1,480.73 rating

Confidence interval: 1,470.22 – 1,491.23

3619 votes

Category
industry_mathematical
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference
Arena text preference / industry_medicine_and_healthcarehuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,464.26 rating

Confidence interval: 1,454.92 – 1,473.61

4774 votes

Category
industry_medicine_and_healthcare
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference

1,460.19 rating

Confidence interval: 1,450.84 – 1,469.54

4754 votes

Category
industry_medicine_and_healthcare
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference
Arena text preference / industry_software_and_it_serviceshuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,488.12 rating

Confidence interval: 1,482.95 – 1,493.29

25123 votes

Category
industry_software_and_it_services
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference

1,482.84 rating

Confidence interval: 1,477.69 – 1,487.98

25722 votes

Category
industry_software_and_it_services
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference
Arena text preference / industry_writing_and_literature_and_languagehuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,460.95 rating

Confidence interval: 1,455.13 – 1,466.76

17290 votes

Category
industry_writing_and_literature_and_language
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference

1,452.52 rating

Confidence interval: 1,446.70 – 1,458.35

17385 votes

Category
industry_writing_and_literature_and_language
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference
Arena text preference / instruction_followinghuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,476.34 rating

Confidence interval: 1,471.02 – 1,481.67

22679 votes

Category
instruction_following
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference

1,463.07 rating

Confidence interval: 1,457.74 – 1,468.39

23258 votes

Category
instruction_following
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference
Arena text preference / japanesehuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,441.10 rating

Confidence interval: 1,419.16 – 1,463.04

892 votes

Category
japanese
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference

1,440.42 rating

Confidence interval: 1,418.88 – 1,461.96

895 votes

Category
japanese
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference
Arena text preference / koreanhuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,431.94 rating

Confidence interval: 1,412.67 – 1,451.20

1107 votes

Category
korean
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference

1,419.03 rating

Confidence interval: 1,399.86 – 1,438.20

1081 votes

Category
korean
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference
Arena text preference / longer_queryhuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,483.51 rating

Confidence interval: 1,478.39 – 1,488.64

30215 votes

Category
longer_query
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference

1,480.04 rating

Confidence interval: 1,474.93 – 1,485.16

30856 votes

Category
longer_query
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference
Arena text preference / mathhuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,486.18 rating

Confidence interval: 1,475.00 – 1,497.37

2969 votes

Category
math
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference

1,470.23 rating

Confidence interval: 1,459.29 – 1,481.17

3088 votes

Category
math
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference
Arena text preference / multi_turnhuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,475.29 rating

Confidence interval: 1,468.62 – 1,481.97

11189 votes

Category
multi_turn
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference

1,472.66 rating

Confidence interval: 1,465.98 – 1,479.34

11385 votes

Category
multi_turn
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference
Arena text preference / non_englishhuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,450.02 rating

Confidence interval: 1,445.38 – 1,454.67

36060 votes

Category
non_english
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference

1,444.27 rating

Confidence interval: 1,439.62 – 1,448.91

36440 votes

Category
non_english
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference
Arena text preference / overallhuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,461.08 rating

Confidence interval: 1,457.21 – 1,464.94

64100 votes

Category
overall
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference

1,453.90 rating

Confidence interval: 1,450.06 – 1,457.74

65172 votes

Category
overall
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference
Arena text preference / polishhuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,459.32 rating

Confidence interval: 1,441.68 – 1,476.96

1090 votes

Category
polish
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference

1,456.60 rating

Confidence interval: 1,438.36 – 1,474.84

1075 votes

Category
polish
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference
Arena text preference / russianhuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,473.98 rating

Confidence interval: 1,466.30 – 1,481.66

7149 votes

Category
russian
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference

1,466.11 rating

Confidence interval: 1,458.39 – 1,473.83

7100 votes

Category
russian
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference
Arena text preference / spanishhuman preference · 2026-10-15 · rating

Claude Opus 4.8

1,464.99 rating

Confidence interval: 1,450.27 – 1,479.71

1865 votes

Category
spanish
Evaluated configuration
claude-opus-4-8-high
Evaluation type
human preference

1,452.87 rating

Confidence interval: 1,438.42 – 1,467.33

1942 votes

Category
spanish
Evaluated configuration
claude-opus-4-8
Evaluation type
human preference

Run it locally

Inference · memory estimate

Claude Opus 4.8

Closed weights. No supported local deployment or GPU requirement is published.

API cost

USD / month

Standard text-token estimate. Excludes taxes, caching, batch discounts, reasoning and multimodal surcharges. Provider limits and regional availability may differ.

Claude Opus 4.8

Claude Opus 4.8 / Pricing tiers 4
Provider / scopeRateCurrencyAmount / unit
AWS BedrockClaude Opus 4.8Standard Global cross-region Cached inputUSD0.5 / 1M tokens
InputUSD5 / 1M tokens
OutputUSD25 / 1M tokens
Google Vertex AIClaude Opus 4.8Batch Token tier: 0 – 200,000 Cached inputUSD0.25 / 1M tokens
InputUSD2.5 / 1M tokens
OutputUSD12.5 / 1M tokens
Google Vertex AIClaude Opus 4.8Standard Token tier: 0 – 200,000 Cached inputUSD0.5 / 1M tokens
InputUSD5 / 1M tokens
OutputUSD25 / 1M tokens
Google Vertex AIClaude Opus 4.8Standard Token tier: 200,000 – No stated upper limit Cached inputUSD0.5 / 1M tokens
InputUSD5 / 1M tokens
OutputUSD25 / 1M tokens

Capabilities

Context window
1,000,000 tokens
Maximum input
1,000,000 tokens*
Maximum output
128,000 tokens
Reasoning
Yes
Tool calling
Yes
Structured output
Yes*
Open weights
No
Knowledge cutoff
2026-01
API mode
Chat*
Declared context window
1M*
Model type
LLM (reasoning)*
Declared modalities
Text, Vision → Text*
Image input
Yes*
PDF input
Yes*
Prompt caching
Yes*
Tool choice
Yes*
Web search
Yes*
Model license
Proprietary

Input modalities

  • text
  • image
  • pdf

Output modalities

  • text

System configuration

Detailed specifications

Specifications 1 · 13 fields
api providers
Anthropic, AWS, Vertex AI, Azure*
Scope and conditions
native_context
metadata
name
Convly AI Models Database*
description
Hand-verified specifications, licences, API pricing (USD) and local-hardware VRAM requirements for frontier and open-weight AI models.*
generated
2026-09-22T15:41:48+00:00*
model_count
43*
fields
  • slug*
  • name*
  • developer*
  • model_type*
  • modality*
  • parameters*
  • context_window*
  • max_output*
  • open_weights*
  • license*
  • release_date*
  • input_price*
  • output_price*
  • price_unit*
  • price_tiers*
  • vram_q4*
  • vram_q4_gb*
  • min_gpu*
  • intelligence*
  • api_providers*
  • official_url*
  • retired*
  • convly_url*
developer
Anthropic*
price_unit
Not reported*
price_tiers
Not reported*
context window
1M*
Scope and conditions
native_context
metadata
name
Convly AI Models Database*
description
Hand-verified specifications, licences, API pricing (USD) and local-hardware VRAM requirements for frontier and open-weight AI models.*
generated
2026-09-22T15:41:48+00:00*
model_count
43*
fields
  • slug*
  • name*
  • developer*
  • model_type*
  • modality*
  • parameters*
  • context_window*
  • max_output*
  • open_weights*
  • license*
  • release_date*
  • input_price*
  • output_price*
  • price_unit*
  • price_tiers*
  • vram_q4*
  • vram_q4_gb*
  • min_gpu*
  • intelligence*
  • api_providers*
  • official_url*
  • retired*
  • convly_url*
developer
Anthropic*
price_unit
Not reported*
price_tiers
Not reported*
developer
Anthropic*
Scope and conditions
native_context
metadata
name
Convly AI Models Database*
description
Hand-verified specifications, licences, API pricing (USD) and local-hardware VRAM requirements for frontier and open-weight AI models.*
generated
2026-09-22T15:41:48+00:00*
model_count
43*
fields
  • slug*
  • name*
  • developer*
  • model_type*
  • modality*
  • parameters*
  • context_window*
  • max_output*
  • open_weights*
  • license*
  • release_date*
  • input_price*
  • output_price*
  • price_unit*
  • price_tiers*
  • vram_q4*
  • vram_q4_gb*
  • min_gpu*
  • intelligence*
  • api_providers*
  • official_url*
  • retired*
  • convly_url*
developer
Anthropic*
price_unit
Not reported*
price_tiers
Not reported*
input price
5*
Scope and conditions
native_context
metadata
name
Convly AI Models Database*
description
Hand-verified specifications, licences, API pricing (USD) and local-hardware VRAM requirements for frontier and open-weight AI models.*
generated
2026-09-22T15:41:48+00:00*
model_count
43*
fields
  • slug*
  • name*
  • developer*
  • model_type*
  • modality*
  • parameters*
  • context_window*
  • max_output*
  • open_weights*
  • license*
  • release_date*
  • input_price*
  • output_price*
  • price_unit*
  • price_tiers*
  • vram_q4*
  • vram_q4_gb*
  • min_gpu*
  • intelligence*
  • api_providers*
  • official_url*
  • retired*
  • convly_url*
developer
Anthropic*
price_unit
Not reported*
price_tiers
Not reported*
intelligence
55.7*
Scope and conditions
native_context
metadata
name
Convly AI Models Database*
description
Hand-verified specifications, licences, API pricing (USD) and local-hardware VRAM requirements for frontier and open-weight AI models.*
generated
2026-09-22T15:41:48+00:00*
model_count
43*
fields
  • slug*
  • name*
  • developer*
  • model_type*
  • modality*
  • parameters*
  • context_window*
  • max_output*
  • open_weights*
  • license*
  • release_date*
  • input_price*
  • output_price*
  • price_unit*
  • price_tiers*
  • vram_q4*
  • vram_q4_gb*
  • min_gpu*
  • intelligence*
  • api_providers*
  • official_url*
  • retired*
  • convly_url*
developer
Anthropic*
price_unit
Not reported*
price_tiers
Not reported*
license
Proprietary*
Scope and conditions
native_context
metadata
name
Convly AI Models Database*
description
Hand-verified specifications, licences, API pricing (USD) and local-hardware VRAM requirements for frontier and open-weight AI models.*
generated
2026-09-22T15:41:48+00:00*
model_count
43*
fields
  • slug*
  • name*
  • developer*
  • model_type*
  • modality*
  • parameters*
  • context_window*
  • max_output*
  • open_weights*
  • license*
  • release_date*
  • input_price*
  • output_price*
  • price_unit*
  • price_tiers*
  • vram_q4*
  • vram_q4_gb*
  • min_gpu*
  • intelligence*
  • api_providers*
  • official_url*
  • retired*
  • convly_url*
developer
Anthropic*
price_unit
Not reported*
price_tiers
Not reported*
max output
128K*
Scope and conditions
native_context
metadata
name
Convly AI Models Database*
description
Hand-verified specifications, licences, API pricing (USD) and local-hardware VRAM requirements for frontier and open-weight AI models.*
generated
2026-09-22T15:41:48+00:00*
model_count
43*
fields
  • slug*
  • name*
  • developer*
  • model_type*
  • modality*
  • parameters*
  • context_window*
  • max_output*
  • open_weights*
  • license*
  • release_date*
  • input_price*
  • output_price*
  • price_unit*
  • price_tiers*
  • vram_q4*
  • vram_q4_gb*
  • min_gpu*
  • intelligence*
  • api_providers*
  • official_url*
  • retired*
  • convly_url*
developer
Anthropic*
price_unit
Not reported*
price_tiers
Not reported*
modality
Text, Vision → Text*
Scope and conditions
native_context
metadata
name
Convly AI Models Database*
description
Hand-verified specifications, licences, API pricing (USD) and local-hardware VRAM requirements for frontier and open-weight AI models.*
generated
2026-09-22T15:41:48+00:00*
model_count
43*
fields
  • slug*
  • name*
  • developer*
  • model_type*
  • modality*
  • parameters*
  • context_window*
  • max_output*
  • open_weights*
  • license*
  • release_date*
  • input_price*
  • output_price*
  • price_unit*
  • price_tiers*
  • vram_q4*
  • vram_q4_gb*
  • min_gpu*
  • intelligence*
  • api_providers*
  • official_url*
  • retired*
  • convly_url*
developer
Anthropic*
price_unit
Not reported*
price_tiers
Not reported*
model type
LLM (reasoning)*
Scope and conditions
native_context
metadata
name
Convly AI Models Database*
description
Hand-verified specifications, licences, API pricing (USD) and local-hardware VRAM requirements for frontier and open-weight AI models.*
generated
2026-09-22T15:41:48+00:00*
model_count
43*
fields
  • slug*
  • name*
  • developer*
  • model_type*
  • modality*
  • parameters*
  • context_window*
  • max_output*
  • open_weights*
  • license*
  • release_date*
  • input_price*
  • output_price*
  • price_unit*
  • price_tiers*
  • vram_q4*
  • vram_q4_gb*
  • min_gpu*
  • intelligence*
  • api_providers*
  • official_url*
  • retired*
  • convly_url*
developer
Anthropic*
price_unit
Not reported*
price_tiers
Not reported*
name
Claude Opus 4.8*
Scope and conditions
native_context
metadata
name
Convly AI Models Database*
description
Hand-verified specifications, licences, API pricing (USD) and local-hardware VRAM requirements for frontier and open-weight AI models.*
generated
2026-09-22T15:41:48+00:00*
model_count
43*
fields
  • slug*
  • name*
  • developer*
  • model_type*
  • modality*
  • parameters*
  • context_window*
  • max_output*
  • open_weights*
  • license*
  • release_date*
  • input_price*
  • output_price*
  • price_unit*
  • price_tiers*
  • vram_q4*
  • vram_q4_gb*
  • min_gpu*
  • intelligence*
  • api_providers*
  • official_url*
  • retired*
  • convly_url*
developer
Anthropic*
price_unit
Not reported*
price_tiers
Not reported*
open weights
no*
Scope and conditions
native_context
metadata
name
Convly AI Models Database*
description
Hand-verified specifications, licences, API pricing (USD) and local-hardware VRAM requirements for frontier and open-weight AI models.*
generated
2026-09-22T15:41:48+00:00*
model_count
43*
fields
  • slug*
  • name*
  • developer*
  • model_type*
  • modality*
  • parameters*
  • context_window*
  • max_output*
  • open_weights*
  • license*
  • release_date*
  • input_price*
  • output_price*
  • price_unit*
  • price_tiers*
  • vram_q4*
  • vram_q4_gb*
  • min_gpu*
  • intelligence*
  • api_providers*
  • official_url*
  • retired*
  • convly_url*
developer
Anthropic*
price_unit
Not reported*
price_tiers
Not reported*
output price
25*
Scope and conditions
native_context
metadata
name
Convly AI Models Database*
description
Hand-verified specifications, licences, API pricing (USD) and local-hardware VRAM requirements for frontier and open-weight AI models.*
generated
2026-09-22T15:41:48+00:00*
model_count
43*
fields
  • slug*
  • name*
  • developer*
  • model_type*
  • modality*
  • parameters*
  • context_window*
  • max_output*
  • open_weights*
  • license*
  • release_date*
  • input_price*
  • output_price*
  • price_unit*
  • price_tiers*
  • vram_q4*
  • vram_q4_gb*
  • min_gpu*
  • intelligence*
  • api_providers*
  • official_url*
  • retired*
  • convly_url*
developer
Anthropic*
price_unit
Not reported*
price_tiers
Not reported*
release date
2026*
Scope and conditions
native_context
metadata
name
Convly AI Models Database*
description
Hand-verified specifications, licences, API pricing (USD) and local-hardware VRAM requirements for frontier and open-weight AI models.*
generated
2026-09-22T15:41:48+00:00*
model_count
43*
fields
  • slug*
  • name*
  • developer*
  • model_type*
  • modality*
  • parameters*
  • context_window*
  • max_output*
  • open_weights*
  • license*
  • release_date*
  • input_price*
  • output_price*
  • price_unit*
  • price_tiers*
  • vram_q4*
  • vram_q4_gb*
  • min_gpu*
  • intelligence*
  • api_providers*
  • official_url*
  • retired*
  • convly_url*
developer
Anthropic*
price_unit
Not reported*
price_tiers
Not reported*
Specifications 2 · 72 fields
bedrock converse supports strict tools
No*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
bedrock output config effort ceiling
xhigh*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
cache creation input token cost
6.25e-6*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
cache creation input token cost
6.25e-6*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
cache creation input token cost above 1hr
1.0e-5*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
cache creation input token cost above 1hr
1.0e-5*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
cache creation input token cost batches
3.125e-6*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
cache read input token cost
5.0e-7*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
cache read input token cost
5.0e-7*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
cache read input token cost batches
2.5e-7*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
input cost per token
5.0e-6*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
input cost per token
5.0e-6*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
input cost per token batches
2.5e-6*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
litellm provider
bedrock_converse*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
litellm provider
anthropic*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
max input tokens
1,000,000*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
max input tokens
1,000,000*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
max output tokens
128,000*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
max output tokens
128,000*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
max tokens
128,000*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
max tokens
128,000*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
mode
chat*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
mode
chat*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
output cost per token
2.5e-5*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
output cost per token
2.5e-5*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
output cost per token batches
1.25e-5*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
prompt cache min tokens
1,024*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
prompt cache min tokens
1,024*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
provider specific entry
us
1.1*
fast
2.0*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
search context cost per query
search_context_size_high
0.01*
search_context_size_low
0.01*
search_context_size_medium
0.01*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
search context cost per query
search_context_size_high
0.01*
search_context_size_low
0.01*
search_context_size_medium
0.01*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports adaptive thinking
Yes*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports adaptive thinking
Yes*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports anthropic compaction
Yes*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports assistant prefill
No*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports assistant prefill
No*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports bedrock native structured output
Yes*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports computer use
Yes*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports computer use
Yes*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports fast mode
Yes*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports function calling
Yes*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports function calling
Yes*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports max reasoning effort
Yes*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports max reasoning effort
Yes*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports mid conversation system
Yes*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports mid conversation system
Yes*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports native structured output
No*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports native structured output
Yes*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports output config
Yes*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports output config
Yes*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports parallel tool use config
Yes*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports pdf input
Yes*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports pdf input
Yes*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports prompt caching
Yes*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports prompt caching
Yes*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports reasoning
Yes*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports reasoning
Yes*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports response schema
Yes*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports response schema
Yes*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports sampling params
No*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports sampling params
No*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports speed
Yes*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports thinking cache preservation
Yes*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports tool choice
Yes*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports tool choice
Yes*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports tool search
Yes*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports vision
Yes*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports vision
Yes*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports web search
Yes*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports xhigh reasoning effort
Yes*
Scope and conditions
native_context
provider
anthropic*
mode
chat*
full_native_model_id
claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
supports xhigh reasoning effort
Yes*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
token count model
model
anthropic.claude-opus-4-8*
litellm_provider
bedrock_mantle*
region
us-east-1*
Scope and conditions
native_context
provider
bedrock_converse*
mode
chat*
full_native_model_id
anthropic.claude-opus-4-8*
pricing_scope
Literal source field names and values retained; provider, region, cache/batch/threshold conditions and per-unit meaning must be checked. max_tokens is not automatically a context limit.*
Specifications 3 · 32 fields
QwenSWEBench
suite
QwenSWEBench
workload
reported score · Claude Code · Internal · avg@3
score
84.0*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
Harness
Claude Code*
Statistic
avg@3*
Timeout
8 hours*
Task set
Internal*
Maximum output
32,768*
Temperature
1.0*
Context length
256K tokens*
QwenQoderBench
suite
QwenQoderBench
workload
reported score · Claude Code · Internal · avg@5
score
62.7*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
Harness
Claude Code*
Statistic
avg@5*
Timeout
6 hours*
Task set
Internal*
Maximum output
32,768*
Temperature
1.0*
Context length
256K tokens*
QwenReactBench
suite
QwenReactBench
workload
reported score · Claude Code · Internal
score
1,694.0*
unit
Elo
Scope and conditions
Test conditions
Metric
BT/Elo rating*
Checkpoint version
Not specified*
Languages
English and Chinese*
Judge
Multimodal automatic judge*
Task set
Internal*
Harness
Claude Code*
QwenSVGBench
suite
QwenSVGBench
workload
reported score · Not specified · Internal
score
1,648.0*
unit
Elo
Scope and conditions
Test conditions
Metric
BT/Elo rating*
Checkpoint version
Not specified*
Languages
English and Chinese*
Judge
Multimodal automatic judge*
Task set
Internal*
Harness
Not specified*
CoWorkBench
suite
CoWorkBench
workload
reported score · Internal
score
72.3*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
Task set
Internal*
WorkSpaceBench
suite
WorkSpaceBench
workload
reported score
score
66.8*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
JobBench
suite
JobBench
workload
reported score
score
48.4*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
SkillsBench
suite
SkillsBench
workload
reported score · Claude Code · Public SkillsBench v1.1 · Average of three runs per task
score
65.1*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
Harness
Claude Code*
Tasks
87*
Statistic
Average of three runs per task*
Task set
Public SkillsBench v1.1*
Agents' Last Exam (Pass / Score)
suite
Agents' Last Exam (Pass / Score)
workload
Pass
score
27.0*
Scope and conditions
Test conditions
Metric
Pass*
Checkpoint version
Not specified*
Agents' Last Exam (Pass / Score)
suite
Agents' Last Exam (Pass / Score)
workload
Score
score
45.1*
Scope and conditions
Test conditions
Metric
Score*
Checkpoint version
Not specified*
Terminal Bench 2.1
suite
Terminal Bench 2.1
version
2.1
workload
reported score · Terminus 2 · Best published result across harnesses
score
84.6*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
Harness
Terminus 2*
Statistic
Best published result across harnesses*
Automation-Bench (Pass@1)
suite
Automation-Bench (Pass@1)
workload
reported score · Public subset
score
27.2*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
Tasks
600*
Task set
Public subset*
Toolathlon Verified (Pass@1)
suite
Toolathlon Verified (Pass@1)
workload
reported score
score
76.2*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
WideSearch
suite
WideSearch
workload
reported score · Claude Code · Average item F1 over four runs
score
72.9*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
Harness
Claude Code*
Statistic
Average item F1 over four runs*
HLE w/ tools
suite
HLE w/ tools
workload
reported score · Enabled
score
57.9*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
Tool access
Enabled*
GPQA Diamond
suite
GPQA Diamond
workload
reported score
score
92.0*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
HLE
suite
HLE
workload
reported score · Disabled
score
45.7*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
Tool access
Disabled*
IFBench
suite
IFBench
workload
reported score
score
62.2*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
$OneMillion-Bench (expert score)
suite
$OneMillion-Bench (expert score)
workload
reported score
score
41.8*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
Judge
gemini-3.1-pro-preview*
HealthBench
suite
HealthBench
workload
reported score
score
52.4*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
SWE-bench Pro
suite
SWE-bench Pro
workload
reported score · Claude Code · Refined task set with corrected problematic tasks
score
69.2*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
Harness
Claude Code*
Task set
Refined task set with corrected problematic tasks*
Temperature
1.0*
Top p
0.95*
Context length
256K tokens*
PLawBench
suite
PLawBench
workload
reported score
score
69.6*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
Judge
gemini-3.1-pro-preview*
PRBench-Legal
suite
PRBench-Legal
workload
reported score
score
52.7*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
PRBench-Finance
suite
PRBench-Finance
workload
reported score
score
51.9*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
MRCR v2 256K (8-needle)
suite
MRCR v2 256K (8-needle)
version
2
workload
reported score
score
83.2*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
Variant
8-needle*
Context length
256K tokens*
LongBench v2
suite
LongBench v2
version
2
workload
reported score
score
69.1*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
DeepSWE 1.1
suite
DeepSWE 1.1
version
1.1
workload
reported score · Claude Code or mini-SWE-agent; winning harness unspecified · Highest result across two harnesses
score
59.0*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
Harness
Claude Code or mini-SWE-agent; winning harness unspecified*
Statistic
Highest result across two harnesses*
Temperature
1.0*
Top p
0.95*
Context length
256K tokens*
NL2Repo-Bench
suite
NL2Repo-Bench
workload
reported score · Claude Code · Repository acquisition commands disabled
score
69.4*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
Harness
Claude Code*
Tool access
Repository acquisition commands disabled*
FrontierSWE
suite
FrontierSWE
workload
reported score · Not specified · Dominance recomputed from raw results
score
70.0*
Scope and conditions
Test conditions
Metric
Dominance score*
Checkpoint version
Not specified*
Harness
Not specified*
Statistic
Dominance recomputed from raw results*
Evaluation date
2026-08-03*
Variant
MEAN@5 submissions*
MLS-Bench-Lite
suite
MLS-Bench-Lite
workload
reported score · Not specified
score
42.8*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
Harness
Not specified*
PaperBench
suite
PaperBench
workload
reported score · BasicAgent
score
80.3*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
Harness
BasicAgent*
Variant
Code-Dev*
Judge
Claude Opus 4.6*
Runs
3*
Timeout
12 hours per run*
AndroidBench
suite
AndroidBench
workload
reported score · Public subset · avg@3
score
69.8*
Scope and conditions
Test conditions
Metric
reported score*
Checkpoint version
Not specified*
Tasks
95*
Statistic
avg@3*
Task set
Public subset*

* Provisional data