GeekRadar / The comparison desk

COMPARE.

AI models

anthropic

Claude Sonnet 3.5 v2

AI model Released Oct 22, 2024

Context window: 200,000 tokens
Maximum output: 8,192 tokens
Open weights: No

Capability & preference

Human preference · Arena

Overall text preferences from head-to-head votes. Not factual accuracy or a task-success rate.

Claude Sonnet 3.5 v2 1297.8 claude-3-5-sonnet-20241022 · 95% interval 1294.6–1301.0 · 87,769 votes · 2026-10-15

Model performance

Radar task score: four tests, equal weight. An experimental summary of these tasks, not a universal intelligence rating.

Claude Sonnet 3.5 v2

Tested configuration Radar task score

Unrated

1/4 core tests · not enough evidence

1 benchmark results for this configuration

Scientific reasoning

GPQA Diamond · 25%

Expert-written science questions; strict multiple-choice scoring. Chance accuracy is 25%.

Claude Sonnet 3.5 v2 55.3^*% Approx. 95% interval: 49.8–60.8% · 2025-01-27 · Older result

Novel problem solving

Mystery Game Puzzles · 25%

Best-next-move puzzles in an undisclosed game variant. A narrow reasoning test, not general intelligence.

Claude Sonnet 3.5 v2 Not available

Advanced mathematics

FrontierMath Tiers 1-3 · 25%

Private v2 mathematics problems with Python tools. Earlier versions and Tier 4 are excluded.

Claude Sonnet 3.5 v2 Not available

Factual answers

SimpleQA Verified · 25%

Correct factual answers under a guessing prompt, graded by a model. Not a hallucination or abstention score.

Claude Sonnet 3.5 v2 Not available

Bars show reported task scores. Whiskers are approximate 95% intervals where standard errors are available. Different reasoning budgets are not equal-cost tests. Missing results are never zero.

Recorded results

Only the same benchmark, version, workload, scenario and unit share a row. Missing results are not zero. System results include the full submitted hardware and software configuration.

Arena text preference / chinesehuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,271.63 rating

Confidence interval: 1,263.43 – 1,279.83

6921 votes

Oct 15, 2026

Category: chinese
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / codinghuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,342.28 rating

Confidence interval: 1,336.81 – 1,347.76

14906 votes

Oct 15, 2026

Category: coding
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / creative_writinghuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,291.88 rating

Confidence interval: 1,285.62 – 1,298.13

11714 votes

Oct 15, 2026

Category: creative_writing
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / englishhuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,307.02 rating

Confidence interval: 1,303.09 – 1,310.94

45072 votes

Oct 15, 2026

Category: english
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / exclude_tieshuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,235.80 rating

Confidence interval: 1,231.52 – 1,240.09

61470 votes

Oct 15, 2026

Category: exclude_ties
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / experthuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,264.33 rating

Confidence interval: 1,255.14 – 1,273.52

5042 votes

Oct 15, 2026

Category: expert
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / frenchhuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,305.05 rating

Confidence interval: 1,283.83 – 1,326.27

848 votes

Oct 15, 2026

Category: french
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / germanhuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,297.44 rating

Confidence interval: 1,283.99 – 1,310.88

2025 votes

Oct 15, 2026

Category: german
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / hard_promptshuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,304.84 rating

Confidence interval: 1,300.52 – 1,309.16

27142 votes

Oct 15, 2026

Category: hard_prompts
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / hard_prompts_englishhuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,311.19 rating

Confidence interval: 1,305.79 – 1,316.59

15032 votes

Oct 15, 2026

Category: hard_prompts_english
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / industry_business_and_management_and_financial_operationshuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,254.96 rating

Confidence interval: 1,248.64 – 1,261.28

11265 votes

Oct 15, 2026

Category: industry_business_and_management_and_financial_operations
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / industry_entertainment_and_sports_and_mediahuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,278.81 rating

Confidence interval: 1,273.16 – 1,284.46

14258 votes

Oct 15, 2026

Category: industry_entertainment_and_sports_and_media
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / industry_legal_and_governmenthuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,276.99 rating

Confidence interval: 1,267.99 – 1,285.98

5495 votes

Oct 15, 2026

Category: industry_legal_and_government
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / industry_life_and_physical_and_social_sciencehuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,269.76 rating

Confidence interval: 1,264.04 – 1,275.47

15554 votes

Oct 15, 2026

Category: industry_life_and_physical_and_social_science
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / industry_mathematicalhuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,306.65 rating

Confidence interval: 1,299.69 – 1,313.61

8596 votes

Oct 15, 2026

Category: industry_mathematical
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / industry_medicine_and_healthcarehuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,256.70 rating

Confidence interval: 1,246.84 – 1,266.55

4533 votes

Oct 15, 2026

Category: industry_medicine_and_healthcare
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / industry_software_and_it_serviceshuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,315.88 rating

Confidence interval: 1,311.30 – 1,320.46

24996 votes

Oct 15, 2026

Category: industry_software_and_it_services
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / industry_writing_and_literature_and_languagehuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,313.68 rating

Confidence interval: 1,308.74 – 1,318.61

21816 votes

Oct 15, 2026

Category: industry_writing_and_literature_and_language
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / instruction_followinghuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,297.00 rating

Confidence interval: 1,292.58 – 1,301.41

31239 votes

Oct 15, 2026

Category: instruction_following
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / japanesehuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,233.88 rating

Confidence interval: 1,219.78 – 1,247.98

2041 votes

Oct 15, 2026

Category: japanese
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / koreanhuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,197.57 rating

Confidence interval: 1,178.95 – 1,216.19

1273 votes

Oct 15, 2026

Category: korean
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / longer_queryhuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,310.97 rating

Confidence interval: 1,305.30 – 1,316.63

13768 votes

Oct 15, 2026

Category: longer_query
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / mathhuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,306.51 rating

Confidence interval: 1,299.89 – 1,313.13

10012 votes

Oct 15, 2026

Category: math
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / multi_turnhuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,325.80 rating

Confidence interval: 1,320.14 – 1,331.47

15897 votes

Oct 15, 2026

Category: multi_turn
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / non_englishhuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,283.07 rating

Confidence interval: 1,279.22 – 1,286.93

42689 votes

Oct 15, 2026

Category: non_english
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / overallhuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,297.78 rating

Confidence interval: 1,294.59 – 1,300.96

87769 votes

Oct 15, 2026

Category: overall
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / polishhuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,287.89 rating

Confidence interval: 1,274.34 – 1,301.44

2219 votes

Oct 15, 2026

Category: polish
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / russianhuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,305.45 rating

Confidence interval: 1,298.72 – 1,312.17

10313 votes

Oct 15, 2026

Category: russian
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Arena text preference / spanishhuman preference · 2026-10-15 · rating

Claude Sonnet 3.5 v2

1,285.48 rating

Confidence interval: 1,266.07 – 1,304.89

967 votes

Oct 15, 2026

Category: spanish
Evaluated configuration: claude-3-5-sonnet-20241022
Evaluation type: human preference

Run it locally

Inference · memory estimate

Claude Sonnet 3.5 v2

Closed weights. No supported local deployment or GPU requirement is published.

API cost

USD / month

Input tokens / monthOutput tokens / month

Standard text-token estimate. Excludes taxes, caching, batch discounts, reasoning and multimodal surcharges. Provider limits and regional availability may differ.

Claude Sonnet 3.5 v2

Provider offer

Capabilities

Context window: 200,000 tokens
Maximum output: 8,192 tokens
Reasoning: No
Tool calling: Yes
Open weights: No
Knowledge cutoff: 2024-04-30
Model license: Proprietary

Input modalities

text
image
pdf

Output modalities

text

* Provisional data