GeekRadar / The comparison desk

COMPARE.

AI models

openai

o1-preview

AI model Released Sep 12, 2024

Capability & preference

General capability · ECI

A cross-benchmark capability estimate. Index points, not a percentage; intervals show uncertainty.

o1-preview 133.9^* o1-preview · 95% interval 131.3–136.3

Human preference · Arena

Overall text preferences from head-to-head votes. Not factual accuracy or a task-success rate.

o1-preview 1352.7 o1-preview · 95% interval 1347.7–1357.8 · 31,122 votes · 2026-10-28

Model performance

Radar task score: four tests, equal weight. An experimental summary of these tasks, not a universal intelligence rating.

o1-preview

Tested configuration Radar task score

Unrated

1/4 core tests · not enough evidence

1 benchmark results for this configuration

Scientific reasoning

GPQA Diamond · 25%

Expert-written science questions; strict multiple-choice scoring. Chance accuracy is 25%.

o1-preview 50.3^*% Approx. 95% interval: 44.8–55.9% · 2025-01-27 · Older result

Novel problem solving

Mystery Game Puzzles · 25%

Best-next-move puzzles in an undisclosed game variant. A narrow reasoning test, not general intelligence.

o1-preview Not available

Advanced mathematics

FrontierMath Tiers 1-3 · 25%

Private v2 mathematics problems with Python tools. Earlier versions and Tier 4 are excluded.

o1-preview Not available

Factual answers

SimpleQA Verified · 25%

Correct factual answers under a guessing prompt, graded by a model. Not a hallucination or abstention score.

o1-preview Not available

Bars show reported task scores. Whiskers are approximate 95% intervals where standard errors are available. Different reasoning budgets are not equal-cost tests. Missing results are never zero.

Recorded results

Only the same benchmark, version, workload, scenario and unit share a row. Missing results are not zero. System results include the full submitted hardware and software configuration.

Arena text preference / chinesehuman preference · 2026-10-28 · rating

o1-preview

1,325.46 rating

Confidence interval: 1,312.94 – 1,337.99

2763 votes

Oct 28, 2026

Category: chinese
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / codinghuman preference · 2026-10-28 · rating

o1-preview

1,366.35 rating

Confidence interval: 1,357.04 – 1,375.66

5123 votes

Oct 28, 2026

Category: coding
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / creative_writinghuman preference · 2026-10-28 · rating

o1-preview

1,317.87 rating

Confidence interval: 1,307.91 – 1,327.84

4508 votes

Oct 28, 2026

Category: creative_writing
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / englishhuman preference · 2026-10-28 · rating

o1-preview

1,382.63 rating

Confidence interval: 1,376.33 – 1,388.93

15891 votes

Oct 28, 2026

Category: english
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / exclude_tieshuman preference · 2026-10-28 · rating

o1-preview

1,313.86 rating

Confidence interval: 1,306.51 – 1,321.21

21521 votes

Oct 28, 2026

Category: exclude_ties
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / experthuman preference · 2026-10-28 · rating

o1-preview

1,337.45 rating

Confidence interval: 1,322.96 – 1,351.95

1982 votes

Oct 28, 2026

Category: expert
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / frenchhuman preference · 2026-10-28 · rating

o1-preview

1,344.76 rating

Confidence interval: 1,309.88 – 1,379.63

290 votes

Oct 28, 2026

Category: french
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / germanhuman preference · 2026-10-28 · rating

o1-preview

1,309.38 rating

Confidence interval: 1,287.61 – 1,331.15

774 votes

Oct 28, 2026

Category: german
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / hard_promptshuman preference · 2026-10-28 · rating

o1-preview

1,354.01 rating

Confidence interval: 1,346.36 – 1,361.65

8496 votes

Oct 28, 2026

Category: hard_prompts
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / hard_prompts_englishhuman preference · 2026-10-28 · rating

o1-preview

1,375.55 rating

Confidence interval: 1,366.12 – 1,384.98

4917 votes

Oct 28, 2026

Category: hard_prompts_english
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / industry_business_and_management_and_financial_operationshuman preference · 2026-10-28 · rating

o1-preview

1,304.22 rating

Confidence interval: 1,293.69 – 1,314.76

3778 votes

Oct 28, 2026

Category: industry_business_and_management_and_financial_operations
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / industry_entertainment_and_sports_and_mediahuman preference · 2026-10-28 · rating

o1-preview

1,330.87 rating

Confidence interval: 1,321.23 – 1,340.50

4823 votes

Oct 28, 2026

Category: industry_entertainment_and_sports_and_media
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / industry_legal_and_governmenthuman preference · 2026-10-28 · rating

o1-preview

1,346.42 rating

Confidence interval: 1,331.98 – 1,360.86

1945 votes

Oct 28, 2026

Category: industry_legal_and_government
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / industry_life_and_physical_and_social_sciencehuman preference · 2026-10-28 · rating

o1-preview

1,348.30 rating

Confidence interval: 1,339.17 – 1,357.43

5419 votes

Oct 28, 2026

Category: industry_life_and_physical_and_social_science
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / industry_mathematicalhuman preference · 2026-10-28 · rating

o1-preview

1,380.07 rating

Confidence interval: 1,369.60 – 1,390.53

3876 votes

Oct 28, 2026

Category: industry_mathematical
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / industry_medicine_and_healthcarehuman preference · 2026-10-28 · rating

o1-preview

1,306.94 rating

Confidence interval: 1,291.43 – 1,322.45

1699 votes

Oct 28, 2026

Category: industry_medicine_and_healthcare
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / industry_software_and_it_serviceshuman preference · 2026-10-28 · rating

o1-preview

1,361.58 rating

Confidence interval: 1,353.72 – 1,369.45

8206 votes

Oct 28, 2026

Category: industry_software_and_it_services
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / industry_writing_and_literature_and_languagehuman preference · 2026-10-28 · rating

o1-preview

1,339.43 rating

Confidence interval: 1,331.42 – 1,347.45

8360 votes

Oct 28, 2026

Category: industry_writing_and_literature_and_language
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / instruction_followinghuman preference · 2026-10-28 · rating

o1-preview

1,341.51 rating

Confidence interval: 1,334.65 – 1,348.38

12782 votes

Oct 28, 2026

Category: instruction_following
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / japanesehuman preference · 2026-10-28 · rating

o1-preview

1,294.56 rating

Confidence interval: 1,270.32 – 1,318.81

674 votes

Oct 28, 2026

Category: japanese
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / koreanhuman preference · 2026-10-28 · rating

o1-preview

1,290.46 rating

Confidence interval: 1,260.16 – 1,320.75

493 votes

Oct 28, 2026

Category: korean
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / longer_queryhuman preference · 2026-10-28 · rating

o1-preview

1,343.66 rating

Confidence interval: 1,333.99 – 1,353.33

4578 votes

Oct 28, 2026

Category: longer_query
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / mathhuman preference · 2026-10-28 · rating

o1-preview

1,372.99 rating

Confidence interval: 1,363.28 – 1,382.71

4569 votes

Oct 28, 2026

Category: math
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / multi_turnhuman preference · 2026-10-28 · rating

o1-preview

1,368.52 rating

Confidence interval: 1,359.18 – 1,377.87

5871 votes

Oct 28, 2026

Category: multi_turn
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / non_englishhuman preference · 2026-10-28 · rating

o1-preview

1,314.93 rating

Confidence interval: 1,308.51 – 1,321.34

15231 votes

Oct 28, 2026

Category: non_english
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / overallhuman preference · 2026-10-28 · rating

o1-preview

1,352.75 rating

Confidence interval: 1,347.72 – 1,357.78

31122 votes

Oct 28, 2026

Category: overall
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / russianhuman preference · 2026-10-28 · rating

o1-preview

1,315.96 rating

Confidence interval: 1,306.22 – 1,325.69

4488 votes

Oct 28, 2026

Category: russian
Evaluated configuration: o1-preview
Evaluation type: human preference

Arena text preference / spanishhuman preference · 2026-10-28 · rating

o1-preview

1,301.90 rating

Confidence interval: 1,270.56 – 1,333.25

347 votes

Oct 28, 2026

Category: spanish
Evaluated configuration: o1-preview
Evaluation type: human preference

Run it locally

Inference · memory estimate

o1-preview

Not enough verified architecture information for a reliable VRAM estimate.

Capabilities

Model license: Proprietary

* Provisional data