Private v2 mathematics problems with Python tools. Earlier versions and Tier 4 are excluded.
050100%
GPT-4 (Jun 2023)Not available
Factual answers
SimpleQA Verified · 25%
Correct factual answers under a guessing prompt, graded by a model. Not a hallucination or abstention score.
050100%
GPT-4 (Jun 2023)Not available
Competition mathematics
OTIS Mock AIME 2024-2025 · Separate result
Competition problems with model-extracted answers checked against an exact key. Earlier grading procedures are excluded.
050100%
GPT-4 (Jun 2023)1.1*%Approx. 95% interval: 0.1–2.2% · Older result
Bars show reported task scores. Whiskers are approximate 95% intervals where standard errors are available. Different reasoning budgets are not equal-cost tests. Missing results are never zero.
Run it locally
Inference · memory estimate
GPT-4 (Jun 2023)
Closed weights. No supported local deployment or GPU requirement is published.