The AI Benchmark Index.
Every model release since ChatGPT launched in November 2022, and the GDPval‑AA, SWE‑bench, AA Index and ARC‑AGI‑3 scores.
GDPval
The benchmark that scores models on real deliverables from 44 occupations, graded by working professionals. The one to quote if you are buying AI for knowledge work rather than code.
SWE-bench
Real GitHub issues, real repositories, a passing test suite or nothing. The coding benchmark, and the reason its Verified split had to be retired.
Artificial Analysis Intelligence Index
The composite that most launch posts cite without naming. What goes into it, and what happened when it was rebuilt.
ARC-AGI-3
Interactive games no model has seen, played with no instructions and no rules given. ARC Prize runs every model on one standard harness.
A new frontier model every 9 days
A ten-week procurement cycle now spans 7 model releases, which is why the board below changes leader almost every month. September was the peak, with 6 dated releases on this page. October had 3.
Every update, month by month
Every monthly update since November 2025, and the releases and scores before them back to ChatGPT's launch.
Bars count the frontier releases listed for each year. Releases in bold took the lead.
- GDPval-AAGPT-6 Astra Is 3rd on the Index and 43rd on Real Work.
Removed from the Intelligence Index on 27 October and still running. Claude Opus 5.5 leads at 1866, Kimi K3.1 Pro is second, and OpenAI’s best entry is 37th.
ARC-AGI-3Claude Fable 5.2 81.2%, a new high (97.7% on the Provider Adapter; xhigh effort)Gemini 4 Carbon 65.1% (99.1% on the Provider Adapter)Released
Claude Fable 5.2ARC-AGI-3 first at 81.2%
Kimi K3.1 Pro
- Reset: Index v5IndexFable 5.1 Falls From First to Eighth.
The index went to v5 on 27 October and v5.0.1 eight days later. Fable lost four points and Astra gained four, neither having shipped a thing. FrontierCode 2.0 moved inside the composite.
ARC-AGI-3Gemini 4 Argon 49.8% (97.5% on the Provider Adapter; 42 of 55 games scored)Released
Claude Haiku 5.5
GPT-6.1 Terra
Gemini 4 CarbonIndex first at 60 by 4 November
- IndexFable 5.1 and GPT-6 Astra Tie at 53.
Identical scores at identical list prices, and a 2.3x gap in cost per task. GLM 5.3 Flash ran the whole index for 25 cents.
ARC-AGI-3GPT-6 Astra 62.7%, a new high (99.9% on the Provider Adapter)Claude Sonnet 5.5 24.3% (72.4% on the Provider Adapter)GPT-6.1 Sol 52.7% (96.4% on the Provider Adapter)Released
Claude Fable 5.1Index first at 53
GPT-6 Astra
GLM-5.3
Grok 4.7
Claude Opus 5.5Now GDPval-AA first at 1866
Claude Sonnet 5.5
GPT-6.1 Sol
- Reset: GDPval-AA v2IndexGrok 4.6 Joins the Frontier.
Opus 5 at 63, Fable 5 at 62, Grok 4.6 and Sol at 61 on the AA index, with prices spanning 8x across four models separated by two points.
GDPval-AAOpus 5 Leads at 1849 Elo.GDPval-AA moved to v2 and reset every June number. Grok 4.6 sits at 1746, and xAI's number-one claim does not hold.
Released - IndexOpus 5 Matches Fable at Half.
The busiest benchmark month of the year, and the arrival of the priced leaderboard. Opus 5 hit 66.7 against Fable's 66.5 on CursorBench at half the cost.
ARC-AGI-3GPT-5.6 Sol 7.8%, a new highClaude Opus 5 30.2%, a new highReleased
Fable 5 backAfter twenty days off
GPT-5.6 Sol, Terra, Luna
Kimi K3
Claude Opus 5Took the GDPval-AA lead
- GDPval-AAThe Benchmark That Actually Matters.
Claude Fable 5 leads at 1932 Elo. What expert parity on real deliverables means, and the catch in the fine print.
SWE-benchFable 5 Tops a Dying Leaderboard.80.3% on SWE-bench Pro, OpenAI kills Verified for good, and FrontierCode resets the leaderboard underneath.
ARC-AGI-3Claude Opus 4.8 1.5%, a new highReleased
Claude Fable 5First on both: 1932 and 80.3%
Fable 5 switched offUS export-control directive
- GDPval-AAThe Leaderboard Reshuffles.
Opus 4.8 takes the lead at 1890 Elo, Grok 4.3 jumps 321 points, and Gemini 3.5 Flash beats Google's own Pro tier on real work.
SWE-benchOpus 4.8 Takes the Lead.69.2% on Pro, and open-weights models close to within 6 points at 8x lower cost.
Released
Gemini 3.5 Flash
Claude Opus 4.8First on both: 1890 and 69.2%
- Reset: 8 benchmarks brokenGDPval-AAGPT-5.5 Sets the Bar.
Launches at 84.9% expert parity, and economists start writing about AI eating analyst work.
SWE-benchThe Month the Benchmark Broke.Berkeley researchers break eight agent benchmarks, and Mythos Preview exposes the Verified-versus-Pro gap.
ARC-AGI-3Claude Opus 4.6 (Max) 0.5%, a new highReleased
Claude Mythos Preview
Grok 4.3 (beta)
Kimi K2.6
GPT-5.5GDPval-AA first at 1769
Claude Opus 4.7SWE-bench Pro first at 64.3%
- GDPval-AAGPT-5.4 Crowds the Top.
1674 Elo, 41 points above Sonnet 4.6. Three labs within 70 points, and what that compression means for buyers.
SWE-benchGPT-5.4 Takes Pro.59.1% on SEAL's standardized Pro split, with Opus 4.6 holding the commercial subset.
Released
GPT-5.4GDPval-AA 1674, SWE-bench Pro 59.1%
- Reset: Verified deprecatedGDPval-AAAnthropic Takes Both Top Slots.
Sonnet 4.6 at 1633 Elo and Opus 4.6 at 1606, with Gemini 3.1 Pro last among the majors.
SWE-benchThe Month Verified Died.OpenAI deprecates Verified over contamination, and Pro takes over as the number that counts.
Released
Claude Opus 4.6GDPval-AA first at 1606
GPT-5.3-CodexSWE-bench Pro 56.8%
Claude Sonnet 4.6GDPval-AA first at 1633
Gemini 3.1 Pro
- Reset: Index rebuiltIndexThe Index Overhaul.
Artificial Analysis rebuilt the Intelligence Index: ten work-shaped evals in, saturated exams out, top scores down 20 points overnight.
ReleasedNo frontier release
- GDPval-AAThe Leaderboard Arrives.
GPT-5.2 hits 70.9% win or tie against professionals, and Artificial Analysis launches GDPval-AA as an independent Elo leaderboard.
Released
GPT-5.2Led GDPval-AA by late December
- SWE-benchOpus 4.5 Breaks 80.
The first model over 80% on Verified, after four frontier releases in twelve days. Pro sat at 45.9%.
Released
GPT-5.1
Gemini 3 Pro
GPT-5.1-Codex-Max
Claude Opus 4.580.9% Verified, first past 80
- SWE-bench VerifiedClaude Haiku 4.5 73.3%Released
ChatGPT hits 800M weekly usersAnnounced at DevDay 2025
Claude Haiku 4.5
- SWE-bench VerifiedClaude Sonnet 4.5 77.2%, a new highAA IndexGPT-5 (high) 68 (Index v3.0; ahead of Grok 4 at 65)GDPvalClaude Opus 4.1 47.6% won or tied with experts, top model at launchGPT-5 (high) About 39%Released
GPT-5-Codex
Grok 4 Fast
Claude Sonnet 4.577.2% Verified, new high
DeepSeek-V3.2-Exp
- SWE-bench Verifiedgpt-oss-120b 62.4%Claude Opus 4.1 74.5%, a new highGPT-5 74.9%, a new highAA IndexGPT-5 (high) 68, top of the index (Index v2.2, a new scale; not comparable to 73)Released
gpt-oss-120b and 20b
Claude Opus 4.174.5% Verified, new high
GPT-574.9% Verified, leads AA Index
DeepSeek-V3.1
- SWE-bench VerifiedKimi K2 65.8%AA IndexGrok 4 73, top of the index (Index v2; the first time OpenAI did not lead)Released
Grok 4Tops AA Index at 73
Kimi K2
Gemini Deep Think IMO goldCertified gold at IMO 2025
- SWE-bench Verifiedcodex-1 72.1% (On 477 of 500 tasks, reported by press)Claude Sonnet 4 72.7%, a new highClaude Opus 4 72.5%Released
Codex (codex-1)
Claude Opus 4 and Sonnet 4Sonnet 4 72.7% Verified, new high
- SWE-bench VerifiedGPT-4.1 54.6% (52.1% counting 23 skipped tasks as fails)o3 69.1%, a new higho4-mini 68.1%Released
Llama 4 Scout and Maverick
GPT-4.1
o3 and o4-minio3 69.1% Verified, new high
Gemini 2.5 Flash- Qwen3
- SWE-bench VerifiedGemini 2.5 Pro 63.8%, a new highReleased
Gemini 2.5 Pro63.8% Verified, custom agent
- SWE-bench VerifiedClaude 3.7 Sonnet 62.3%, a new high (62.3% on all 500 tasks; 70.3% with extra compute)Released
Grok 3
Claude 3.7 Sonnet62.3% Verified, new high
GPT-4.5
- Released
DeepSeek-R1
DeepSeek shockNvidia loses about $593B in a day
o3-mini
- SWE-bench Verifiedo3 (preview) 71.7% (Shown before release; o3 shipped at 69.1%)Released
- SWE-bench VerifiedClaude 3.5 Sonnet (new) 49.0%, a new highClaude 3.5 Haiku 40.6%Released
Claude 3.5 Sonnet (new)49.0% Verified, new high
Claude 3.5 Haiku
- Released
- SWE-bench VerifiedGPT-4o 33.2%, a new high (Verified launch, best setup tested)Released
Grok-2
SWE-bench Verified launchedGPT-4o scores 33.2%
- Released
- Released
- Released
- Released
- Released
- Released
- Released
- SWE-benchClaude 2 1.96%, a new high (Original SWE-bench paper)
- Released
- Released
ChatGPT hits 100M usersUBS estimate, about two months in
Bard
LLaMA
- Released
What twelve months of scores actually show
The benchmark broke five times. The models did not stop improving. The Intelligence Index was rebuilt in January and top scores fell 20 points overnight. Verified was deprecated in February over contamination. Berkeley broke eight agent benchmarks in April. GDPval-AA moved to v2 in August and reset every June number. The Intelligence Index went to v5 in October and dropped GDPval-AA. Through all of it, SWE-bench Pro went from 45.9% in November 2025 to 80.3% in June 2026. The instrument kept failing and the thing being measured kept moving.
No model held the lead for two consecutive updates. GPT-5.2, then Sonnet 4.6, then GPT-5.4, then GPT-5.5, then Opus 4.8, then Fable 5, then Opus 5, then Opus 5.5. Eight leaders in eleven months. If you picked a model because it was top of a leaderboard, you have been wrong roughly every six weeks, and the cost of being wrong has been near zero. That is the argument for routing rather than standardising.
The top of the table compressed until price became the tiebreak. In March, three labs sat within 70 Elo points. By August, four models sat within two points on the AA index with prices spanning 8x. A two-point capability gap and an 8x price gap is not a capability decision. It is a procurement decision, and that is why the July update started publishing cost next to score.
Open weights stopped being a separate conversation. By May 2026 open-weights models were within 6 points of the frontier on SWE-bench Pro at 8x lower cost. If you have not priced that option since then, you are working from a stale number. Start with What Are Open Weights?
The caveat that belongs on every number here
Every figure in this index is a published score, and published scores are marketing before they are measurement. Labs choose which benchmark to lead with, contamination is real and was the stated reason Verified died, and a reset means last month's number is not comparable to this month's even when both are on the page.
Read The Benchmarks Are Getting Gamed before you put any of these numbers in a deck. The short version: use benchmarks to eliminate models, not to pick one. The picking is done on your own work, with your own evals, which is the only test that cannot be trained against.
Updated monthly.
