---
title: "The Benchmark Index: 12 Months of Scores"
url: https://nativefirst.ai/blog/the-benchmark-index/
description: "Every model release since ChatGPT launched in November 2022, and the GDPval-AA, SWE-bench, AA Index and ARC-AGI-3 scores."
author: John Tan
category: "AI Benchmarks"
published: 2026-09-21
updated: 2026-09-21
publisher: nativefirst.ai
format: Markdown copy of the web page at the url above, same text.
---

# The AI Benchmark Index.

Every model release since ChatGPT launched in November 2022, and the GDPval‑AA, SWE‑bench, AA Index and ARC‑AGI‑3 scores.

4 benchmarks

### GDPval

The benchmark that scores models on real deliverables from 44 occupations, graded by working professionals. The one to quote if you are buying AI for knowledge work rather than code.

- [Explainer What Is GDPval?](https://nativefirst.ai/blog/what-is-gdpval/)
- [Latest November 2026 update](https://nativefirst.ai/blog/gdpval-aa-november-2026/)

### SWE-bench

Real GitHub issues, real repositories, a passing test suite or nothing. The coding benchmark, and the reason its Verified split had to be retired.

- [Explainer What Is SWE-bench?](https://nativefirst.ai/blog/what-is-swe-bench/)
- [Latest June 2026 update (final)](https://nativefirst.ai/blog/swe-bench-update-june-2026/)

### Artificial Analysis Intelligence Index

The composite that most launch posts cite without naming. What goes into it, and what happened when it was rebuilt.

- [Explainer What Is the AA Intelligence Index?](https://nativefirst.ai/blog/what-is-the-aa-intelligence-index/)
- [Latest October 2026 update](https://nativefirst.ai/blog/benchmark-update-october-2026/)

### ARC-AGI-3

Interactive games no model has seen, played with no instructions and no rules given. ARC Prize runs every model on one standard harness.

- [Explainer What Is ARC-AGI?](https://nativefirst.ai/blog/what-is-arc-agi/)
- [Latest November 2026 leaderboard](https://arcprize.org/leaderboard)

## A new frontier model every 9 days

It used to be **~10 weeks** between new Anthropic and OpenAI models

Now **~9 days** between new Anthropic and OpenAI models

Since September 1, 2026, on this page **8** Anthropic and OpenAI releases in 74 days

Six months to October 2026 **127** launches across Anthropic, Google and OpenAI

A ten-week procurement cycle now spans 7 model releases, which is why the board below changes leader almost every month. September was the peak, with 6 dated releases on this page. October had 3.

## Every update, month by month

Every monthly update since November 2025, and the releases and scores before them back to ChatGPT's launch.

Bars count the frontier releases listed for each year. Releases in **bold** took the lead.

1. November 2026
   GDPval-AA
   [GPT-6 Astra Is 3rd on the Index and 43rd on Real Work.](https://nativefirst.ai/blog/gdpval-aa-november-2026/)
   Removed from the Intelligence Index on 27 October and still running. Claude Opus 5.5 leads at 1866, Kimi K3.1 Pro is second, and OpenAI’s best entry is 37th.
   ARC-AGI-3
   [Claude Fable 5.2](https://arcprize.org/results/anthropic-claude-fable-5-2) **81.2%**, a new high *(97.7% on the Provider Adapter; xhigh effort)*
   [Gemini 4 Carbon](https://arcprize.org/results/google-gemini-4-carbon) **65.1%** *(99.1% on the Provider Adapter)*
   Released
   - [Claude Fable 5.2](https://nativefirst.ai/blog/fable-52-day-zero-reactions/) Nov 9 *ARC-AGI-3 first at 81.2%*
   - [Kimi K3.1 Pro](https://nativefirst.ai/blog/gdpval-aa-november-2026/)
2. October 2026 Reset: Index v5
   Index
   [Fable 5.1 Falls From First to Eighth.](https://nativefirst.ai/blog/benchmark-update-october-2026/)
   The index went to v5 on 27 October and v5.0.1 eight days later. Fable lost four points and Astra gained four, neither having shipped a thing. FrontierCode 2.0 moved inside the composite.
   ARC-AGI-3
   [Gemini 4 Argon](https://arcprize.org/results/google-gemini-4-argon) **49.8%** *(97.5% on the Provider Adapter; 42 of 55 games scored)*
   Released
   - [Claude Haiku 5.5](https://nativefirst.ai/blog/claude-opus-sonnet-haiku-all-5-5/) Oct 7
   - [GPT-6.1 Terra](https://nativefirst.ai/blog/benchmark-update-october-2026/) Oct 26
   - [Gemini 4 Carbon](https://nativefirst.ai/blog/gdpval-aa-november-2026/) Oct 28 *Index first at 60 by 4 November*
3. September 2026
   Index
   [Fable 5.1 and GPT-6 Astra Tie at 53.](https://nativefirst.ai/blog/benchmark-update-september-2026/)
   Identical scores at identical list prices, and a 2.3x gap in cost per task. GLM 5.3 Flash ran the whole index for 25 cents.
   ARC-AGI-3
   [GPT-6 Astra](https://arcprize.org/results/openai-gpt-6-astra) **62.7%**, a new high *(99.9% on the Provider Adapter)*
   [Claude Sonnet 5.5](https://arcprize.org/leaderboard) **24.3%** *(72.4% on the Provider Adapter)*
   [GPT-6.1 Sol](https://arcprize.org/leaderboard) **52.7%** *(96.4% on the Provider Adapter)*
   Released
   - [Claude Fable 5.1](https://nativefirst.ai/blog/benchmark-update-september-2026/) Sep 1 *Index first at 53*
   - [GPT-6 Astra](https://nativefirst.ai/blog/benchmark-update-september-2026/) Sep 3
   - [GLM-5.3](https://nativefirst.ai/blog/benchmark-update-september-2026/)
   - [Grok 4.7](https://nativefirst.ai/blog/xai-made-it-a-three-horse-race/) Sep 21
   - [Claude Opus 5.5](https://nativefirst.ai/blog/anthropic-cut-opus-40-percent/) Sep 22 *Now GDPval-AA first at 1866*
   - [Claude Sonnet 5.5](https://nativefirst.ai/blog/claude-opus-sonnet-haiku-all-5-5/) Sep 28
   - [GPT-6.1 Sol](https://nativefirst.ai/blog/working-with-gpt-6-astra/) Sep 29
4. August 2026 Reset: GDPval-AA v2
   Index
   [Grok 4.6 Joins the Frontier.](https://nativefirst.ai/blog/benchmark-update-august-2026/)
   Opus 5 at 63, Fable 5 at 62, Grok 4.6 and Sol at 61 on the AA index, with prices spanning 8x across four models separated by two points.
   GDPval-AA
   [Opus 5 Leads at 1849 Elo.](https://nativefirst.ai/blog/gdpval-update-august-2026/)
   GDPval-AA moved to v2 and reset every June number. Grok 4.6 sits at 1746, and xAI's number-one claim does not hold.
   Released
   - [Grok 4.6](https://nativefirst.ai/blog/benchmark-update-august-2026/) Aug 12
5. July 2026
   Index
   [Opus 5 Matches Fable at Half.](https://nativefirst.ai/blog/benchmark-update-july-2026/)
   The busiest benchmark month of the year, and the arrival of the priced leaderboard. Opus 5 hit 66.7 against Fable's 66.5 on CursorBench at half the cost.
   ARC-AGI-3
   [GPT-5.6 Sol](https://arcprize.org/results/openai-gpt-5-6-sol) **7.8%**, a new high
   [Claude Opus 5](https://arcprize.org/results/anthropic-claude-opus-5) **30.2%**, a new high
   Released
   - [Fable 5 back](https://nativefirst.ai/blog/fable-5-is-back/) Jul 2 *After twenty days off*
   - [GPT-5.6 Sol, Terra, Luna](https://nativefirst.ai/blog/benchmark-update-july-2026/) Jul 9
   - [Kimi K3](https://nativefirst.ai/blog/kimi-k3-open-source-gap/) Jul 16
   - [Claude Opus 5](https://nativefirst.ai/blog/benchmark-update-july-2026/) Jul 24 *Took the GDPval-AA lead*
6. June 2026
   GDPval-AA
   [The Benchmark That Actually Matters.](https://nativefirst.ai/blog/gdpval-update-june-2026/)
   Claude Fable 5 leads at 1932 Elo. What expert parity on real deliverables means, and the catch in the fine print.
   SWE-bench
   [Fable 5 Tops a Dying Leaderboard.](https://nativefirst.ai/blog/swe-bench-update-june-2026/)
   80.3% on SWE-bench Pro, OpenAI kills Verified for good, and FrontierCode resets the leaderboard underneath.
   ARC-AGI-3
   [Claude Opus 4.8](https://arcprize.org/leaderboard) **1.5%**, a new high
   Released
   - [Claude Fable 5](https://nativefirst.ai/blog/claude-fable-5-reactions-day-1/) Jun 9 *First on both: 1932 and 80.3%*
   - [Fable 5 switched off](https://nativefirst.ai/blog/fable-5-is-back/) Jun 12 *US export-control directive*
7. May 2026
   GDPval-AA
   [The Leaderboard Reshuffles.](https://nativefirst.ai/blog/gdpval-update-may-2026/)
   Opus 4.8 takes the lead at 1890 Elo, Grok 4.3 jumps 321 points, and Gemini 3.5 Flash beats Google's own Pro tier on real work.
   SWE-bench
   [Opus 4.8 Takes the Lead.](https://nativefirst.ai/blog/swe-bench-update-may-2026/)
   69.2% on Pro, and open-weights models close to within 6 points at 8x lower cost.
   Released
   - [Gemini 3.5 Flash](https://nativefirst.ai/blog/gdpval-update-may-2026/) May 19
   - [Claude Opus 4.8](https://nativefirst.ai/blog/gdpval-update-may-2026/) May 28 *First on both: 1890 and 69.2%*
8. April 2026 Reset: 8 benchmarks broken
   GDPval-AA
   [GPT-5.5 Sets the Bar.](https://nativefirst.ai/blog/gdpval-update-april-2026/)
   Launches at 84.9% expert parity, and economists start writing about AI eating analyst work.
   SWE-bench
   [The Month the Benchmark Broke.](https://nativefirst.ai/blog/swe-bench-update-april-2026/)
   Berkeley researchers break eight agent benchmarks, and Mythos Preview exposes the Verified-versus-Pro gap.
   ARC-AGI-3
   [Claude Opus 4.6 (Max)](https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf) **0.5%**, a new high
   Released
   - [Claude Mythos Preview](https://nativefirst.ai/blog/swe-bench-update-april-2026/) Apr 6
   - [Grok 4.3 (beta)](https://nativefirst.ai/blog/gdpval-update-april-2026/) Apr 17
   - [Kimi K2.6](https://nativefirst.ai/blog/swe-bench-update-april-2026/) Apr 20
   - [GPT-5.5](https://nativefirst.ai/blog/gdpval-update-april-2026/) Apr 23 *GDPval-AA first at 1769*
   - [Claude Opus 4.7](https://nativefirst.ai/blog/swe-bench-update-april-2026/) *SWE-bench Pro first at 64.3%*
9. March 2026
   GDPval-AA
   [GPT-5.4 Crowds the Top.](https://nativefirst.ai/blog/gdpval-update-march-2026/)
   1674 Elo, 41 points above Sonnet 4.6. Three labs within 70 points, and what that compression means for buyers.
   SWE-bench
   [GPT-5.4 Takes Pro.](https://nativefirst.ai/blog/swe-bench-update-march-2026/)
   59.1% on SEAL's standardized Pro split, with Opus 4.6 holding the commercial subset.
   Released
   - [GPT-5.4](https://nativefirst.ai/blog/gdpval-update-march-2026/) Mar 5 *GDPval-AA 1674, SWE-bench Pro 59.1%*
10. February 2026 Reset: Verified deprecated
    GDPval-AA
    [Anthropic Takes Both Top Slots.](https://nativefirst.ai/blog/gdpval-update-february-2026/)
    Sonnet 4.6 at 1633 Elo and Opus 4.6 at 1606, with Gemini 3.1 Pro last among the majors.
    SWE-bench
    [The Month Verified Died.](https://nativefirst.ai/blog/swe-bench-update-february-2026/)
    OpenAI deprecates Verified over contamination, and Pro takes over as the number that counts.
    Released
    - [Claude Opus 4.6](https://nativefirst.ai/blog/gdpval-update-february-2026/) Feb 5 *GDPval-AA first at 1606*
    - [GPT-5.3-Codex](https://nativefirst.ai/blog/swe-bench-update-february-2026/) Feb 5 *SWE-bench Pro 56.8%*
    - [Claude Sonnet 4.6](https://nativefirst.ai/blog/gdpval-update-february-2026/) Feb 17 *GDPval-AA first at 1633*
    - [Gemini 3.1 Pro](https://nativefirst.ai/blog/swe-bench-update-february-2026/) Feb 19
11. January 2026 Reset: Index rebuilt
    Index
    [The Index Overhaul.](https://nativefirst.ai/blog/benchmarks-update-january-2026/)
    Artificial Analysis rebuilt the Intelligence Index: ten work-shaped evals in, saturated exams out, top scores down 20 points overnight.
    Released
    No frontier release

## What twelve months of scores actually show

45.9% to 80.3%

Five resets, and the line kept climbing

*Chart: SWE-bench Pro rose from 45.9 percent in November 2025 to 80.3 percent in June 2026, while the benchmarks were reset in January, February, April, August and October.*

**The benchmark broke five times. The models did not stop improving.** The Intelligence Index was rebuilt in January and top scores fell 20 points overnight. Verified was deprecated in February over contamination. Berkeley broke eight agent benchmarks in April. GDPval-AA moved to v2 in August and reset every June number. The Intelligence Index went to v5 in October and dropped GDPval-AA. Through all of it, SWE-bench Pro went from 45.9% in November 2025 to 80.3% in June 2026. The instrument kept failing and the thing being measured kept moving.

8 leaders

in eleven months on GDPval

*Chart: The GDPval leader by update: GPT-5.2 in December, Sonnet 4.6 in February, GPT-5.4 in March, GPT-5.5 in April, Opus 4.8 in May, Fable 5 in June, Opus 5 in July, Opus 5.5 in November.*

**No model held the lead for two consecutive updates.** GPT-5.2, then Sonnet 4.6, then GPT-5.4, then GPT-5.5, then Opus 4.8, then Fable 5, then Opus 5, then Opus 5.5. Eight leaders in eleven months. If you picked a model because it was top of a leaderboard, you have been wrong roughly every six weeks, and the cost of being wrong has been near zero. That is the argument for routing rather than standardising.

2 points, 8x

Score converged, price did not

*Chart: In August 2026 four models sat within two points on the AA index while their prices spanned eight times.*

**The top of the table compressed until price became the tiebreak.** In March, three labs sat within 70 Elo points. By August, four models sat within two points on the AA index with prices spanning 8x. A two-point capability gap and an 8x price gap is not a capability decision. It is a procurement decision, and that is why the July update started publishing cost next to score.

6 points

Open weights, at an eighth of the cost

*Chart: By May 2026 open-weights models were within six points of the frontier on SWE-bench Pro at eight times lower cost.*

**Open weights stopped being a separate conversation.** By May 2026 open-weights models were within 6 points of the frontier on SWE-bench Pro at 8x lower cost. If you have not priced that option since then, you are working from a stale number. Start with [What Are Open Weights?](https://nativefirst.ai/blog/what-are-open-weights/)

## The caveat that belongs on every number here

Every figure in this index is a published score, and published scores are marketing before they are measurement. Labs choose which benchmark to lead with, contamination is real and was the stated reason Verified died, and a reset means last month's number is not comparable to this month's even when both are on the page.

Read [**The Benchmarks Are Getting Gamed**](https://nativefirst.ai/blog/benchmarks-are-getting-gamed/) before you put any of these numbers in a deck. The short version: use benchmarks to eliminate models, not to pick one. The picking is done on your own work, with your own evals, which is the only test that cannot be trained against.

Updated monthly.

[John Tan ↗](https://www.linkedin.com/in/johntans/)

Founder of nativefirst.ai. Runs everything on agents and writes down what works.

Sources

1 Artificial Analysis, GDPval-AA leaderboard and Intelligence Index, November 2025 through August 2026. GDPval-AA moved to v2 in August 2026, resetting prior Elo figures. [artificialanalysis.ai](https://artificialanalysis.ai/evaluations/gdpval-aa)

2 OpenAI, GDPval, covering real deliverables across 44 occupations graded by working professionals. [openai.com](https://openai.com/index/gdpval/)

3 SWE-bench (Princeton) and SWE-bench Pro, including OpenAI's deprecation of the Verified split in February 2026 over contamination and its retirement in June 2026. [openai.com](https://openai.com/index/why-we-no-longer-evaluate-swe-bench-verified/) · [scale.com](https://scale.com/blog/swe-bench-pro)

4 Scale AI SEAL leaderboard, standardized SWE-bench Pro results, March 2026. [labs.scale.com](https://labs.scale.com/leaderboard/swe_bench_pro_public)

5 UC Berkeley agent-benchmark analysis covering eight agent benchmarks, April 2026. [moogician.github.io](https://moogician.github.io/blog/2026/trustworthy-benchmarks-cont/)

6 All figures are published scores from the labs and independent leaderboards named in each linked update, recorded on the date of that update. A benchmark reset means figures either side of it are not directly comparable.

7 ARC Prize, ARC-AGI-3 leaderboard and verified results pages, April to November 2026. [arcprize.org](https://arcprize.org/leaderboard)

8 ARC Prize Foundation, ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence, April 22, 2026. [arcprize.org](https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf)

9 @jfonsecarivera on X, September 29, 2026: the interval between new Anthropic and OpenAI models.

10 a16z, Top 100 Consumer AI Apps, October 5, 2026: 127 launches across Anthropic, Google and OpenAI in six months.
