The PlaybookAI Benchmarks

GDPval-AA (November 2026): GPT-6 Astra is 3rd on the index and 43rd on real work.

GPT-6 Astra is third on the Artificial Analysis Intelligence Index and 43rd on GDPval-AA. Claude Opus 5.5 leads at 1866, Kimi K3.1 Pro is second.

John TanNovember 4, 20264 min read

GPT-6 Astra is the third most capable model in the world, on the index everyone quotes. On the benchmark that scores economically valuable work, it is 43rd.

Not 43rd among open-weights models or 43rd on some niche subset. Forty-third on GDPval-AA, out of 308 entries, behind Claude Opus 5 from July, behind Gemini 4 Carbon, and behind several Chinese open-weights models.

1866

Claude Opus 5.5 at max effort, first on GDPval-AA.

37th

OpenAI's best entry, GPT-6.1 Sol at 1609.

308

Entries on the board, up from 210 in August.

The board

One note before the numbers: GDPval-AA ranks entries, not models. Each model appears once per reasoning-effort setting, which is why Anthropic occupies six of the top twelve slots. Read it as a list of configurations you could actually deploy, not a league table of labs.

Figure 1. GDPval-AA v2.1, top twelve
220 tasks, 44 occupations, graded head to head.
ENTRY ELO 95% CI 1 Claude Opus 5.5 (Max)1866±26 2 Kimi K3.1 Pro1858±28 3 Muse Wonder Full (Max)1857±25 4 Claude Sonnet 5.5 (Max)1838±23 5 Claude Opus 5.5 (Xhigh)1835±23 6 Muse Wonder (Max)1770±23 7 Claude Fable 5.1 (Max)1763±19 8 DeepSeek V4.1 Flash 1017 (Max)1746±21 9 Claude Fable 5.1 (Xhigh)1740±19 10 Claude Sonnet 5.5 (Xhigh)1735±23 11 Gemini 4 Carbon (High)1733±24 The scale is pinned: DeepSeek V4.1 Flash (Max) sits at exactly 1600, rank 39.
GDPval-AA v2.1 leaderboard, read from Artificial Analysis on 4 November 2026. Confidence intervals run around ±20 Elo, so adjacent entries are frequently tied. Figure by nativefirst.

The same models, two benchmarks, forty places apart

Set this board beside the Intelligence Index and the disagreement is not subtle.

Figure 2. Two benchmarks, one week apart
The ranking inverts almost completely.
MODEL INDEX v5.0.1 GDPVAL-AA MOVE Claude Opus 5.52nd1st+1 Claude Sonnet 5.55th4th+1 Claude Fable 5.18th7th+1 Kimi K3.1 Pro9th2nd+7 Gemini 4 Carbon1st11th−10 GPT-6.1 Sol4th37th−33 GPT-6 Astra3rd43rd−40 Anthropic's models rank the same on both. OpenAI's fall about forty places. Gemini 4 Carbon and Kimi K3.1 Pro swap ends of the table entirely.
Artificial Analysis, read 4 November 2026. The Index counts one entry per base model; GDPval-AA ranks every effort setting separately.

Something has to give. Either these two benchmarks measure genuinely different things, or one of them is wrong about several of the best-funded models in the world. The first explanation is the likelier one, and it is also the more useful one, because it tells you the question to ask before quoting either number: capable at what?

GDPval-AA runs 220 tasks drawn from 44 occupations, graded by pairwise comparison between model outputs. It rewards finished deliverables. The Intelligence Index aggregates nine evaluations weighted toward coding, science and knowledge recall. A model tuned for hard reasoning in short bursts can top one and sit mid-table on the other without anything being broken.

The index dropped this one on October 27

Worth stating the sequence plainly. When the Intelligence Index moved to v5 on 27 October it removed four evaluations, and GDPval-AA was one of them. Artificial Analysis now labels it a standalone evaluation and published no per-evaluation reasoning for the removals, so there is nothing here to read as motive and I am not offering one.

What is worth noticing is the effect on you. The headline index number now carries more coding weight and no measure of economically valuable knowledge work at all. If your agents write documents, build spreadsheets and produce deliverables, the benchmark that was closest to your workload left the number you are most likely to quote.

And it did not get abandoned. The board went from 210 entries in August to 308 now, and Gemini 4 Carbon was scored within days of release. Removing it from the composite did not stop the work.

The scale moved under everyone in September

GDPval-AA went from v2 to v2.1 in the second half of September, and the change was to how the Elo scale is fixed: the ratings are now anchored by pinning DeepSeek V4.1 Flash at max effort to exactly 1600.

Artificial Analysis says rank ordering is "largely preserved," and at the top that holds. The first six places did not move at all. Further down it is looser: comparing the last v2 board with the first v2.1 board, 99 of the 227 entries present on both changed position relative to each other, almost all by one to four places and almost all between entries already inside each other's confidence intervals.

Every score fell, by between 10 and 254 Elo, averaging about 122, as the whole scale compressed onto the new anchor. Be careful attributing all of that to the rescale: seven new entries arrived in the same five-day window and fresh judgements may have landed too, so the clean claim is that ranks shifted for roughly 44% of entries across the version change, not that the rescale alone caused it.

The practical consequence is the usual one. A GDPval-AA number from before late September is on a different scale from one after it, and the two should not appear in the same chart without a break.

Two things to carry forward

Name the benchmark whenever you quote a rank. "Third best model in the world" is true of GPT-6 Astra and so is "43rd." Neither is usable without the board attached, and the gap between them is the size of the entire mid-table.

Pick the board that matches your workload, then ignore the other one. If your agents produce deliverables, GDPval-AA is closer to your reality than the index is, and it is now the only place that measure lives. What it actually tests is worth understanding before you act on it.

One footnote for anyone reading the method. Artificial Analysis describes the v2.1 ratings as fitted with a Crowd-BT model in one place and as a Bradley-Terry rating computed by maximum likelihood in another, without reconciling the two. It does not change the ordering, but it is the kind of detail worth knowing is unresolved before you build a procurement case on three Elo points.

Same week, same lab, same model. Third place and forty-third.

Pick the right board.

Portrait of John Tan
John Tan ↗

Founder of nativefirst.ai. Runs everything on agents and writes down what works.

Sources
1GDPval-AA v2.1 leaderboard, read from artificialanalysis.ai/evaluations/gdpval-aa on 4 November 2026, 308 entries. Top twelve by Elo with 95% confidence intervals: Claude Opus 5.5 (Max, Default Fallback) 1866 ±26, Kimi K3.1 Pro 1858 ±28, Muse Wonder Full (Max) 1857 ±25, Claude Sonnet 5.5 (Max, Default Fallback) 1838 ±23, Claude Opus 5.5 (Xhigh) 1835 ±23, Muse Wonder (Max) 1770 ±23, Claude Fable 5.1 (Max) 1763 ±19, DeepSeek V4.1 Flash 1017 (Max) 1746 ±21, Claude Fable 5.1 (Xhigh) 1740 ±19, Claude Sonnet 5.5 (Xhigh) 1735 ±23, Gemini 4 Carbon (High) 1733 ±24, Claude Opus 5 (Max) 1730 ±19. The board ranks one entry per reasoning-effort setting, not per model. artificialanalysis.ai
2OpenAI's entries on the same board: GPT-6.1 Sol best at 1609 (Max), rank 37, with Xhigh 1555, High 1520, Medium 1459 and Low 1322. GPT-6 Astra best at 1584 (Max), rank 43, with Xhigh 1569, High 1536, Medium 1498 and Low 1403.
3Intelligence Index v5.0.1 ranks used for comparison, read from artificialanalysis.ai on 4 November 2026, one entry per base model: Gemini 4 Carbon (High) 60, Claude Opus 5.5 (Max with fallback) 59, GPT-6 Astra (Max) 57, GPT-6.1 Sol (Max) 55, Claude Sonnet 5.5 (Max) 55, GPT-6.1 Terra (Max) 51, Muse Wonder (Full) 50, Claude Fable 5.1 (Max with fallback) 49, Kimi K3.1 Pro 46. artificialanalysis.ai
4GDPval-AA v2.1 definition from Artificial Analysis's intelligence benchmarking methodology page, read 4 November 2026: 220 tasks across 44 occupations, agentic task completion with file outputs, scored by pairwise comparison (Elo) from a judge panel, tool usage permitted, test set not private. The scale is anchored by pinning DeepSeek V4.1 Flash (Max) at 1600; that entry sits at exactly 1600 at rank 39 on the board read here. artificialanalysis.ai
5Status, in Artificial Analysis's own words on the same page: "Status: Standalone evaluation (not part of Artificial Analysis Intelligence Index v5.0.1). GDPval-AA was removed from the Intelligence Index in v5.0, having been a constituent up to and including v4.3.2." Artificial Analysis published no per-evaluation reasoning for the four removals made in v5.0. artificialanalysis.ai
6Activity: the board carried 188 entries on 6 August 2026, 210 on 22 August, 253 on 24 September, 294 on 15 October, 307 on 27 October and 308 on 4 November 2026. Gemini 4 Carbon, release date 28 October 2026, was added between 27 October and 4 November. Entry counts established by comparing archived snapshots of the leaderboard page against the live board; Artificial Analysis publishes no "last updated" timestamp for it. artificialanalysis.ai
7The v2 to v2.1 change took effect in the second half of September 2026, narrowed by archived page titles to between 15 September (still v2) and 20 September (first v2.1), and dated to "September 2026" by Artificial Analysis's own version history. Comparing the last v2 board with the first v2.1 board, 99 of the 227 entries present on both changed position relative to each other, almost all by one to four places and between entries inside each other's confidence intervals; the top six were unchanged. Every common entry's Elo fell, by 10 to 254 points, averaging about 122, as the scale compressed onto the 1600 anchor. Seven new entries arrived in the same five-day window, so the rescale cannot be cleanly isolated as the sole cause. artificialanalysis.ai
8Method note: the methodology page describes the v2.1 ratings as fitted with a Crowd-BT model in one place and as a Bradley-Terry rating computed by maximum likelihood estimation in another, without reconciling the two. Paper: arxiv.org/abs/2510.04374. Agent harness: github.com/ArtificialAnalysis/Stirrup.
9Cost per task is published by Artificial Analysis for GDPval-AA only as client-rendered charts; no per-model figures are stated as text, so none are quoted here. artificialanalysis.ai

Read next

AI is moving really fast. Keep up.

Read the Playbook