The PlaybookAI Benchmarks

Artificial Analysis Intelligence Index (October 2026): Fable 5.1 falls from first to eighth.

Fable 5.1 led the index at 53 in September and now sits eighth at 49. It has not shipped since 1 September. The index changed twice instead.

John TanOctober 27, 20265 min read

On September 17 Claude Fable 5.1 sat at the top of the Artificial Analysis Intelligence Index with a score of 53.37. Today it is eighth, at 49.

Anthropic has not shipped a new Fable since September 1. The model that fell seven places is the same model, byte for byte. What moved was the index.

1st → 8th

Claude Fable 5.1, from 53 in September to 49 now. No new release.

53 → 57

GPT-6 Astra over the same period. Also no new release.

2

Index versions in eight days. v5.0 on 27 October, v5.0.1 by 3 November.

Two models stood still and moved in opposite directions

This is the cleanest demonstration of the problem you will get. Between early September and now, neither Claude Fable 5.1 nor GPT-6 Astra shipped a new version. OpenAI's own changelog shows one Astra entry in the window, on September 29, and it adds a faster serving tier rather than a new model. Anthropic's lineup still lists Fable at 5.1.

Over that same stretch Astra gained four points and Fable lost four. If you plotted those two lines without knowing the index had changed, you would conclude that OpenAI pulled ahead and Anthropic's flagship regressed. Neither thing happened.

Figure 1. The same models, two indexes
Fable fell seven places without changing.
17 SEP 2026 · INDEX v4.3 4 NOV 2026 · INDEX v5.0.1 1 Claude Fable 5.1 53 2 GPT-6 Astra 53 3 Claude Opus 551 4 Muse Spark 1.348 5 GPT-5.6 Sol47 6 GLM-5.345 7 Grok 4.644 8 Kimi K344 1 Gemini 4 Carbon60 2 Claude Opus 5.559 3 GPT-6 Astra57 4 GPT-6.1 Sol55 5 Claude Sonnet 5.555 6 GPT-6.1 Terra51 7 Muse Wonder50 8 Claude Fable 5.149 Neither highlighted model shipped a new version between these dates. The index went from ten evaluations at 45% private to nine at 60%.
Artificial Analysis Intelligence Index: v4.3 from an archived snapshot of 17 September 2026, v5.0.1 read 4 November 2026. One entry per base model.

What actually changed

Version 4.3 of the index, the one September ran on, aggregated ten evaluations, with 45% of the weighting coming from evaluations whose questions or answers are held private. Version 5 cut that to nine evaluations and raised the private share to 60%.

That is a real improvement. An index built on public questions gets optimised against, and holding the answers back is how you stop measuring memorisation.

But look at what the change actually consisted of, because "the index was revised" undersells it. Artificial Analysis published the list. Four evaluations were removed outright: GDPval-AA, SciCode, AA-LCR and Humanity's Last Exam. FrontierCode 2.0 came in at 15% and Terminal-Bench-Science at 15%, CritPt was replaced by a hard subset at the same 10% weight, GDP.pdf was cut from 10% to 5%, AA-Omniscience went to v1.1, and the category weights were rebalanced.

Four out of ten evaluations gone and two new ones at 15% each. That is not a revision of the index, it is substantially a different index wearing the same name, which is why a model can lose four points without being touched.

Fable 5.1 did worse on the new mix. Astra did better. That is information, and it is useful information, but it is information about how each model performs on nine specific tasks chosen in late October, not about anything that changed inside either model since September.

And the index moved twice. Version 5.0 landed on October 27, and by November 3 the live board was labelled v5.0.1. Two methodology revisions in eight days, which is faster than most teams refresh the slide they are tracking this on.

Worth noting the same caution applies on the September side. The page said v4.3, but the published version history shows v4.3.1 and v4.3.2 also landed during September, and it was v4.3.2 that v5 replaced. The dates are given at month granularity, so the honest statement is "the v4.3 line of the index, as the page showed it on 17 September."

The coding benchmark moved inside the index

The other half of this column is usually SWE-bench, and the honest answer this month is that the question has changed shape.

SWE-bench Verified has been dead since February, when OpenAI's Frontier Evals team stopped evaluating it and said why: effectively saturated, and highly contaminated. Berkeley then broke eight agent benchmarks, and by June FrontierCode had reset the leaderboard to under 30%. None of that is new, and this column covered each step as it happened.

What is new is where FrontierCode ended up. FrontierCode 2.0 is one of the nine evaluations in Intelligence Index v5, at 15% of the weight, and it was not in v4.3's ten. The benchmark that replaced SWE-bench did not get its own leaderboard. It got absorbed into the composite.

GDPval went the other way, and this is the swap worth noticing. Artificial Analysis states it plainly: GDPval-AA is now a standalone evaluation, removed from the Intelligence Index in v5.0 after being a constituent through v4.3.2. It still runs, 220 tasks across 44 occupations, and its board tells a very different story about who is ahead. It just no longer feeds the number everyone quotes. So in one release the headline index took on more coding weight and dropped its measure of economically valuable knowledge work entirely.

Figure 2. Where the coding benchmark went
Not dead. Absorbed.
FEB 2026 JUN 2026 OCT 2026 SWE-bench Verified abandoned: saturated and contaminated → FrontierCode the reset. everyone back under 30% → FrontierCode 2.0 1 of 9 evals inside Intelligence Index v5 STILL TRUE, AND NOT NEWS Official Verified board's newest entry: 26 Feb 2026 · Vals archived its last run: 1 Sep 2026 The standalone coding leaderboard did not come back. It became one ninth of a composite score, which means a model's coding ability is now something you read off an aggregate instead of a board of its own.
Lineage from this column's February and June 2026 editions and from Artificial Analysis's published v4.3 and v5 evaluation lists, read 4 November 2026. Figure by nativefirst.

That is a quieter change than a leaderboard dying, and a more consequential one. A standalone board lets you ask how good a model is at code. A composite lets you ask how good it is, full stop, with coding folded in at roughly a ninth of the weight. If coding is most of what your agents do, the headline index number is now a worse proxy for your workload than it was in September, not a better one.

Three third-party trackers still publish standalone SWE-bench figures for the period. All three failed verification: none states which harness version or which problem subset produced its numbers, which makes them unusable even where they happen to be right.

How to read a month like this

Date every score, and version it. "Fable 5.1 scores 49" is not a fact. "Fable 5.1 scored 49 on AA Intelligence Index v5.0.1 on 4 November 2026" is. The habit costs you nine words and it is the difference between a number you can defend next year and one you cannot. The alternative is the Revolut cost figure that four accounts repeated and none could source.

Break your time series at the version change. Not a footnote, a visible break. Otherwise your chart shows Anthropic's flagship declining through October, which is false, and it will be the one slide somebody remembers.

When a benchmark goes quiet, say so. The temptation is to reach for whichever tracker still has numbers. A quarter with no SWE-bench data is a true statement about the state of public measurement, and more useful to a decision than a figure from a site that will not say how it ran the harness.

Rank on your own evals, not on a public board. The top three here are inside three points across three labs, and the fourth-place model is two points off that. Anthropic's own documentation routes you by eval failure rather than by tier, which is the same instruction from the other direction.

A flagship fell seven places without being touched, a rival gained four the same way, and the coding benchmark stopped being a board you can read on its own.

Date every score.

Portrait of John Tan
John Tan ↗

Founder of nativefirst.ai. Runs everything on agents and writes down what works.

Sources
1Artificial Analysis Intelligence Index, archived snapshot of the model comparison page dated 17 September 2026 and labelled Intelligence Index v4.3. Top of the board, one entry per base model: Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback) 53.37, GPT-6 Astra (max) 52.81, Claude Opus 5 (max) 50.70, Muse Spark 1.3 (max) 48.17, GPT-5.6 Sol (max) 47.06, GLM-5.3 (max) 44.86, Grok 4.6 (high) 44.41, Kimi K3 (max) 43.78. No score on that board is flagged by Artificial Analysis as estimated. artificialanalysis.ai
2Artificial Analysis Intelligence Index v5.0.1, read from artificialanalysis.ai on 4 November 2026: Gemini 4 Carbon (high) 60, Claude Opus 5.5 (max with fallback) 59, GPT-6 Astra (max) 57, GPT-6.1 Sol (max) 55, Claude Sonnet 5.5 (max) 55, GPT-6.1 Terra (max) 51, Muse Wonder (full) 50, Claude Fable 5.1 (max) 49. artificialanalysis.ai
3Index composition. v4.3 aggregated ten evaluations with 45% of the weighting from evaluations with private questions or held-out answers, per Artificial Analysis's own v4.3 launch article: "Evaluations with private questions or answers account for 45% of the Intelligence Index v4.3 weighting, up from 40% in v4.2." v5 aggregates nine evaluations (AA-Briefcase v1.1, AutomationBench-AA, Terminal-Bench 4.0, FrontierCode 2.0, AA-Omniscience v1.1, GDP.pdf, HLE-Diamond, CritPt v2 Hard Subset, Terminal-Bench-Science 0.1) with 60% private. artificialanalysis.ai
4Artificial Analysis Intelligence Benchmarking methodology page, read 4 November 2026. The published v5.0 change list: added Terminal-Bench-Science at 15% to the Science category; added FrontierCode 2.0 at 15% to the Coding category; reduced GDP.pdf from 10% to 5%; replaced CritPt with CritPt v2 (Hard Subset) at the same 10% weight; removed GDPval-AA, SciCode, AA-LCR and Humanity's Last Exam from the Intelligence Index; upgraded AA-Omniscience to v1.1; rebalanced category weights. On GDPval-AA v2.1 the same page states: "Status: Standalone evaluation (not part of Artificial Analysis Intelligence Index v5.0.1). GDPval-AA was removed from the Intelligence Index in v5.0, having been a constituent up to and including v4.3.2." GDPval-AA v2.1 covers 220 tasks across 44 occupations. artificialanalysis.ai
5Version dates. Artificial Analysis's site changelog records the methodology update to Intelligence Index v5.0 on 27 October 2026; the live board was labelled v5.0.1 when read on 4 November 2026. The published version history gives month granularity only for the v4.3 line, and records v4.3.1 and v4.3.2 as also falling within September 2026, with v4.3.2 being the version v5 replaced. This post therefore describes the September figure as the v4.3 line of the index as the page displayed it on 17 September 2026. artificialanalysis.ai
6No release of GPT-6 Astra occurred between 7 September and 27 October 2026. OpenAI's developer changelog records gpt-6-astra released 3 September, and the only Astra-specific entry in the window is 29 September, adding an "ultrafast" service tier, which is a serving option rather than a model version. Anthropic's model overview lists Claude Fable at 5.1, shipped 1 September 2026, with no later release. platform.openai.com
7Coding benchmark lineage. SWE-bench Verified was abandoned in February 2026, when OpenAI's Frontier Evals team published "Why we no longer evaluate SWE-bench Verified," calling it effectively saturated and highly contaminated; this column covered it at the time. FrontierCode reset the field to under 30% by June 2026. FrontierCode 2.0 appears among the nine evaluations in Intelligence Index v5.0.1 and does not appear in the ten that made up v4.3, so it entered the composite with v5 on 27 October 2026. The official SWE-bench Verified leaderboard's most recent entry remains dated 26 February 2026, and Vals archived its last independent run on 1 September 2026; neither fact is new. Three third-party trackers carry standalone figures for September and October; all three were excluded because none states its harness version or problem subset.
8Note on comparability: scores from before 27 October 2026 are on the v4.3 line of the index and are not comparable with v5 scores. The effort labels on the board (max, xhigh, high) are reasoning-effort settings of a single model, not separate versions.

Read next

AI is moving really fast. Keep up.

Read the Playbook