---
title: "GPT-6 Astra Scored 99.9% and 62.7% on the Same Benchmark."
url: https://nativefirst.ai/blog/gpt-6-astra-day-1/
description: "OpenAI shipped GPT-6 Astra claiming the AGI era. Its two ARC-AGI-3 scores differ by 37 points, and the gap came entirely from the harness."
author: John Tan
category: "AI Models"
published: 2026-09-04
updated: 2026-09-04
publisher: nativefirst.ai
format: Markdown copy of the web page at the url above, same text.
---

# GPT-6 Astra Scored 99.9% and 62.7% on the Same Benchmark.

OpenAI shipped GPT-6 Astra claiming the AGI era. Its two ARC-AGI-3 scores differ by 37 points, and the gap came entirely from the harness.

John Tan · September 4, 2026 · 6 min read

OpenAI shipped GPT-6 Astra on September 3 with a one-line pitch: "Anything you can do on a computer, Astra can do for you. Fast."

Greg Brockman, president and co-founder of OpenAI, went further. "I think it's not unreasonable to feel that we are now in the AGI era." Tobi Lutke replied to the benchmark chart with a single word: "Singularity."

Then the numbers arrived, and the most important one is not on OpenAI's chart. On ARC-AGI-3, Astra scored **99.9%**. It also scored **62.7%**. Same model, same benchmark, same day. What changed was the harness it ran inside. The 99.9% is the best Provider Adapter run, at high reasoning effort; the 62.7% is the best Standard run, at maximum. Hold both at maximum and the gap barely narrows: 62.7% neutral, 98.6% on OpenAI's own rig, for $8,766 less.

**ARC Prize · September 3, 2026** The two clusters are the same model. The top one is the Provider Adapter, the curve below it is the standard harness.

ARC-AGI-3 Semi-Private, score against total cost. Both axes are log scale.

**Fig. 1 · One model, one benchmark, two harnesses**

*Chart: Comparison of GPT-6 Astra on ARC-AGI-3 under two harnesses. Under the standard harness it scored 62.7 percent at a cost of 26,098 dollars. Under OpenAI's Provider Adapter harness it scored 99.9 percent at a cost of 18,817 dollars, running 3.66 times faster and using 49 percent fewer tokens. The higher score cost less. Prior models Claude Opus 5 at 30.2 percent and GPT-5.6 Sol at 7.8 percent are shown for scale.*

For scale, the same benchmark eight weeks earlier: GPT-5.6 Sol scored 7.8% in July, Claude Opus 5 scored 30.2%. The jump is real under either harness. The headline 99.9% is the one that is not reproducible outside OpenAI's own rig.

Data: ARC Prize blog and ARC-AGI leaderboard, September 3, 2026. Chart: nativefirst.ai

## What the two harnesses actually differ on

ARC Prize runs a standard harness on purpose: a provider-neutral interface so numbers can be compared across labs. Under it, a model carries forward only the notes it explicitly chooses to keep. Everything it is thinking is visible and portable.

OpenAI's Provider Adapter does something different. It **preserves opaque reasoning state between requests** and uses compaction to manage long conversations, so the model reuses prior work rather than reconstructing it. That is a legitimate product capability, shipped in the Responses API, and it is genuinely useful. It is also not what the other models in the comparison chart were given.

ARC Prize was careful about the conclusion, and the caution belongs next to the number.

> "While we believe Astra represents meaningful progress towards generalization, we are not claiming that it is AGI."
>
> **Greg Kamradt** · ARC Prize · [September 3, 2026](https://arcprize.org/blog/astra)

## The index that did not move

Here is the other number worth holding. On Artificial Analysis's Intelligence Index, Astra scores **61**. GPT-5.6 Sol, the model it replaces, also scores 61. Claude Fable 5.1 scores 66.

So an independent aggregate shows no gain at all, on the same day OpenAI published a sweep of state-of-the-art claims. That looks like a contradiction and it is not one. Read the two together and a coherent picture appears:

**Astra is a large jump in doing and roughly flat in thinking.** The benchmarks it wins measure long-horizon agentic execution: computer use, terminal work, persistence across hours. The index it does not move measures general reasoning quality. Both can be true, and the consequence for you is that this is not a blanket upgrade. It is a task-shaped one.

**ARC Prize · action efficiency** Each dot is one level Astra completed. Below the solid line means it used fewer actions than the median tested human.

Astra used fewer actions than the median tested human on 96% of levels. This is the "doing" half of the picture, measured.

## What people who actually used it said

The early-access reads were unusually consistent: enormous at operating software, less special at judgment.

> "It's a big upgrade from 5.6-Sol, with some frustrating habits that keep it from matching Fable at the top end."
>
> **Dan Shipper** · Dan Shipper, Every · [September 3, 2026](https://every.to/vibe-check/gpt-6-astra-vibe-check)

Shipper calls it the best writing model he has used, with very little slop and easy steering, and says computer use is "wild, it can go for hours at a time using complicated apps to get work done." Against it: it overcomplicates, and Anthropic's Fable still has better instincts for building a product. The tell is buried in the piece itself. Astra wrote the vibe check's first draft from a single prompt, and Shipper initially mistook it for a colleague's work.

> "It is now the best model we've ever tested on our expanded and hardest test set."
>
> **Aaron Levie** · Aaron Levie, Box, on their enterprise complex-work eval · [September 3, 2026](https://x.com/levie/status/2095598710311067716)

**Box · posted by Aaron Levie, September 3, 2026** The best model Box had tested, and on the full set the margin is three points.

Worth holding both numbers. The per-industry jumps are large, and the full-dataset gain is 74% to 77%.

Claire Vo, who builds ChatPRD, produced the most concrete build log: one-shot coding wins that Fable and 5.6 could not match, an AIM-style Mac desktop app in a single attempt, a hardware CLI she had chased since GPT-5.5, and a ChatPRD feature she had "thrown every model at for six months" finally landing at 90%.

Ethan Mollick pointed it at tens of thousands of his own emails, writings and calendar entries and had it assemble a personal knowledge base of research, contacts, ideas, relationships and tasks. That is the clearest published example of the long-horizon mode, and it is the one most likely to matter inside a company.

## The counter-take of the day

Against Brockman's AGI framing, the sharpest reply came from someone describing his own setup rather than the model.

> "All of my personal assistants are now reading each other's updates, and summarizing what the others learned. This feels way more like agi than astra."
>
> **signüll** · @signulll · [September 3, 2026](https://x.com/signulll/status/2095598202107900150)

That is the same lesson the ARC-AGI asterisk teaches, arrived at from the opposite direction. In both cases the thing producing the capability is the system around the model, not the weights. A 37-point swing from a harness change and a set of agents summarising each other are two versions of one finding.

## What it costs, and what it does not tell you

**$10 per million input tokens, $50 per million output.** That is 2.5x GPT-5.6 Sol, and it is exactly level with Claude Fable. Note the second comparison, because it is the one that matters for routing: at identical prices, Fable currently sits five points higher on the independent reasoning index, and Astra is far ahead on computer use.

Two billing details that are not in the headline rate. Search and computer-use tool calls bill **per call** on top of tokens. And rollout started through a limited Daybreak Access programme before ChatGPT tiers, the API and AWS, which Daring Fireball called a soft release and Forbes called a curious false start.

On safety, both frontier labs have now shipped a model with a cyber gate attached. Astra supports defensive work, secure code review and patching, and refuses advanced offensive work such as writing proof-of-concept exploits.

## The one thing to take from day one

Not "AGI is here" and not "the benchmarks are fake." Both readings skip the finding.

The most valuable number OpenAI published this week is the gap between 62.7% and 98.6% at identical reasoning effort, because that gap was produced entirely by scaffolding: reasoning state carried between calls, and compaction across a long context. At matched maximum effort it made the model 36 points better and 34% cheaper, and across every effort level it ran 3.66x faster on 49% fewer tokens.

If a harness change is worth 36 points to a frontier lab on a benchmark, it is worth more than that to you on your own workflows, where nobody has tuned anything yet. [The scaffolding is the product](https://nativefirst.ai/blog/what-a-software-factory-actually-takes/), and it is the part you own.

Buy the harness, not the headline.

[John Tan ↗](https://www.linkedin.com/in/johntans/)

Founder of nativefirst.ai. Runs everything on agents and writes down what works.

Sources

1 OpenAI, "GPT-6 Astra: A new generation of intelligence", September 3, 2026: claimed state of the art on FrontierMath Tier 4, ARC-AGI 3, TerminalBench-4.0, Terminal-Bench Science 0.1 and HealthBench Pro; ExploitBench 100% (GPT-5.6 Sol 78.5%); OSWorld 2.0 offline 72.6% (Sol 65.7%, Claude Opus 5 70.2%). Rollout via limited Daybreak Access, then ChatGPT Plus/Pro/Business/Enterprise, the API and AWS. Defensive cyber work supported; advanced offensive work such as proof-of-concept exploits refused. [openai.com](https://openai.com/index/gpt-6-astra/)

2 ARC Prize, "OpenAI's GPT-6 Astra on ARC-AGI-3", September 3, 2026: 62.7% on the Semi-Private set at maximum reasoning effort under the Standard harness at a total cost of $26,098, versus 99.9% at high reasoning effort under the Provider Adapter harness at $18,817, which was ~3.66x faster by elapsed time and used 49% fewer tokens. The Provider Adapter "preserves opaque reasoning state between requests and uses compaction for longer conversations, allowing the model to reuse prior work"; the Standard harness carries forward only notes the model chooses to keep. Prior scores: Claude Opus 5 30.2%, GPT-5.6 Sol 7.8%. ARC Prize's caveat: "saturating the benchmark would not represent proof of achieving AGI." [arcprize.org](https://arcprize.org/blog/astra)

3 Simon Willison, "GPT-6 Astra", September 3, 2026, for pricing and the benchmark caveat: $10/million input and $50/million output, matching Claude Fable; "clearly OpenAI's Fable competitor"; on Artificial Analysis's Intelligence Index Astra scores 61, level with GPT-5.6 Sol and behind Claude Fable 5.1 at 66. [simonwillison.net](https://simonwillison.net/2026/Sep/3/gpt6-astra/)

4 Sam Altman, September 3, 2026: "We believe it is the best model in the world for computer use, professional work, science, coding, cybersecurity, and more." Greg Brockman's "not unreasonable to feel that we are now in the AGI era" and Tobi Lutke's one-word "Singularity" reaction date from the same day. [x.com](https://x.com/sama/status/2095600005772104059)

5 Dan Shipper, "Vibe Check: GPT-6 Astra Is a Big Upgrade With Some Bad Habits", Every, September 3, 2026 (partly paywalled): "a big upgrade from 5.6-Sol, with some frustrating habits that keep it from matching Fable at the top end." Astra wrote the piece's first draft from a single prompt. [every.to](https://every.to/vibe-check/gpt-6-astra-vibe-check)

6 Aaron Levie, September 3, 2026, on Box's enterprise complex-work eval: "It is now the best model we've ever tested on our expanded and hardest test set." [x.com](https://x.com/levie/status/2095598710311067716)

7 Claire Vo, "GPT-6 Astra is a banger, here's everything I've built", Lenny's Newsletter, September 3, 2026: one-shot wins including a ChatPRD product-intelligence feature she had failed at for six months, an AIM-style Mac desktop app, a Divoom MiniToo hardware CLI chased since GPT-5.5, and Blender 3D assets. Her rule: browser use "for QA, not building." Also argues "UI is genuinely back" and what that means for SaaS and MCPs. [lennysnewsletter.com](https://www.lennysnewsletter.com/p/gpt-6-astra-is-a-banger-heres-everything)

8 Ethan Mollick, September 3, 2026: assigned GPT-6 tens of thousands of emails, writings and calendar appointments to assemble a personal knowledge base of research, contacts, ideas, relationships and tasks. [x.com](https://x.com/emollick/status/2095606622055760159)

9 @signulll, September 3, 2026: "all of my personal assistants are now reading each other's updates, & summarizing what the others learned. this feels way more like agi than astra." [x.com](https://x.com/signulll/status/2095598202107900150)

10 Daring Fireball, "OpenAI Soft-Releases GPT-6 Astra", September 3, 2026, on the staggered rollout. [daringfireball.net](https://daringfireball.net/linked/2026/09/03/gpt-6-astra)

11 Box, "Box AI Complex Work Eval for GPT-6 Astra", chart posted by Aaron Levie, September 3, 2026: GPT-5.6 Sol against GPT-6 Astra on Box AI V13, hard reasoning tasks on enterprise content. Media and Entertainment 62% to 77%, Technology 62% to 77%, Legal 64% to 72%, Consumer Products 87% to 90%, Energy 77% to 86%, full dataset 74% to 77%. Figures read from Box's published chart. [x.com](https://x.com/levie/status/2095598710311067716)
