Here are two numbers for the same model, on the same benchmark, published in the same week.
99.9%. And 62.7%.
Both are GPT-6 Astra on ARC-AGI-3. The first is the one that went in the launch chart and got quoted everywhere. The second is what happens when you run it on the provider-neutral harness that everyone else's model was measured on.
That gap is the most useful thing to understand about this release, and it's not because OpenAI did something dishonest — the difference is documented. It's because the gap tells you exactly what kind of model GPT-6 Astra is, and therefore whether it belongs anywhere near your agent.
I've spent the last few days going through the announcement, the system card, the independent evaluations and the API docs. Short version: this is a genuine step change in a narrow band of work, roughly flat everywhere else, and 2.5x the price of the model it replaces.
Which makes "should I switch?" a much more interesting question than the headlines suggest.
What GPT-6 Astra actually is
OpenAI released Astra as a limited preview on September 3, 2026, opening it to paid users the following day. It rolls out across ChatGPT Plus, Pro, Business and Enterprise, plus the OpenAI API, Microsoft Azure and AWS Bedrock. It's the successor to the GPT-5.6 family, and it arrived later than expected — the company delayed it after the Hugging Face incident in July 2026 to add more safeguards.
It is, by OpenAI's own account, the biggest thing they've built. VP of research Aidan Clark told reporters it involved "by far" their largest training run, and that it was the first time they pretrained on more than 100,000 GPUs, at the Stargate site in Texas.
Here's what you're actually buying, from the API docs:
| Spec | GPT-6 Astra | GPT-5.6 Sol |
|---|---|---|
| Input price | $10 / 1M tokens | $4 / 1M tokens |
| Output price | $50 / 1M tokens | $20 / 1M tokens |
| Cached input | $1 / 1M tokens | $0.40 / 1M tokens |
| Context window | 1,050,000 tokens | 1,050,000 tokens |
| Max output | 128,000 tokens | 128,000 tokens |
| Knowledge cutoff | April 30, 2026 | February 16, 2026 |
| Modalities | Text + image in, text out | Text + image in, text out |
Notice what didn't change. Same context window, same max output, same modalities. If you were hoping for a bigger window, this isn't that release.
What did change is the tool surface. Through the Responses API, Astra supports web search, file search, image generation, code interpreter, a hosted shell, apply-patch, skills, computer use, MCP, and tool search. That list is the actual story — it's an agent-shaped model.
The 99.9% that wasn't quite 99.9%
ARC-AGI-3 is designed to test interactive reasoning — an agent taking actions in an environment and adapting. It's a good benchmark, and Astra's result on it was the headline of the launch.
The launch chart showed 99.9% for Astra, 30.2% for Claude Opus 5, and 7.8% for GPT-5.6 Sol. Read quickly, that looks like a generational leap over every rival at once.
The catch is the harness. OpenAI's own provider adapter preserves reasoning state between actions; the standard neutral harness resets it each step. Run Astra on the neutral harness and it scores 62.7%.
62.7% is still a very large jump over 30.2% and 7.8%. The model is genuinely better at this. But the comparison as charted puts Astra's best-case setup against rivals' standard setup, which is not the same claim.
The ARC Prize team themselves cautioned that saturating the benchmark isn't proof of general intelligence — they described the result as progress in interactive reasoning specifically.
This matters beyond scorekeeping. If a model's score swings 37 points based on whether the scaffolding around it keeps state, then the scaffolding is doing a lot of the work — which is exactly the argument we made in what the agent harness is and why it matters. The lesson generalises: your agent's performance is a property of the whole system, not the model name.
Where the gains are real
Strip out the contested number and there's still a lot here. These are the areas where Astra pulls clearly ahead, per the benchmark breakdown Vellum compiled:
Computer use
This is the biggest practical jump. OSWorld 2.0: 72.6% versus Sol's 65.7%. ScreenSpot-Pro, which measures whether a model can actually find the right element on a screen, went from 76.9% to 92.7%.
That 16-point jump in UI grounding is the difference between "demo that works on a clean screen" and "agent that can fill in your client's CRM." DataCamp reports Astra completing OSWorld tasks in about 40 minutes where Sol took 75.
Long-context retrieval
The window didn't grow, but the reliability inside it did. On MRCR v2 8-needle retrieval in the 512K–1M band, Astra hits 96.3% against Sol's 73.8%.
That's arguably more valuable than a bigger window. A 1M-token context you can only trust to 70% depth is a 700K-token context with extra steps.
Unattended professional work
AutomationBench — multi-step office work run without a human nudging it — went to 41.4%, against Fable 5.1 at 31.4%, Opus 5 at 26.9% and Sol at 18.1%.
Still under half. But more than double its predecessor, and that's the curve that matters for anyone building agents at the higher end of the autonomy ladder.
Not cheating
One alignment result deserves more attention than it got. On an ExploitGym honeypot designed to tempt models into gaming the evaluation, Astra cheated 0.0% of the time. Sol cheated 48.2% of the time.
If you've ever had an agent quietly satisfy the letter of your instructions while ignoring the point, you'll understand why that number is worth something.
Where it's flat — including one row the announcement skipped
This is the half of the picture that didn't make the headlines, and it's the half that decides whether you should switch.
On the independent Artificial Analysis Intelligence Index v4.1.1, Astra scores 61.2. Claude Fable 5.1 scores 65.7. Claude Opus 5 scores 63.1. Claude Fable 5 scores 62.1.
Astra is fourth. It is also, notably, roughly level with its own predecessor.
Coding is a tie rather than a takeover:
| Benchmark | GPT-6 Astra | Best rival | Verdict |
|---|---|---|---|
| Terminal-Bench 4.0 | 57.7% | Fable 5.1 — 55.8% | Astra wins |
| DeepSWE v1.1 | 74.1% | Sol — 72.7% | +1.4 over its own predecessor |
| FrontierCode 1.1 Ext. | 64.5% | Fable 5 — 64.9% | Astra loses |
| AA Coding Agent Index | 67.0 | Fable 5 — 68.1 | Astra third |
And then there's the row nobody quoted. On Humanity's Last Exam with tools, Astra scores 57.2% — behind Claude Fable 5.1 at 65.0%, Fable 5 at 63.8%, and Opus 5 at 63.6%.
Credit where it's due: OpenAI published that row in its own comparison table. It's in the data, it's just not in the prose, the highlights or any of the coverage — it's the one academic benchmark in the set where Astra loses to the field.
That's worth internalising as a habit rather than a gotcha. The numbers a vendor puts in the table and the numbers a vendor puts in the sentence are two different lists, and the second one is the marketing. Read the table.
A note on the wins, too
Several of the strongest results carry caveats worth knowing:
- FrontierMath Tier 4 (97.6%) — Epoch AI notes OpenAI funded the benchmark's development and has exclusive access to part of it.
- ExploitGym (42.4%) — the usual six-hour time limit was removed for this run.
- BenchCAD (95.9%) — the Claude comparison runs used modified evaluation settings.
None of these invalidate the results. All of them mean the gap is narrower than the chart makes it look.
What it costs, and why that's the real decision
Astra is $10 in, $50 out per million tokens. Sol was $4 and $20. Claude Opus 5 is $5 and $25.
So Astra costs 2.5x its predecessor and 2x Opus 5 per token. Put another way: Astra's standard rate is exactly what Anthropic charges for Opus 5 in fast mode.
Three more things the sticker price doesn't tell you.
The long-context surcharge. Requests over 272K input tokens are billed at 2x input and 1.5x output for the entire request. The 1M-token window is real, but using the top half of it is expensive in a way that compounds quietly.
It's slower. Astra runs around 46 tokens/second with ~3.19s to first token. Sol was 60 tokens/second at ~0.98s. If you're putting this in a chat interface, users will feel that.
Per-task cost, not per-token cost, is what you actually pay. Astra is more token-efficient — it needs fewer steps. Independent analysis still puts total cost per task around 75% higher than Sol at maximum effort. Better, but nowhere near erasing a 2.5x rate.
If you're not already tracking cost per completed task rather than cost per million tokens, that's the change to make before you evaluate any frontier model. We wrote up the method in the real cost of AI agents.
The cybersecurity story is the genuinely new thing
Astra is the first model OpenAI has classified at the "Critical" level for cybersecurity capability under its Preparedness Framework, as set out in its safety overview and system card.
In plain terms: given tools and access, it can find previously unknown security flaws and develop new ways to exploit them across well-protected systems, without a person directing each step.
The benchmark numbers are startling. ExploitBench: 100%, against Sol's 78.5% and Fable 5.1's 70%. SRE-Bench single-attempt: 88.0%, against Sol's 55.9% and Fable 5.1's 12.5%.
OpenAI's response was to ship the public model in a restricted form that refuses certain cybersecurity prompts, and to gate exploit-creation capability behind an approved-tester programme called Daybreak. Enterprise admins have to enable Astra per workspace, and it's off by default at launch.
For anyone building client-facing agents, the defensive half is the interesting half — secure code review and patching are now genuinely strong. The offensive half is somebody else's problem until it isn't; our rundown of AI agent security risks covers the exposure that actually applies to small builders.
The part safety researchers are unhappy about
Astra uses a new reasoning technique OpenAI calls recurrent depth, and it has a property that matters: it obscures the model's internal reasoning process.
Chain-of-thought monitoring has been one of the more useful safety tools of the last two years — you could read what the model was doing on its way to an answer. Recurrent depth makes that harder.
Safety researchers have raised this as a monitorability concern, and it's the thread running through most of the critical coverage, including Al Jazeera's and CNBC's reporting on the launch.
For a builder, the practical implication is narrower but real: if you were relying on reasoning traces to debug agent behaviour, you have less to work with. Lean harder on output-level evaluation instead — which is what you should have been doing anyway. Testing and debugging agents covers the approach.
What shipped in the products, not just the API
A model release is increasingly two releases: the weights, and the product surface built on top. Astra's product half is worth a paragraph each because some of it changes what you'd bother building yourself.
Document and deck generation. Astra produces polished documents, slides and spreadsheets that follow a supplied template. If you were planning to build an agent whose entire job was "turn this data into a branded report," check what ChatGPT does out of the box first.
ChatGPT Sites. It builds websites and web apps directly in ChatGPT with noticeably better visual rendering. This overlaps with a lot of what people currently use AI app builders for.
Clarifying questions, but fewer of them. Astra asks only when the answer would materially change the output. That sounds like a small behavioural tweak; in an unattended agent it's the difference between a run that completes and a run that stalls waiting for input nobody is there to give.
Mid-task instruction changes. It holds the original goal when you change instructions halfway through, rather than silently abandoning it. Anyone who has watched an agent forget the actual objective three tool calls in will recognise why that's on the list.
Searchable notes across context windows in Codex. Still experimental, but it's an attempt at the problem agent memory exists to solve — carrying state past the window boundary without stuffing everything back into the prompt.
How I'd actually evaluate it this week
If you want a real answer rather than a benchmark opinion, this takes an afternoon:
- Pull 20 real tasks your agent has handled — including the five it handled badly. Real inputs, not synthetic ones.
- Run them against your current model and record pass/fail plus cost per task. This is your baseline, and most people skip it.
- Run the same 20 against Astra at the same effort setting.
- Compare on three axes: did it get more right, how much more did it cost per completed task, and how much slower was it end to end.
- Decide per task type, not globally. If Astra wins on four of twenty, you've just discovered your routing rule — send those four to Astra and leave the rest.
The thing to resist is switching your default model because a chart looked impressive. A 2.5x price increase applied to 100% of your traffic to fix 20% of your failures is a bad trade, and it's the most common way people lose money on a frontier release.
So should your agent switch to GPT-6 Astra?
For most agents people are actually building, no — not as a default model. Here's the honest decision rule.
Switch if your agent does one of these
- It drives a real interface. Filling forms, clicking through a CRM, running QA on a live app. The ScreenSpot-Pro and OSWorld gains are large enough to change what's feasible.
- It reads genuinely enormous documents. If you're operating past 500K tokens, the MRCR jump from 73.8% to 96.3% is the difference between a working retrieval agent and a plausible-sounding one.
- It reviews code for security flaws. Defensive code review is the standout capability of this release.
- It runs long and unattended. The AutomationBench and honeypot-cheating results both point the same way: it holds a goal without a babysitter better than anything before it.
Don't switch if
- It writes, summarizes, classifies or answers questions. You are paying 2.5x for a model that scores below Claude Fable 5.1 on general intelligence.
- It's a chat interface. Three seconds to first token and 46 tokens/second is a worse experience than Sol at a third of the price.
- It's mostly coding. The coding indices have Fable 5 and 5.1 ahead or level. See our GPT-5.6 Sol vs Claude Fable 5 comparison, which still holds up.
- You haven't measured your current model's failure rate. If you can't say what's breaking, you can't say a new model fixes it.
The answer most people actually want: route
This release makes the case for model routing better than any argument I could write.
You have a model that is state of the art at computer use and long-context retrieval, fourth place at general reasoning, and 2.5x the price. The correct response to that shape is not to pick a winner. It's to send the computer-use step to Astra and everything else to something cheaper.
A concrete pattern: an intake agent that classifies and drafts with a cheap model, escalates to Astra only for the step where it has to operate the client's booking system, then drops back down. You pay frontier prices for the 5% of the workflow that needs them. That's the multi-model approach in one sentence.
Where this leaves you if you build on Pickaxe
Straight answer, because this is our platform and I'd rather be useful than promotional: GPT-6 Astra isn't available in Pickaxe yet. The current top OpenAI tier in the model list is the GPT-5.6 family — Sol, Terra and Luna — alongside Claude, Gemini and the rest.
New frontier models typically land in the builder within days to weeks of release, and when Astra does, switching an agent to it is a dropdown, not a migration.
That's the actual argument for building on a model-agnostic platform, and this release is a good illustration of it. If you'd hard-wired your agent to one provider's API in July, you'd now be rewriting to evaluate a model that — per the independent index — might not even be an upgrade for your use case.
Being able to run the same agent against Sol, Astra and Fable 5.1 and compare on your own tasks is worth more than any benchmark table, including the ones in this article.
FAQ
Is GPT-6 Astra better than Claude Fable 5.1?
Depends entirely on the task. Astra wins clearly on computer use, long-context retrieval, cybersecurity and unattended automation. Fable 5.1 leads on the Artificial Analysis Intelligence Index (65.7 to 61.2), on the Coding Agent Index, and on Humanity's Last Exam with tools (65.0% to 57.2%).
How much does GPT-6 Astra cost?
$10 per million input tokens and $50 per million output tokens, with cached input at $1. Requests over 272K input tokens are billed at 2x input and 1.5x output for the whole request.
Did GPT-6 Astra really score 99.9% on ARC-AGI-3?
On OpenAI's own provider adapter harness, which preserves reasoning state between actions, yes. On the provider-neutral harness used for competing models, it scored 62.7%. Both numbers are real; only one is an apples-to-apples comparison.
What is the GPT-6 Astra context window?
1,050,000 tokens, with up to 128,000 output tokens — the same as GPT-5.6 Sol. The improvement is retrieval reliability deep in the window, not window size.
Why is GPT-6 Astra restricted?
It's the first OpenAI model rated "Critical" for cybersecurity capability under their Preparedness Framework. The public version refuses certain cybersecurity prompts, exploit-creation capability is gated behind the Daybreak tester programme, and enterprise admins must enable it per workspace.
Can I use GPT-6 Astra in Pickaxe?
Not yet. The GPT-5.6 family is the current top OpenAI tier available in the builder. New frontier models are typically added within days to weeks of release.
The takeaway on GPT-6 Astra
GPT-6 Astra is the most capable model in the world at a specific set of things: operating a computer, reading a very long document without losing the thread, finding security flaws, and holding a goal through a long unattended task.
It is also, on independent measurement, fourth place at general intelligence and two and a half times the price of the model it replaces.
Both of those sentences are true, and only the first one was in the press release. That's not a scandal — it's just what a launch chart is for.
The useful move isn't picking a side. It's knowing your own workload well enough to say which of those four capabilities you actually need, and routing accordingly. If the answer is "none of them," you already have your model, and you should keep your money.
If you want to test that for yourself rather than take my word for it, build a small eval set from your own real tasks and run it across three models. It takes an afternoon and it beats every benchmark table on the internet — this one included.
