AI · Oct 21, 2026
OpenAI on Google Cloud lets companies spend Google commitments on GPT-6, and that changes procurementOpenAI trains GPT-6 Astra on 11 Ironclad contracting tasks; average score rises to 55%
· Updated · Signal Desk newsroom
Analysis of reporting by OpenAI. How we work

OpenAI and contracting platform Ironclad built 11 legal, commercial and procurement tasks to train and evaluate agents. GPT-6 Astra averaged 55.0% on rubric criteria versus 41.6% for GPT-5.6 Sol, with an internal model at 63.7%, results that show progress and how far agents remain from full trust.
Key takeaways
- Ironclad is OpenAI’s first partner for turning real workflows into agent training tasks.
- The 11 tasks include NDA setup, procurement approvals and jurisdiction-specific clauses.
- GPT-6 Astra averaged 55.0% versus 41.6% for GPT-5.6 Sol; an internal model reached 63.7%.
- Estimated time per attempt fell from 37.0 to 19.2 minutes, based on simulated estimates.
- Training tasks were built from public SEC filings with personal information filtered out.
What OpenAI and Ironclad built
OpenAI says it is partnering with a small number of software companies to turn hard, high-value workflows into training and evaluation tasks for its agents, and Ironclad, a contracting platform, is the first. Together they built 11 tasks across legal, commercial and procurement work, such as setting up nondisclosure agreements, building procurement approval processes and updating a reusable clause for a chosen jurisdiction. OpenAI estimates each would take an experienced user about 30 to 40 minutes.
GPT-6 Astra is the first OpenAI frontier model trained on these tasks (Astra launch). Across them it averaged a 55.0% rubric score against 41.6% for GPT-5.6 Sol, with estimated time per attempt falling from 37.0 to 19.2 minutes. An internal model reached 63.7%. Ironclad CTO Sunita Verma framed the goal as agents that understand the full contracting lifecycle while preserving the controls teams rely on.
A better evaluation method
Each task is graded against 8 to 50 specific criteria, agents practice in hosted copies of the real software, and success is defined with people who do the work. That is a concrete step beyond broad leaderboards and vague demos. OpenAI says training tasks were built from public SEC filings with personal information filtered out, not from customer or internal contracts. It is inviting other software companies to apply, asking for concrete failing tasks, deep domain experts, secure test environments and research-safe data.
What the numbers do and do not show
A 55% average means the agent misses a large share of what these tasks require. Even the showcase clip, where Astra met about 94% of criteria, leaves some requirements unmet. In contracting, the missing few percent can be the approval rule that lets a large purchase skip Finance or a clause citing the wrong jurisdiction. OpenAI and Ironclad acknowledge that an agent losing track of one rule midway limits what anyone can confidently delegate, which is why human oversight still matters.
OpenAI’s footnotes say the times are simulated estimates based on assumed model speeds, not measured customer savings, and the results cover only these 11 tasks rather than Ironclad’s workflows broadly. That is candid, and it means the results are not proof that contract operations can be automated today. If the partner model scales, frontier labs may shape models around the workflows of early vendors, and legal and procurement teams may face pressure to trust systems before error rates justify it. OpenAI has since introduced newer models (GPT-6.1 Sol). More in our OpenAI coverage, agents coverage and enterprise coverage.
Who feels it
- Legal and procurement teams
- Agents are improving, but error rates demand careful review.
- Enterprise software vendors
- A path to have frontier models trained on their workflows.
- Ironclad customers
- Potentially more capable AI features over time.
- AI evaluators
- Criteria-based task rubrics as a more concrete benchmark model.
What to watch
- Which software companies join next.
- Measured rather than simulated time savings.
- When the internal model’s 63.7% reaches a shipping model.
- Review steps Ironclad builds around agent-configured workflows.
FAQ
What is the OpenAI–Ironclad project?
A partnership turning 11 real contracting workflows into agent training and evaluation tasks.
How did GPT-6 Astra score?
55.0% on average, versus 41.6% for GPT-5.6 Sol; an internal model reached 63.7%.
Are the time savings measured?
No, OpenAI says they are simulated estimates.
Was customer data used?
No, tasks were built from public SEC filings with personal information filtered out, OpenAI says.
Companies: OpenAI
More on OpenAI
AI · Oct 14, 2026
White Circle says newer OpenAI and Google image models are easier to jailbreak than their predecessorsAI · Oct 19, 2026
Microsoft ships Guardian Agents, saying frontier models ‘can’t be trusted’AI · Oct 13, 2026
After months off the frontier, Google ships Gemini 4 Argon with benchmark claims and aggressive pricingAI · Sep 22, 2026
Opus 5.5 hands risky cyber prompts to an older Claude model, and that routing is the real launch story