SDSignal Desk

AI can now redraw the molecule. Choosing which one to make next is still half-solved

· Updated · Signal Desk newsroom

Analysis of reporting by OpenAI. How we work

Image: OpenAI

OpenAI’s MedChemBench grades models on real medicinal chemistry problems. Its frontier models nearly ace structure reconstruction but earn only about half the credit on the choices that steer drug programs.

Key takeaways

  • MedChemBench has 99 problems drawn from 117 papers and patents, covering 761 distinct molecular structures.
  • At the xhigh setting GPT-6.2 Astra earned 84.5% of available credit, versus 83.3% for GPT-6 Astra and 58.2% for GPT-5.6 Sol.
  • GPT-6 Astra earned 95.4% of structure credit but only about half the credit for selecting compounds and predicting numerical properties.
  • Older models broke molecules outright, shrinking an 18-atom ring to 15 or splitting a compound into fragments.
  • OpenAI says the results do not show models can discover drugs, and prior exposure to the published sources cannot be ruled out.

A benchmark built on the hard part

Medicinal chemistry is a game of tradeoffs. A compound that binds its target tightly may be poorly absorbed or broken down too quickly, so chemists design, make and test related molecules and then decide what to try next. OpenAI’s new MedChemBench tries to grade AI on exactly that loop. Its 99 problems come from 117 published papers and patents and include 761 distinct structures. Models get the context a research team had at a given moment, including the target, the goals, the compounds tried and selected results, and must explain, rank and predict.

Scoring is split four ways: reconstructing molecular structures correctly, explaining findings with the supplied evidence, ranking and selecting compounds against stated requirements, and predicting measured or calculated properties. That breakdown is the most valuable thing in the release, because it shows where the overall score hides weakness.

The scoreboard, and what it hides

At the xhigh reasoning setting, GPT-6.2 Astra earned 84.5% of the available credit, GPT-6 Astra 83.3%, GPT-6.1 Sol 82.0% and GPT-5.6 Sol 58.2%. GPT-6 Astra at maximum effort reached 83.8%. On its own, that curve looks like a test nearly solved within a model generation.

The component scores tell a different story. GPT-6 Astra earned 95.4% of the structure credit and was strongest at explaining evidence, yet it collected only about half the credit for selecting compounds and predicting numbers. Those are the decisions that cost money in a real program: which analog to synthesize, which to drop. A model that writes lucid rationales but earns only half the credit on the choices themselves is a good teaching assistant and a risky project lead. OpenAI acknowledges substantial room to improve on exactly those tasks.

Why structure accuracy still matters

It is tempting to dismiss the structure results as table stakes, but the examples show why they are not. In one problem, models had to assemble a molecule while preserving an 18-atom ring. GPT-6 Astra got it right in all ten attempts. GPT-5.6 Sol and Luna failed all ten, one producing a 15-atom ring and the other splitting the compound into disconnected pieces. Once the molecule is wrong, every downstream judgment is about a different compound. That older models failed this basic step so consistently is a reminder of how recently chemistry reasoning from language models was unreliable at the foundation.

Another problem, drawn from work on PRMT5 inhibitors studied in cancer research, asked models to predict how three related compounds would perform on cell potency, permeability and liver-enzyme breakdown. The withheld results showed one compound was more potent while another crossed the barrier more readily and lasted longer. That kind of split is where real programs live, and where the benchmark says models still struggle.

Honest limits, and a useful yardstick

OpenAI is unusually explicit about what this does not show. The results do not establish that models can discover new drugs or improve lab decisions. Each answer was a single attempt with no tools, feedback or new experiments, and the company says prior exposure to the published sources cannot be ruled out. That last caveat is significant: if the model has seen the paper, predicting its results is recall, not reasoning.

Even so, a public, componentized benchmark is good for the field. It gives chemists a way to test vendor claims and gives labs a target that matches real work rather than trivia. It sits alongside efforts like Microsoft’s Quine biology research system and the agentic science push Nvidia showcased at SC26. The next honest test is a prospective one: compounds chosen by a model, made in a lab, and measured against what human chemists would have picked.

Who feels it

Medicinal chemists
Models look ready to help check structures and explain data, but not yet to pick which compounds to make.
Pharma and biotech R&D leaders
The component scores offer a sober way to scope pilots toward explanation and review rather than selection.
AI labs
A shared yardstick raises the bar for drug-discovery claims and makes contamination questions harder to dodge.
Patients
Any benefit is indirect and years away, dependent on models improving at the decisions this benchmark finds hardest.

What to watch

  1. Whether OpenAI or outside groups test models with tools and feedback rather than single attempts.
  2. Results on problems drawn from work published after model training, which would address contamination.
  3. Scores from non-OpenAI models if the benchmark is opened to other labs.
  4. Progress on compound selection and property prediction, the categories stuck near half credit.

FAQ

What is MedChemBench?

An OpenAI benchmark of 99 medicinal chemistry problems drawn from 117 papers and patents that tests structure reconstruction, scientific reasoning, compound selection and property prediction.

How did the best model score?

GPT-6.2 Astra earned 84.5% of available credit at the xhigh reasoning setting. GPT-6 Astra scored 83.3% overall but only about half the credit on selecting compounds and predicting numerical properties.

Does this mean AI can discover drugs?

No. OpenAI says the results do not establish that, noting single attempts without tools or experiments and possible prior exposure to the source papers.

Read the original

Continue at the source.

OpenAI

More on AI

All AI coverage →