Measuring how AI agents handle everyday wet-lab science

Paper-ready figures are a poor proxy for bench reality. BioBasicsBench asks agents to interpret raw experimental evidence across protein methods, imaging and cytometry, quantification, nucleic acids, and molecular cloning — including failed runs, overlapping dye signals, mislabeled plates, and cases where the data cannot support a conclusion.

At the highest shared reasoning setting, GPT-6 Astra earned 66.7% mean reward versus 43.8% for GPT-5.6 Sol, with gains OpenAI ties partly to stronger native multimodal handling. Seven models and 37 effort settings produced 18,500 graded attempts. Progress is real; reliability for unsupervised lab judgment is not.

If agents are going to help biologists accelerate disease and treatment research, they have to get the everyday evidence right — or they will burn weeks of wet-lab time on confident mistakes.

We're taking this benchmark seriously because it refuses the highlight reel. Tasks come from Ligo scientists' own experiments, with expert rubrics and a three-stage review that includes an external PhD wet-lab scientist. Credit is partial and criteria-based, so a right-looking conclusion with broken reasoning does not skate.

Astra's wins are the ones that matter at the bench: noticing overlapping detection channels instead of declaring cells dead from a noisy dye; catching that a "negative control" plate was labeled with the wrong antibiotic; reading a z-stack as depth, not as cells vanishing for biological reasons. That is multimodal competence plus experimental context, not trivia recall.

OpenAI is also honest about the misses, and those should set the tone. Astra can still let a familiar cloning assumption override what the controls say and purify the wrong colonies. It can inspect lighter and darker vacuoles, comment on focus-dependent contrast, then generalize only the darker rule. It can overcount colonies by roughly 12% by merging neighbors, skipping visible colonies, and marking empty agar. Those are not edge hallucinations — they are ordinary scientific failure modes with downstream cost.

I want this kind of measurement in the open. Useful scientific AI will earn trust by refusing unsupported conclusions as often as by reading a gel. The downside if scores get marketed as lab-readiness: biosafety and biosecurity stakes rise as agents get better at protocol-shaped work. Track the capability. Keep humans on the controls. Treat 66.7% as a promising student, not a colleague you leave alone with the freezer.

OpenAI published the benchmark introduction November 6, 2026, developed with Ligo, a biotech focused on enzyme design. Inputs span gels and blots, microscopy, sequencing files, instrument exports, protocol videos, and more — often several modalities per task.