OpenAI announced a new benchmark called "GeneBench-Pro"! AI's genomics research skills are still only about 30% accurate

Hello, it's Shii! Today OpenAI announced a new benchmark that tests how far AI can make "researcher-like judgments" in the world of biology and genomics. Whenever I hear about a "test that measures what AI can really do," I just can't stop getting excited!

OpenAI News openai.com

What was announced?

On June 30, 2026, OpenAI announced a new benchmark called "GeneBench-Pro," which measures how well AI agents can handle realistic, multi-step data analysis tasks in the fields of genomics, quantitative biology, and translational medicine (the field that bridges basic research findings into clinical practice).

The benchmark is made up of 129 questions spanning 10 areas and 21 sub-areas of computational biology: statistical genetics, population genomics, quantitative genetics, regulatory omics, functional genomics, proteomics, clinical pharmacogenomics, cancer somatic genomics, microbial genomics, and forensic genetics. Just hearing the names made me think it sounds tough.

Each question comes with a dataset that deliberately includes realistic noise, an experimental background, and a target question. So it's not just a knowledge quiz; it's built to reproduce the actual research setting itself.

The story so far

It turns out OpenAI already had a predecessor benchmark called "GeneBench." But back then, even GPT-5 scored under 5% accuracy, so AI was still a long way from helping with practical-level biomedical research.

On top of that, most previous benchmarks mainly measured whether you know a fact, and they couldn't properly evaluate the points that researchers really wrestle with in the field, like measurement error, selection bias, confounding factors, and QC (quality control) failures. Research doesn't get to deal only with clean data, so if you can't measure that part, it's hard to tell whether an AI is truly usable.

What changes

The key point of GeneBench-Pro is that it builds that real-world messiness right into its evaluation criteria. It looks at the ability to handle measurement error, bias, and confounding factors, as well as reasoning and decision-making across an entire analysis workflow. That means we can now check whether AI can truly be a research partner in a way that's much closer to reality.

For researchers and AI development teams, this gives them a new yardstick for judging how far their own models might go in real work. Since even the top model only scores a bit over 30%, a future where AI takes over researchers' jobs wholesale still seems a long way off, but you can really feel it steadily evolving.

Dive Deep

From here, let me get a little serious with the tech talk.

First, about those scores you're curious about: OpenAI's latest model "GPT-5.6 Sol" reached 28.7% at its maximum reasoning level, and with Pro mode it climbed to 31.5%. Considering GPT-5 scored under 5% on the predecessor GeneBench, that's quite a jump.

Other companies' models were compared too, with these results: Anthropic's "Claude Opus 4.8" at 16.0%, Google's "Gemini 3.5 Flash" at 8.1%, "Gemini 3.1 Pro" at 3.1%, "Grok 4.3" at 1.5%, "GLM 5.2" at 4.6%, and "DeepSeek V4 Pro" at 2.4%. The GPT family is leading, but none of the models can get much past the 30% wall, which really tells you how tough this benchmark is.

The cost side is interesting too. It's estimated that if a human expert tried to solve a typical GeneBench-Pro question, it would take around 20 to 40 hours. Calculated at $200 an hour, that works out to thousands of dollars in labor costs per question. By contrast, the inference cost of today's AI models is only a few dollars per question. Accuracy isn't high yet, but these numbers suggest that even partial automation could already carry real economic value.

Third-party verification is properly considered as well. A representative set of 10 questions has been open-sourced on Hugging Face, and a 50-question subset was provided to Artificial Analysis, entrusting them to run independent benchmarking. I really thought it was honest of them to make it checkable by outside eyes, not just their own announcement.

Wrap-up

So, today was a story about OpenAI's new benchmark "GeneBench-Pro." With 129 questions close to the real world of genomics and biology, including measurement error and bias, and even the top model scoring just over 30%, it turned out to be a pretty tough test. The fact that AI can now take on problems that would cost a human expert thousands of dollars' worth of time, at a cost of just a few dollars, is a quiet but amazing change, I think. I want to keep an eye on how these scores grow from here!

Here's the original: Introducing GeneBench-Pro