AI Agents Are Changing Scientific Computing, Even Research Code!

Hey there, it's Shiichan! Today I found a pretty interesting report about how AI coding agents are being used in research.

OpenAI News openai.com

What was announced?

OpenAI's News published a field report on how AI agents are changing scientific computing software development. It analyzes eight projects, mostly in the life sciences, five using Codex alone and three combining Codex with Claude Code.

Why it matters

A lot of the software used in research started as code accompanying a paper, built by small academic teams with limited engineering experience. There usually wasn't much time for packaging, testing, optimization, or long-term support, so the software often ended up fragile and hard to maintain. That's exactly the kind of problem that's drawing attention to how AI agents can lower the cost of engineering work.

What changes

One case study centers on cyvcf2, a Python library for reading and writing genomic variant files. GPT‑5.5 replaced its legacy build and packaging system with a modern, unified process, making it easier to install, test, and release.

Here's how the developer, Brent Pedersen, put it:

With coding agents, it's quite easy to go fast; for now, to go far in science, there's still a need for expert guidance, understanding, taste, and care.

A recurring theme across the report is that researchers' role is shifting from doing the implementation themselves to verification and orchestration, deciding what to build, how to measure correctness, and when something's ready to ship.

Dive Deep

Agents handle specific, well-scoped requests well, but they can't reliably judge whether their output is scientifically valid. To make things trickier, they often sound confident even when their work contains clear errors, so human review is essential.

The validation approaches that worked best relied on an external reference or a measurable target: things like exact agreement with an existing tool, parity checks, appropriate statistical behavior, or answers established ahead of time with simulated data.

The projects also shared a common approach: none of them were one-shot efforts. They proceeded in stages, using feedback-driven iteration, breaking big goals into smaller changes and using intermediate benchmarks and tests to evaluate and refine the work. Initial implementations came together quickly, but resolving edge cases and subtle numerical differences, the last mile, took the most time.

Lower implementation costs also come with a risk: similar rewrites can proliferate and spread thin the expert attention any one tool needs. In these case studies, changes to MHCflurry and cyvcf2 were folded back into their original upstream projects, while rustar-aligner moved under new community stewardship because its original project had been abandoned. That's why the report closes on the importance of lining up clear long-term ownership.

Wrap-up

• OpenAI published a field report on eight AI-agent-assisted scientific computing projects (five with Codex alone, three combining Codex and Claude Code)
• One case study shows GPT‑5.5 modernizing the build and packaging system of cyvcf2, a genomics library
• Researchers' role is shifting from implementation toward verification and orchestration
• Judging scientific validity is still a human job; external benchmarks and feedback-driven, staged progress worked best
• Long-term stewardship of the resulting tools is called out as a key remaining challenge
• Worth a read if you're an engineer or researcher building or maintaining scientific software