Braintrust turns customer requests into previews in minutes with Codex!

Hey everyone, it's Shii-chan! Today's story is about how sheer speed can reshape the way a team builds. I had a lot of fun reading it.

OpenAI News openai.com

What was announced?

From OpenAI's News, we got a customer story about Braintrust, the observability and eval platform for shipping quality AI products. Braintrust's engineers can now use Codex to turn customer feature requests into preview branches and show working ideas to customers in minutes. And in just one month, half of the Braintrust team moved to Codex.

Founder and CEO Ankur Goyal says the biggest change isn't only faster coding, it's a faster feedback loop with customers.

The story so far

Before, when someone sent a feature request, it would sit in a backlog and get prioritized later. In Goyal's words,

"Previously, if someone gave us a feature request, it would enter a backlog and get prioritized later."

So requests waited instead of getting tried right away.

What changes

With Codex, the team can copy and paste a request, spin up a preview branch, and show the finished result to the customer in minutes. Instead of letting requests pile up, iteration becomes part of the workflow.

That means engineers can try ideas with customers in real time and build on them together.

"The more code we write, the more customer problems we can solve, and Codex is the most effective way to do that right now."

The more code you can write, the more customer problems you can solve.

Dive Deep

The fun part is how speed unlocks a whole new way of working. With other models, Goyal had to prompt the model step by step to solve a specific problem, and all that hand-holding raised the cost of experimenting.

With Codex, he shifted the flow like this.

• Write a test that demonstrates the problem
• Create a sandbox environment
• Let Codex run inside that environment

He calls it a novel use case, made possible by the speed. Rather than guiding every step, he defines the problem, lets Codex work in a controlled environment, and moves from idea to working solution fast.

On that speed, Goyal adds,

"Codex can literally print more text in the terminal without getting slow, and other models just can't replicate that."

Printing lots of text in the terminal without slowing down adds up, and it changes the whole experience.

Wrap-up

• Braintrust is an observability and eval platform for AI products, and its engineers now turn customer requests into previews in minutes with Codex
• In one month, half the team moved to Codex, and the biggest win is a faster feedback loop with customers
• Requests no longer wait in a backlog, they become previews the team can try with customers in real time
• A new, more autonomous style emerged: write a test, create a sandbox, and let Codex run

If you're a startup that wants to build shoulder-to-shoulder with customers, or a team that competes on shipping speed, this one is for you.