Parloa's AMP: service agents customers actually want to talk to!

Hey there, it's me, Shii-chan! Today I want to share an exciting case study about how smart the AI agent on the other end of the phone line has become.

OpenAI News openai.com

What was announced?

This one comes from OpenAI's News. It's the story of how Berlin-based Parloa builds its "AI Agent Management Platform (AMP)", a system that hands entire call-center conversations over to AI.

In Parloa's early days, the team sat inside an insurance call center for a full day and heard the same conversations over and over: password resets, policy questions, routine changes. That's when they realized much of it could be automated. They started with rule-based voice agents, but when ChatGPT arrived they rebuilt everything into AMP on top of a new generation of models including GPT-5.4.

The story so far

The old way meant humans had to hand-write rigid intent trees and flow charts: if the caller says this, respond with that. The more branches you added, the harder it was to build and fix, and once it grew into one giant monolithic prompt, a tiny change could cause unexpected side effects.

AMP changes that by letting you define an agent's role, instructions, tools, and boundaries in natural language, without writing code. Subject matter experts who know the business can build the agents and connect the APIs themselves.

What changes

The biggest shift is who gets to build. You don't have to be an engineer anymore; the experts in each business unit can design their own agents.

Reliability in production is impressive too. Even across millions of conversations, most are resolved without friction, and when a call is handed to a human, it's usually not because something failed. In one deployment, a global travel company reduced requests for a human agent by 80%.

Dive Deep

There's a lot going on under the hood of AMP.

First, testing before going live. Parloa uses two models like GPT-5.4, one playing the customer who calls in and the other playing the configured agent, to simulate conversations. Then a separate model evaluates those conversations with a mix of deterministic checks and LLM-as-a-judge scoring, checking whether the agent followed instructions, used tools correctly, and completed the task.

Here is how the team thinks about picking models:

The models only matter if they work in production. We work closely with OpenAI on how to make the models fast and reliable enough for real-time conversations.

During a live call, the orchestration layer prompts an OpenAI model with the agent configuration and conversation context to generate a response, retrieve information via RAG, or trigger tools that talk to the customer's backend. After the call, separate workflows summarize the interaction, classify customer intent, and evaluate performance against defined rules.

As agents grew complex, a single monolithic prompt became painful, so Parloa switched to a modular approach, splitting tasks like authentication, booking changes, or account updates into separate sub-agents. That improves instruction-following and makes the system easier to evolve. For steps that absolutely must happen in order, they add deterministic controls like structured API chains and event-based logic, balancing flexibility with predictable execution.

Voice is even trickier. Everything runs through a low-latency pipeline of speech-to-text, model reasoning, and text-to-speech, so a small delay in the model layer turns into a noticeable pause for the caller. That's why Parloa evaluates each part of the voice stack separately:

• Speech-to-text: checked for word error rate, especially on sensitive inputs like policy or account numbers.
• Text-to-speech: assessed with blind listening tests for how natural the voice sounds.
• Speech-to-speech: currently being evaluated for production readiness, focusing on latency, accuracy, and cost.

It was built for global scale from the start, with benchmarks spanning multiple languages and customers across regions. Today Parloa's agents handle millions of conversations in retail, travel, insurance, and more, and they also use models like GPT-4.1 and GPT-5-mini for simulation and evaluation.

There's also an evaluation-first mindset: whenever a new model ships, they run their own benchmarking suite against it, because it matters that things work in real use cases, not just theoretical benchmarks. Enterprises face real migration costs, so they only switch when the benefits are clear.

Wrap-up

• Parloa's AMP is a no-code platform for building customer-service agents purely in natural language, on top of OpenAI models like GPT-5.4.
• An evaluation-first flow simulates conversations between models before launch, then scores them with LLM-as-a-judge plus deterministic rules to keep things reliable.
• Instead of one monolithic prompt, it uses a modular design of per-task sub-agents, with deterministic controls for critical steps.
• The voice stack is evaluated part by part (STT, TTS, speech-to-speech), built multilingual and global, handling millions of conversations across retail, travel, and insurance.
• One travel company cut requests to reach a human agent by 80%.

If you run enterprise voice or chat customer service you want to automate, or you're into evaluation and reliability design for running LLMs in production, this case study is worth a read.