How ChatGPT Ads Are Delivered: Inside the LLM-Orchestrated Ad Stack

Hi everyone, Shii-chan here! Ever wondered how ads in ChatGPT work behind the scenes? OpenAI just shared the design, so let's look.

OpenAI News openai.com

What was announced?

OpenAI News explains how the company built ChatGPT's LLM-powered ad stack. It is designed to understand conversational intent, keep ads separate from answers, and deliver relevant ads at scale.

Why it matters

Traditional ad systems respond to discrete events such as a search query or page view. ChatGPT conversations develop gradually, intent can shift over several turns, and there's no obvious moment for an ad. So OpenAI built a bespoke delivery system around a post-trained frontier LLM that makes tool calls for retrieval, ranking, relevance evaluation, and auction-based selection and pricing.

What changes

For users, ads appear below the response, labeled as sponsored, without influencing the answer. Returning no ad is treated as a normal, successful outcome.

Dive Deep

There are three key design decisions:

• Stateful ad sessions: context is kept across turns and conversations instead of treating each message as standalone
• LLM orchestrator: it interprets conversation context and intent and runs the whole flow in an agentic loop by calling ads tools. It must tell current intent from stale context, recognize hypotheticals, and avoid treating every product mention as a purchase signal
• Separate ads service: ad selection runs in its own service with separate models and code, in parallel with answer generation, so ads cannot influence the organic response

The pipeline stages:

• Query preparation: the conversation is summarized, rewritten to broaden commercial possibilities, and converted to embeddings. It can generate multiple queries, such as transportation and lodging for a trip
• Retrieval: multiple indices are searched with approximate nearest-neighbor (ANN) search to build a broad candidate pool quickly, including specialized ones such as a product feed index
• Filtering: candidates are removed based on creative and campaign status, locale targeting, policy compliance, budget, pacing, placement compatibility, and per-advertiser deduplication
• Scoring: semantic relevance, predicted engagement (views, clicks, conversions), and historical performance are evaluated, plus relevance and safety checks
• Auction: the orchestrator nominates candidates, but the auction system makes the final decisions. The orchestrator neither sets prices nor picks the winner

In the article's example, a conversation about a two-day Yosemite trip, followed by questions about weather and a shoe brand's product lines, could yield a query like "shoes appropriate for hiking trip".

Delivery details:

• The orchestrator can block an ad that feels inappropriate in a given chat, even if the conversation and ad passed policy checks
• Ads that arrive after ChatGPT's response is ready are dropped for that turn, so ChatGPT never waits for ads
• Each ad bundle contains the creative (title, description, destination link, call to action) and a model-generated neutral preamble
• An ad counts as delivered only after ChatGPT displays it and the display is acknowledged
• Ad outages or latency do not affect organic responses

For measurement, outcomes are logged at each stage (turn not eligible, no candidates retrieved, delivered, and so on) and feed advertiser reporting and internal optimization. Privacy-controlled processing governs what data can be used for measurement and model training.

In the improvement loop, the strongest models evaluate system outputs and train smaller, cheaper models for orchestration and retrieval. Human feedback such as clicks and hides, surveys, and advertiser input through Ads Manager is weighted heavily.

OpenAI says ChatGPT Ads reached a $1 billion annualized revenue run rate in under 200 days after launch, with more than 100,000 advertisers. Future work includes more dynamic formats, such as letting users talk directly with businesses through ads.

Wrap-up

• Stateful sessions, an LLM orchestrator, and a separate ads service are the design pillars
• The pipeline runs query preparation, retrieval, filtering, scoring, and auction, with the auction making the final call
• Ads never delay the answer, and ad failures don't affect responses
• Stronger models train smaller ones in a continuous improvement loop

If you're interested in agentic LLM systems or ad and retrieval infrastructure, this is a great read!