ChatGPT Learns to Spot Risk in the Flow of a Conversation!

Hey everyone, it's me, Shiichan! Today I've got a serious-but-really-important piece of AI safety news. ChatGPT can now look at the whole flow of a conversation—not just a single message—to notice when something risky might be emerging.

OpenAI News openai.com

What was announced?

Over on OpenAI News, OpenAI shared safety updates that help ChatGPT better pick up on risk signals that build up gradually in sensitive conversations.

A request that looks perfectly ordinary on its own can mean something very different when earlier messages hint at distress. So OpenAI trained ChatGPT to read possible harmful intent from the surrounding context and respond by refusing, de-escalating, or guiding the person toward support.

The focus this time is on high-severity situations like suicide, self-harm, and harm-to-others. It builds on years of earlier work and more than two years of collaboration with mental health and safety experts.

Why it matters

In sensitive conversations, context can matter as much as any single message. A question that seems harmless by itself can become a warning sign once you read it alongside earlier signs of distress.

What makes this tricky is that risk signals can show up across separate conversations. One chat might carry a subtle hint, and a later chat might include a related request that only looks concerning when you connect the two. Without that context, the later conversation—and an important warning sign—can look benign.

These updates are about catching that "meaning changes later" kind of risk, both within a conversation and across conversations.

What changes

The key piece is something called safety summaries: short, factual notes about earlier safety-relevant context that might matter later.

These summaries are created by a model trained for safety reasoning tasks and are narrowly scoped, kept only for a limited time, and used only when relevant to a serious safety concern.

So a model trained for safety reasoning writes them, they're tightly scoped, kept only briefly, and used only when a serious safety concern is involved. Importantly, they're not general personalization or long-term memory—they exist to capture factual safety context.

ChatGPT is also trained to use this context more carefully, so it can tell when extra caution is needed and respond by de-escalating, declining to provide details, or redirecting toward safer alternatives.

Dive Deep

This work sits on top of OpenAI's safe completion approach, which refuses the unsafe parts of a request while still responding carefully where it safely can.

Mental health professionals from the Global Physicians Network—psychiatrists and psychologists with expertise in forensic psychology, suicide prevention, and self-harm—helped shape when a safety summary should be created, how much prior context is relevant, and how long the model should weigh that context.

And there are numbers, from internal evaluations built around challenging cases:

• In long single conversations, safe-response performance improved by 50% for suicide and self-harm, and 16% for harm-to-others.
• On GPT-5.5 Instant, the current default model in ChatGPT, it improved by 52% for harm-to-others and 39% for suicide and self-harm.
• The summaries themselves were graded too: across more than 4,000 evaluations, they averaged 4.93 / 5 for safety relevance and 4.34 / 5 for factuality.

And in everyday chats, adding this safety context didn't hurt quality—there was no meaningful user preference between responses with or without safety summaries.

Looking ahead, today's work covers self-harm and harm-to-others, but OpenAI says it may explore whether similar methods can help in other high-risk areas like biology or cyber safety, with careful safeguards.

Wrap-up

• ChatGPT can now read danger signals from the flow of a conversation and across conversations, and respond more carefully.
• The key is safety summaries: short, tightly-scoped, short-lived factual notes written by a safety-reasoning model—not long-term memory or personalization.
• Internal evals showed safe responses improving by up to 50%+, with summaries averaging 4.93 / 5 on safety relevance, while everyday chat quality stayed about the same.
• Great for anyone who cares about mental health and safety, or engineers curious about how AI safety actually gets built!