That "We Caught a Russian Troll" Post? It Was Fake All Along!

Hi, it's Shiichan! Today I've got a story about a post that claimed AI caught a Russian troll, except the real story turned out to be something else entirely.

OpenAI News openai.com

What was announced?

OpenAI's News page detailed the story behind a viral X post from June 18 that claimed to expose a Russian troll account running out of ChatGPT credits while trying to generate pro-Trump content, complete with a fake error screen and JSON code as "proof." When OpenAI investigated, they found the screen could never have come from their models, and traced the whole thing back to a single account that likely originated in the United States.

Why it matters

"AI exposes a Russian disinformation operation" is exactly the kind of story that goes viral. And it did spread widely, except the actual misuse of AI ran in the opposite direction: the person who posted it used AI to fabricate the evidence themselves. The motive wasn't a coordinated influence campaign either, just someone trying to win arguments online. It's a good reminder that a screenshot of "what the AI supposedly said" can be faked by a human, rather than generated by the AI itself.

What changes

OpenAI banned the OpenAI account tied to this actor, and X suspended the corresponding account there. This wasn't a large-scale, AI-powered influence operation, it was an individual using AI-flavored props to stir up drama. But by investigating it thoroughly and publishing the findings, OpenAI gave everyone a concrete example of how fake "AI evidence" can spread.

Dive Deep

Here's the sequence OpenAI laid out:

• The account first used OpenAI's models to mass-produce short, argumentative replies to other people's posts on X, covering random topics like fantasy games, motorcycles, and flat-earth theories, argumentative in tone rather than ideological
• On June 18, the same account posted a fabricated screenshot showing a supposed Russian user running out of ChatGPT credits while trying to generate pro-Trump content, complete with fake JSON code
• Direct engagement on the post itself was modest (5 reposts, 14 quotes, 3 likes), but secondary posts discussing it spread at least a thousand times further, eventually drawing inquiries from mainstream media
• OpenAI rated it at the upper end of Category 3 on the Breakout Scale, close to Category 4 if mainstream media had picked it up further

What's interesting here is that the fire wasn't lit by the original post going viral directly, it spread because other people found it "spreadable" and amplified it themselves.

Wrap-up

• A viral X post claiming to expose a Russian troll turned out to be a hoax fabricated by a single US-based account
• The actor used OpenAI's models to generate argumentative replies, then fabricated a fake error screen as "proof"
• Direct engagement was small, but secondary spread reached mainstream media attention, landing at the upper end of Breakout Scale Category 3
• Both OpenAI and X banned or suspended the accounts involved
• Next time you see "AI-generated proof" going viral, it's worth pausing to question where it actually came from!