The 6-Step Playbook Exposed: OpenAI Disrupts an AI-Assisted Romance Scam

Hi everyone, it's Shiichan! Today's story gave me a bit of a chill: an OpenAI report on scammers trying to put AI to work as a fake romantic partner.

OpenAI News openai.com

What was announced?

OpenAI's News reported that it banned a cluster of ChatGPT accounts that were translating and generating short comments in Japanese, Chinese, and English. The activity appears to have originated in Cambodia and matched the pattern of a romance-and-investment scam known as "pig butchering." Around the same time, Meta identified related activity on its own platform tied to a newly stood-up scam compound in Cambodia.

Why it matters

This scam ring leaned on AI for translation and message generation. The operators' primary language seems to have been Chinese, but the model let them hold natural conversations in Japanese and English too. They specifically targeted men over 40 who appeared, from their public profiles, to work in medical professions, and they frequently replied to golf-related posts as a deliberate targeting tactic. When AI can bridge language gaps this easily, it can help scam operations run at scale across multiple languages, which is exactly why this report matters.

What changes

By banning this cluster of accounts, OpenAI cut off the network's ability to keep generating new comments and conversation with ChatGPT. OpenAI also shared information about the case with industry peers and relevant authorities. Meta independently confirmed related scam activity tied to Cambodia, showing how cross-platform information sharing is helping identify these operations.

Dive Deep

The scammers used the models for two main things:

• Generating comments to post on Facebook, X, and Instagram in reply to real people's posts. Profile pictures were copied from real models or influencers rather than AI-generated.
• Translating and generating short conversational messages, mostly between Chinese and the target language (Japanese or English). Operators sometimes asked for a specific tone, like a flirty young woman, and fed the model detailed fake biographies, including name, education, job, location, and hobbies.

Targets were mostly men over 40 whose public profiles suggested they worked in medical professions. The operation frequently replied to golf-related posts, a sign of deliberate targeting, and occasionally touched on political topics too. To reach and manage targets, the scammers used a wide range of tools, including:

• LINE, WhatsApp, X, Facebook, and Instagram
• Apple's "Hide My Email" feature
• Cryptocurrency and foreign exchange trading platforms

OpenAI observed a typical 6-step workflow:

1. Public engagement: comment on a target's post, often ending with a question to spark a reply
2. Likely move to DMs: generate short conversational messages, consistent with direct messaging
3. Move to a "secure" app: within days, suggest switching apps, citing distrust of social media or a logistical excuse
4. Romantic engagement: build increasingly intimate messages, dodging any request for photos
5. Financial pitch: boast about profits from investing in forex, crypto, or gold (often credited to a relative's advice) and suggest the target do the same
6. The fraud: push the target to transfer money into a trading app, then stall withdrawal requests with excuses like unpaid fees

Not every attempt worked. Some targets called out the scam outright or told the scammer to leave them alone, and some posts got no engagement at all. Still, some conversations referenced transactions worth thousands of dollars, which suggests at least some targets were actually defrauded, though OpenAI can't confirm whether the money changed hands. OpenAI's policies strictly prohibit using its tools for fraud, and it says it shared details of this case with industry peers and authorities.

Wrap-up

• OpenAI banned ChatGPT accounts tied to a Cambodia-based romance-and-investment scam network ("pig butchering")
• Fake profiles used photos stolen from real influencers to target men over 40 who appeared to work in medical professions, often via golf-related posts
• The AI-assisted playbook ran 6 steps: public engagement, DMs, moving to a "secure" app, romance, a financial pitch, and finally the fraud
• Contact methods spanned LINE, WhatsApp, several social platforms, and crypto/forex trading platforms
• Many attempts got zero engagement or were called out, but some conversations pointed to real financial losses in the thousands of dollars

If you care about AI safety and security, this report is worth a read - it's a sobering look at how AI's language skills can be turned into a scam accelerator.