Malware Disguised as a Gaming Tool? OpenAI Disrupts a Russian-Speaking Hacker's ChatGPT Misuse, "ScopeCreep"

Hi, it's Shiichan! Today's story is a bit of a chilling one, so let me walk you through an OpenAI report on countering misuse of AI.

OpenAI News openai.com

What was announced?

OpenAI's News reported that it banned a cluster of ChatGPT accounts that appeared to be operated by a Russian-speaking threat actor. This actor used the models to help develop and refine Windows malware, debug code across multiple programming languages, and set up their command-and-control (C2) infrastructure.

OpenAI dubbed the operation "ScopeCreep," based on its use of a trojanized "crosshair" gaming tool and its stealthy tactics — the malware was distributed disguised as Crosshair-X, a popular crosshair overlay tool for video games.

Why it matters

Generative AI is good at writing and debugging code, which makes it an easy target for misuse in speeding up malware development. This particular actor showed real knowledge of Windows internals and some solid operational security habits, so this wasn't an amateur operation. The fact that even an actor like this got caught by AI providers' detection is the key takeaway here.

What changes

OpenAI banned all ChatGPT accounts tied to this activity and coordinated with the code hosting provider to take down the malicious repository. That means ScopeCreep can no longer keep using ChatGPT to continue its development, at least for now.

OpenAI assessed that the actor was likely trying to use its models to speed up malware development. Paradoxically, that also gave OpenAI a chance to spot and disrupt the threat while it still looked like it was in an early stage.

Dive Deep

This threat actor had a notably disciplined approach to operational security:

• Signed up for ChatGPT accounts using temporary email addresses
• Limited each account to a single conversation about one incremental code improvement
• Abandoned the account afterward and created a new one

This spread their footprint across many disposable accounts. The malware itself worked by having victims download and run the fake Crosshair-X, which launched a loader that then downloaded and executed additional malicious files from attacker infrastructure. From there it was built to escalate privileges, establish stealthy persistence, notify the attacker, and exfiltrate sensitive data across multiple stages.

According to OpenAI's report, the malware combined several techniques:

• C2 payloads designed to avoid signature-based detection
• Stealthy execution via DLL side-loading
• Obfuscation via custom packing with Themida
• Privilege escalation and evasion
• HTTPS traffic over port 80
• Credential and session theft
• Attacker notifications via Telegram
• Proxy-based traffic obfuscation

In actual conversations with ChatGPT, the actor asked for help debugging an HTTPS request in Go, and separately asked how to use PowerShell commands via Go to modify Windows Defender settings and add antivirus exclusions. OpenAI mapped these interactions to the LLM ATT&CK framework: compiling a file (python310.dll) so the actor's code runs whenever python.exe executes was classified as "LLM Aided Development," the Windows Defender exclusion work as "LLM-Enhanced Anomaly Detection Evasion," and debugging code that notifies the attacker via Telegram as "LLM-Assisted Post-Compromise Activity."

As for impact, quick reporting and close collaboration with industry partners who took down the malicious repository appear to have limited the damage. The malware's capabilities — privilege escalation, persistence, credential harvesting, and remote access — weren't particularly novel. While the malware was likely active in the wild, with some samples turning up on VirusTotal, OpenAI didn't see evidence of widespread interest or distribution.

Wrap-up

• OpenAI banned a cluster of ChatGPT accounts run by a Russian-speaking threat actor
• The actor used AI to develop and refine malware, "ScopeCreep," disguised as the popular gaming tool Crosshair-X, and to help build its C2 infrastructure
• Operational security was disciplined: temporary emails and one account per single incremental conversation, discarded after use
• The malware used multiple evasion techniques, including DLL side-loading, Themida-based obfuscation, and HTTPS over port 80
• OpenAI coordinated with the hosting provider to remove the repository and banned related accounts; no evidence of wide distribution was found

If you're into AI misuse research or malware analysis, this report is worth a read!