DarkOwl Reports Criminal Interest in AI for Data Poisoning, Prompt Injection and Malware
Summary
DarkOwl reports seeing threat actors discuss and sell AI tools in 2024, including tools associated with data poisoning and malware generation, alongside conversations about prompt injection and exposing sensitive data.
Key points
- DarkOwl says state-sponsored and independent threat actors discussed AI tools and tactics on underground platforms and Telegram.
- The article describes attempts to poison AI training data with extremist, hateful, or otherwise manipulated content.
- Examples of prompt-injection discussions included attempts to make AI reveal sensitive data or produce hateful output.
- DarkOwl says Nightshade was discussed and sold as a tool for poisoning images used in AI training.
- The article describes WormGPT as an AI model marketed without safeguards and claims it could be used for malware generation.
Article Details
- Event Type
- Report on emerging cybersecurity threats involving malicious use of AI
- Impact
- The article reports that threat actors discussed and sold AI tools on dark web platforms and Telegram. It describes risks including data poisoning, prompt injection intended to elicit sensitive data or harmful output, and AI-assisted malware generation. It does not report a specific confirmed victim or operational impact.
Threat Actors
Crimson SandstormNamed among state-sponsored threat actors focusing on malicious use of AI.Fancy BearNamed as a Russia-linked state-sponsored threat actor focusing on malicious use of AI; the article identifies Forrest Blizzard as an alias.KimsukyNamed as a North Korea-linked state-sponsored threat actor focusing on malicious use of AI; the article identifies Emerald Sleet as an alias.