DarkOwl Reports Cybercriminal Interest in Jailbreaking ChatGPT and Fraud-Focused AI Chatbots
Summary
DarkOwl analysts found darknet discussions of jailbreaking ChatGPT and advertisements for fraud-focused chatbots. They report that some users question the tools’ effectiveness and say they could not verify claims about DarkBERT.
Key points
- DarkOwl found more than 2,000 database results mentioning “jailbreak” and “GPT” across darknet forums, marketplaces, and Telegram channels.
- Forum users shared or discussed ChatGPT jailbreaking tutorials; some claimed the methods worked.
- DarkOwl observed discussion of using language models to automate penetration-testing tasks, including a reference to PentestGPT.
- FraudGPT and similar services were advertised as tools for generating phishing messages and fake websites, but some forum users questioned their effectiveness.
- WormGPT was reportedly built using the GPT-J open-source language model; DarkOwl says it was advertised on cybercrime forums and marketplaces.
- A seller known as CanadianKingpin12 advertised several chatbot services, including DarkBard; DarkOwl could not verify the seller’s claims about DarkBERT.
Article Details
- Event Type
- DarkOwl analysts report underground discussion and sale of AI chatbots marketed for cybercrime, alongside attempts to jailbreak ChatGPT.
- Impact
- The chatbots are marketed for generating phishing messages and fake websites that could trick users into sharing personally identifiable information; WormGPT reportedly can write malware using Python. DarkOwl notes that the effectiveness of these services is questioned and that some claims could not be verified.
Threat Actors
Vendors
GoogleFraud GPT is an AI chatbot that uses popular language models created by Google, Microsoft, and OpenAI and strips away any kind of ethical barriers around prompting the AI.MicrosoftFraud GPT is an AI chatbot that uses popular language models created by Google, Microsoft, and OpenAI and strips away any kind of ethical barriers around prompting the AI.OpenAIFraud GPT is an AI chatbot that uses popular language models created by Google, Microsoft, and OpenAI and strips away any kind of ethical barriers around prompting the AI.
Products
BardArtificial Intelligence (AI) has become a popular topic recently with the launch of ChatGPT and Bard.ChatGPTArtificial Intelligence (AI) has become a popular topic recently with the launch of ChatGPT and Bard.DarkBardDespite increased media coverage of fraudster AI chat bots like WormGPT, FraudGPT, and DarkBard, there is skepticism within both the underground cybercriminal community and the threat intelligence community that theDarkBARTCanadianKingpin12 is also tempting users with “DarkBART” and “DarkBERT” advertisements.DarkBERTCanadianKingpin12 is also tempting users with “DarkBART” and “DarkBERT” advertisements.EvilGPTWhile another user suggests using a fraudster chatbot service called EvilGPT, which is similar to FraudGPT:FraudGPTDespite increased media coverage of fraudster AI chat bots like WormGPT, FraudGPT, and DarkBard, there is skepticism within both the underground cybercriminal community and the threat intelligence community that theGPT-JRecently, the 2021 GPT-J open-source language model was leveraged for creating this hacker chatbot.WormGPTDespite increased media coverage of fraudster AI chat bots like WormGPT, FraudGPT, and DarkBard, there is skepticism within both the underground cybercriminal community and the threat intelligence community that the