Gen Threat Labs Finds Fake Stores, Phishing and Malware in AI Chatbot Recommendations

Summary
Gen Threat Labs documents cases where ChatGPT links led users to fake shops, phishing, infostealer downloads or compromised sites that redirected visitors. Its telemetry recorded over 260,000 threat-related events tied to AI chat services in 90 days.
Key points
- Gen Threat Labs reports more than 260,000 threat-related events associated with AI chat services over 90 days, and says its products protected more than 121,000 users.
- In one case, ChatGPT directed a user to a lookalike Upscayl site. Its download used a trojanized DLL and DLL hijacking with msiexec.exe to install information-stealing malware.
- Researchers say a compromised press association website used visitor profiling and cloaking to redirect some users from a ChatGPT citation through a traffic distribution system to scam content.
- Other cases included a fake EcoFlow store, phishing pages impersonating Netflix and Microsoft, and a ChatGPT-referred trading site that collected contact details.
- Gen Threat Labs also describes compromised sites loading malicious code through blockchain infrastructure and sending visitors to ClickFix pages that prompt them to run commands in PowerShell.
- The researchers recommend checking official software domains, independently verifying unfamiliar stores, treating unexpected login or command prompts cautiously, and keeping browser and security protections enabled.
Article Details
- Attack Vectors
- ChatGPT recommended a lookalike Upscayl domain offering a malicious download. The archive contained a trojanized SQLSetupBootstrapper.dll; the attack used DLL hijacking and msiexec.exe to install information-stealing malware.
- A fraudulent store impersonating EcoFlow appeared in ChatGPT shopping recommendations.
- Attackers compromised legitimate websites and injected code that profiled visitors, redirected selected users through traffic-distribution infrastructure, or loaded further malicious code.
- Related compromised sites directed visitors to ClickFix pages that instructed them to paste a command into PowerShell.
- Chatbot referrals led to phishing pages impersonating Netflix and Microsoft; a fake Microsoft login sent credentials to attackers.
- A ChatGPT referral led to BitGPT Pro, which collected names, phone numbers, and email addresses and used browser fingerprinting.
- Attackers can manipulate public web content to influence AI retrieval or recommendations, or register lookalike domains used in chatbot-recommended scams.
- Defensive Notes
- Verify software downloads against the developer's official domain before installing them.
- Independently check unfamiliar stores, including their contact details, policies, reputation, and relationship to the brand they claim to represent.
- Be cautious with investment sites that offer unusually attractive opportunities but provide little verifiable company information.
- Treat unexpected login screens, instructions to run commands, and AI-shared installation guides with skepticism.
- Keep browser and security protections enabled, including when following links recommended by an AI assistant.
- Assess what a website actually serves to a visitor; automated checks may see different content from users because of cloaking or redirects.
Indicators of compromise
| Type | Indicator | Context |
|---|---|---|
| DOMAIN | ecoflowofficial[.]shop | Fraudulent store domain impersonating EcoFlow and appearing in a ChatGPT recommendation. |
| DOMAIN | upscayl[.]io | Lookalike domain recommended by ChatGPT that offered a malicious download unrelated to the legitimate Upscayl developer. |
MITRE ATT&CK
T1056.003 · Web Portal CaptureA fake Microsoft login page captured credentials and sent them to attackers.T1059.001 · PowerShellClickFix pages instructed visitors to paste a command into PowerShell.T1189 · Drive-by CompromiseCompromised legitimate websites profiled visitors and redirected selected users to scams or other malicious content.T1204.002 · Malicious FileA user was directed to download and install a malicious Upscayl lookalike package.T1218.007 · MsiexecThe Upscayl download attack used the legitimate Windows utility msiexec.exe to install information-stealing malware.T1574.001 · DLLThe Upscayl lookalike download used DLL hijacking with a trojanized SQLSetupBootstrapper.dll.T1583.001 · DomainsThe article describes criminals registering plausible or lookalike domains to carry out scams, including the fake Upscayl and EcoFlow sites.T1608.007 · AI Result PoisoningThe article describes attackers manipulating publicly available web content to influence AI systems' retrieval or recommendations.
Malware
Vendors
Products
Agent Trust Hubextend that protection beyond traditional browsing. This is one of the reasons we developed the Gen Agent Trust Hub (ATH), which helps assess the safety of AI agents and the tools they use, monitor their activity andChatGPTor locate websites, but the links they recommend aren't always safe. Gen Threat Labs found examples of ChatGPT directing users to fake online stores, phishing websites and malicious downloads. Sometimes, evenClaudeearlier investigation, cybercriminals used Google sponsored search results to direct people to a shared Claude conversation that contained malicious macOS installation instructions. Users following the instructionsNorton Genie Scam DetectorAI assistant saw earlier. We also use AI to help people identify scams, with solutions such as Norton Genie Scam Detector. PowerShellto real estate and forums, sent visitors into ClickFix pages that tell them to paste a command into PowerShell. UpscaylA lookalike Upscayl download carried an infostealer
Countries
AustraliaWe saw a similar problem at a legitimate Australian medical-supplies shop. ChatGPT cited it as a shopping source, but its page carried an injected script that loads further malicious code through blockchainPerumost instructive cases did not involve counterfeit domains. A user asked ChatGPT about press freedom in Peru and received a citation from the Asociación Nacional de Periodistas (ANP), the country's national press
Industries
Journalismlooking for legitimate software to a website distributing an infostealer, and linking to a respected journalism organization whose website had been compromised to redirect visitors to scams. Over the 90 days coveredRetailmaker of portable power stations and other energy products. The website copied the company's branding and retail presentation, but its domain was recently registered and it lacked basic business information, including
