Gen Threat Labs Finds Fake Stores, Phishing and Malware in AI Chatbot Recommendations

· Original article ↗

Summary

Gen Threat Labs documents cases where ChatGPT links led users to fake shops, phishing, infostealer downloads or compromised sites that redirected visitors. Its telemetry recorded over 260,000 threat-related events tied to AI chat services in 90 days.

Key points

  • Gen Threat Labs reports more than 260,000 threat-related events associated with AI chat services over 90 days, and says its products protected more than 121,000 users.
  • In one case, ChatGPT directed a user to a lookalike Upscayl site. Its download used a trojanized DLL and DLL hijacking with msiexec.exe to install information-stealing malware.
  • Researchers say a compromised press association website used visitor profiling and cloaking to redirect some users from a ChatGPT citation through a traffic distribution system to scam content.
  • Other cases included a fake EcoFlow store, phishing pages impersonating Netflix and Microsoft, and a ChatGPT-referred trading site that collected contact details.
  • Gen Threat Labs also describes compromised sites loading malicious code through blockchain infrastructure and sending visitors to ClickFix pages that prompt them to run commands in PowerShell.
  • The researchers recommend checking official software domains, independently verifying unfamiliar stores, treating unexpected login or command prompts cautiously, and keeping browser and security protections enabled.

Article Details

Attack Vectors
  • ChatGPT recommended a lookalike Upscayl domain offering a malicious download. The archive contained a trojanized SQLSetupBootstrapper.dll; the attack used DLL hijacking and msiexec.exe to install information-stealing malware.
  • A fraudulent store impersonating EcoFlow appeared in ChatGPT shopping recommendations.
  • Attackers compromised legitimate websites and injected code that profiled visitors, redirected selected users through traffic-distribution infrastructure, or loaded further malicious code.
  • Related compromised sites directed visitors to ClickFix pages that instructed them to paste a command into PowerShell.
  • Chatbot referrals led to phishing pages impersonating Netflix and Microsoft; a fake Microsoft login sent credentials to attackers.
  • A ChatGPT referral led to BitGPT Pro, which collected names, phone numbers, and email addresses and used browser fingerprinting.
  • Attackers can manipulate public web content to influence AI retrieval or recommendations, or register lookalike domains used in chatbot-recommended scams.
Defensive Notes
  • Verify software downloads against the developer's official domain before installing them.
  • Independently check unfamiliar stores, including their contact details, policies, reputation, and relationship to the brand they claim to represent.
  • Be cautious with investment sites that offer unusually attractive opportunities but provide little verifiable company information.
  • Treat unexpected login screens, instructions to run commands, and AI-shared installation guides with skepticism.
  • Keep browser and security protections enabled, including when following links recommended by an AI assistant.
  • Assess what a website actually serves to a visitor; automated checks may see different content from users because of cloaking or redirects.

Indicators of compromise

TypeIndicatorContext
DOMAINecoflowofficial[.]shopFraudulent store domain impersonating EcoFlow and appearing in a ChatGPT recommendation.
DOMAINupscayl[.]ioLookalike domain recommended by ChatGPT that offered a malicious download unrelated to the legitimate Upscayl developer.

MITRE ATT&CK

Malware

Vendors

Products

Countries

Industries

Related Articles