AI’s Cybersecurity Benefits and Risks: A Tradecraft Tuesday Recap
Summary
A Huntress webinar recap surveys how generative AI is used in social engineering and malware, as well as threat analysis and security operations, and explains why human review remains essential.
Key points
- The article describes AI use in phishing, translation, fake malware-laden AI tools, deepfake impersonation, and potentially polymorphic malware.
- Deepfakes can undermine identity checks: the article cites a $25 million transfer after an employee joined a video call with deepfake “colleagues.”
- It says the FBI warned in May that threat actors were impersonating senior U.S. officials with voice memos to build rapport before seeking access to personal accounts.
- AI can assist defenders with malware analysis, threat modeling, signal context, compliance tasks, and dynamic threat simulations.
- The article says AI models remain unreliable at binary decisions such as determining whether activity is malicious, and may produce hallucinations or be affected by prompt injection.
- It recommends rethinking verification procedures and requiring human review of AI responses; AI is presented as an aid to security staff, not a replacement.
Article Details
- Topic
- Generative AI's dual-use impact on cybersecurity, including AI-enabled scams and defensive security applications
MITRE ATT&CK
CVE
People
Chris HendersonHuntress Chief Information Security Officer who discussed AI's effects on cybersecurity in the Tradecraft Tuesday session.Sean HeelanResearcher who said OpenAI's o3 model helped find Linux Kernel flaw CVE-2025-37899.Truman KainHuntress Staff Product Researcher who discussed AI's effects on cybersecurity in the Tradecraft Tuesday session.
Vendors
DrataSecurity compliance platform Drata is developing AI tools that it claims will automate evidence collection, which can help with compliance.GoogleAnyone browsing TikTok, X (Twitter), or other social media platforms has likely come across AI-generated deepfakes (and if you haven’t, go Google “Tom Cruise deepfake,” and thank us later).OpenAIRecently, researcher Sean Heelan said that OpenAI’s o3 model helped find a Linux Kernel flaw (CVE-2025-37899).VantaTrust management platform Vanta said its AI security assessment tools will help standardize AI risk evaluation, a major benefit for businesses trying to scope out their compliance.ZySecFor example, ZySec’s CyberPod AI is designed to help engineers walk through threat modeling scenarios with real-time insights.
Products
DrataSecurity compliance platform Drata is developing AI tools that it claims will automate evidence collection, which can help with compliance.Gemini 1.5 ProGoogle’s Sec-Gemini and Gemini 1.5 Pro are both trained to perform malware analysis, and they’re particularly helpful in mitigating the malware polymorphism challenges outlined earlier in this blog.o3Recently, researcher Sean Heelan said that OpenAI’s o3 model helped find a Linux Kernel flaw (CVE-2025-37899).Sec-GeminiGoogle’s Sec-Gemini and Gemini 1.5 Pro are both trained to perform malware analysis, and they’re particularly helpful in mitigating the malware polymorphism challenges outlined earlier in this blog.VantaTrust management platform Vanta said its AI security assessment tools will help standardize AI risk evaluation, a major benefit for businesses trying to scope out their compliance.