Why AI Cybersecurity Benchmarks Can Miss Real-World Attacks
· 13:52 · Watch on YouTube ↗
Summary
Alt Security co-founder and CEO Noman Kehet argues that CTF and code-level benchmarks do not reflect black-box attacks. He describes learning applications before testing and says AI still struggles with complex vulnerabilities and false positives.
Key points
- Kehet argues that many AI cybersecurity benchmarks use CTFs or code-level vulnerabilities with known answers, unlike real-world black-box testing.
- He describes a “targeted attack model” that first learns an application’s services and business flows, then tests and validates vulnerability hypotheses.
- He says AI can chain some vulnerabilities but struggles with business-logic issues, IDORs, remote code execution, and XXEs.
- Kehet says a benchmark he ran against real targets with five AI models and default harnesses found simple vulnerabilities but missed complex chains and server-side vulnerabilities, and produced many false positives.
- He recommends validating AI findings, improving precision, and focusing on harder vulnerabilities and multi-step chains.
- He expects researchers’ intuition and custom harnesses to remain important as AI automates more pattern-matching work.
Article Details
- Topics
- [01:18] Why CTF and code-level benchmarks may not represent real black-box attacks
- [03:07] The targeted attack model: learning application services and business flows before testing
- [05:28] AI’s ability to chain vulnerabilities and its reported limitations
- [06:26] How AI may change vulnerability research and the role of human intuition
- [10:00] Reported findings from a black-box benchmark using five AI models and default harnesses
- [11:11] Advice for researchers: validate findings, focus on harder vulnerabilities, and build custom harnesses