Confidence in Autonomous AI Penetration Testing Falls as Firms Favor Human Oversight
Summary
Cobalt’s report says organizations relying on fully autonomous AI penetration testing fell from 29% in 2025 to 9% in 2026. False positives, missed vulnerabilities, validation demands and unpredictable costs favor human-in-the-loop testing.
Key points
- Cobalt reports that organizations willing to rely on fully autonomous AI penetration testing fell from 29% in 2025 to 9% in 2026.
- Cobalt says 78% of surveyed companies had automated systems miss significant vulnerabilities.
- FIRST analysts reported vulnerabilities were being reported at a rate 46% higher than forecast from the prior year’s data.
- Experts say AI-generated findings require human validation, risk judgment and prioritization; false positives and false negatives remain concerns.
- The article describes a likely near-term model in which AI handles broad, repetitive testing while human testers validate findings and assess risk.
- Unpredictable costs are another concern for organizations considering AI-powered penetration-testing services.
Article Details
- Event Type
- A report and industry commentary describe declining confidence in fully autonomous AI penetration testing.
- Impact
- Cobalt reported that the share of organizations willing to rely on AI-powered penetration testing fell from 29% in 2025 to 9% in 2026. Practitioners cited missed significant vulnerabilities, false positives, human validation workload, and unpredictable costs; most organizations favored a human-in-the-loop approach.
People
Derek RushBishop Fox managing senior consultant who discussed the need for human experts to assess AI-generated leads and validate attack chains.Eireann LeverettFIRST analyst who co-wrote an analysis describing human verification, coordination, and patching as bottlenecks for AI-discovered flaws.Gunter OllmannCobalt chief technology officer who discussed practitioners’ declining confidence in autonomous penetration-testing systems and concerns about their costs.Jerry GamblinFIRST analyst who co-wrote an analysis describing human verification, coordination, and patching as bottlenecks for AI-discovered flaws.Sandeep SinghHackerOne vice president of product strategy who discussed false positives, validation workload, and the role of AI in security testing.
Vendors
Bishop Foxup, requiring more human oversight and better systems, says Derek Rush, managing senior consultant at Bishop Fox, an offensive security services firm.Cobaltare prone to false positives, and can blow through AI budgets, according to a June 25 report released by Cobalt, a penetration-testing-as-a-service firm.HackerOne#CobaltMicrosoft#Cobalt