Less than one in ten cybersecurity professionals trust AI testing tools to find vulnerabilities, and more than three-quarters say their AI vulnerability scanning tools failed to detect critical flaws.

Cobalt’s 2026 State of Pentesting report shows that confidence in fully automated AI testing plummeted from 29% in 2025 to 9% this year.
78% of respondents noted that automated tools were missing critical vulnerabilities; LLM failures proved complex: MTTR increased from 19 to 36 days and most issues remained unresolved
Hybrid models achieved 47% adoption, as experts emphasize that automation should complement, not replace, elite human expertise in business logic risk discovery.

While the world praises Mythos and the Chinese rush to create their own variant, Cobalt publishes a report that shows a completely different picture.

The cybersecurity company has just published the Cobalt State of Pentesting 2026 Report, based on two comparative surveys, one in 2025 and another in 2026. Surveying around 450 cybersecurity professionals, Cobalt wanted to see how confident the cybersecurity community is in automated AI testing to detect vulnerabilities and it turns out, not so much.

Last year, just under a third (29%) relied entirely on AI automation for testing. This year, the figure fell to 9%. Cobalt suggests that the key reason for such a steep drop in trust is the fact that 78% felt that fully automated scanning tools were missing critical vulnerabilities. Another key reason is the complexity of the AI attack surface that the scanners are testing.

Context-dependent vulnerabilities

Approximately one in three AI pentest results is considered “high risk,” which is 2.7 times the average for conventional software. Furthermore, at the time of analysis, less than two-fifths (38%) of LLM vulnerabilities were fixed, while 62% remained open. The mean time to resolution (MTTR) for AI/LLM security issues increased from 19 days to 36 days.

“LLM vulnerabilities are deeply context-dependent and invisible to tools that lack an architectural understanding of the application,” said Andrew Obadiaru, CISO at Cobalt. “To close the validation gap, automation must be deployed exactly where it excels, but elite human expertise remains critical to uncovering and remediating the most complex business logic risks.”

It took less than a year for the cybersecurity community to almost completely abandon fully automated AI testing and replace it with a hybrid model — something about 47% said they now prefer. This model has increased 22% year over year, while the percentage of organizations using automation for low-risk environments has also increased to 47%.

“While the industry is rightfully excited about the potential of Mythos-class tools, unguided algorithms are inherently prone to returning even more false positives and costly false negatives than the automated scanners we have today,” Obadiaru continued.

Through Infosecurity Magazine