Alone, the AI solves 21 percent.With a human beside it: 64.

There is a comfortable story in which the machine takes over security testing and the human becomes a cost you can cut. That story met reality during 2026. Reality won.

The reason is experience, not principle. Seventy-eight percent of organisations report that fully automated tools missed serious vulnerabilities and reported all clear when it was not.[01]

Researchers built a set of tasks called AutoPenBench. They ran the same AI against the same tasks in two modes. First alone, then with a human supporting it.[02]

Another study from December 2025 built 346 tasks across twelve realistic environments and had experts define the correct answers. Systems left to work all the way through on their own managed 31 percent, and the researchers note that the most independent of them fail almost entirely.[03]

That does not make AI weak. In the US research agency DARPA's competition, the competing systems found 54 of 63 planted vulnerabilities, plus 18 previously unknown ones in real code, at an average cost of 152 dollars per task. The machine is fast and cheap across breadth. It is unreliable at the conclusion.[04]

The explanation is simpler than the debate. Some flaws can be proven mechanically. If an AI sends something that makes the program crash, then it crashed. You can see it. The AI can try again and again until reality answers yes or no.

Other flaws have no such proof. Nothing crashes when one user can open another user's invoice. The system does exactly what the code tells it to. What is broken is the intent, meaning what you meant the system to allow. And the intent is written down nowhere in the code.

In June 2025 the news went round that an AI system had taken first place on one of the big platforms where security researchers report vulnerabilities for payment. The headline did its job. The detail did not.

The ranking was the US list, over three months, measured in a score that says nothing about how serious the findings were. Behind the system sat a team of at least 25 people. Every report was read by a human before submission, and how many findings were thrown away internally has never been published.[06]

The interesting part is what the breakthrough actually rested on. Not a bigger AI model, but a separate checking step that proved the vulnerability could be exploited before the report went out, plus human review on top. The industry's most publicised AI success was built on not taking the AI's word for it.[07]

The EU AI Act requires important AI systems to be capable of human oversight. It explicitly names the risk that people trust what an AI says too readily, and requires that its output can be overridden.[08]

An AI-assisted pentest service almost certainly does not fall under those rules, and we do not claim it does. But they were written by people who studied the problem, and they describe what the measurements show: what an AI says needs someone able to say no.[08]

Our AI works in five steps: it maps what is there, examines how it fits together, tries to get in, checks what it managed, and writes the result up. It records everything it does as it goes. That log is both the pentester's live view and the documentation of what was actually done.

There is one thing it cannot do. It cannot approve a finding. There is no button for it and no field to write in. Findings land unconfirmed, a human reproduces the flaw by hand, and only confirmed findings go into the report. What does not hold up is discarded before you see it.

That is not caution. It is the only way of working that matches how the technology actually behaves.