AI vulnerability discovery has been streamlined into an assembly line—are you still doing your security assessments quarterly?

Finding vulnerabilities in a real, large-scale repository used to take an experienced Red Team several days or even weeks.

Now, someone has turned this into a pipeline. For US$1,461 per run, they managed to discover 144 automatically reproducible vulnerabilities.

This is not a demonstration; it is the actual results of testing on 8 real open-source projects, including OpenSSL, WordPress, and Rails.

If you still treat "penetration testing" as a project done once or twice a year, the offensive side's pace has already left your timeline far behind.

Why "AI Vulnerability Hunting" Used to Fail

Let's get one thing straight: letting AI read code to find vulnerabilities is not a new idea. The hard part is getting it to find "actually exploitable" vulnerabilities in real repositories, rather than stopping at the level of "looking like a vulnerability."

In the past, there were two paths, each with its own fatal flaw.

One is Static Application Security Testing (SAST). It relies on manually written rules to match code patterns. It runs fast and can fit into pipelines, but rules can never catch up with the ever-changing ways vulnerabilities are written in the real world — tight rules lead to a flood of false positives, while loose rules lead directly to false negatives. In empirical studies, its false positive rate can skyrocket from single digits to over 40%. As a result, only about 20% of developers are willing to actively use it, while the remaining alerts are drowned in noise.

The other is Fuzzing. It is precise, but it requires preparing complex compilation, instrumentation, and execution environments for every target, making it difficult to scale.

What about throwing the entire repository directly to a large model? That doesn't work either, as it runs into three engineering hurdles.

The context window cannot fit millions of lines of code. Even if it does, the model's ability to recognize information in the middle of long sequences drops significantly;

For APIs billed by token, the cost at this scale is astronomical;

Most fatally, if the model's self-reported "there is a vulnerability here" cannot be verified in a closed loop, it merely moves the false-positive problem from the world of rules to the world of probability, without solving it at all.

OpenAnt Chains Three Technologies into a Closed Loop

This new approach is called OpenAnt, from a security research team, with both the paper and code publicly available. Its philosophy is: do not let the model digest the entire repository at once, but first perform cheap filtering, then expensive reasoning, and finally real verification.

The first step is to reduce the analysis surface. First, static analysis is used to extract all functions and construct a call graph. Then, traversal is performed starting from external entry points — HTTP routes, CLI parameters, WebSockets, etc. — keeping only the functions that can be reached by external input. Internal utility functions, test scripts, and management scripts are all discarded.

The effect of this step is astounding: the 15,232 functions in the OpenSSL repository were compressed into 390 units worth analyzing. A total of 64,132 functions across 8 repositories left just over 2,000 after filtering. This step costs not a single cent in token fees, yet saves massive costs for later stages.

The second step is to let the model act as an attacker. For each candidate, the model is asked to play the role of a remote attacker with limited capabilities: interaction only through a browser, no server privileges, no administrator credentials, and no modification of server files. Under these constraints, it must construct a complete exploit path for the vulnerability, explaining step-by-step which moves are feasible and which ones would be blocked by authentication or input validation.

This step is the core of filtering false positives. Out of the 376 candidates flagged in the fourth phase, 49.5% were eliminated here — either because input sanitization blocked the attacker-controlled data flow, or because identity authentication could not be bypassed, or because the vulnerability only affected the attacker themselves, or because it was blocked by platform mechanisms (like Same-Origin Policy). This is exactly where it sets itself apart from traditional SAST: instead of just judging whether it "looks like a vulnerability," it repeatedly questions "can an attack actually get in."

The third step is a real run in a sandbox. For candidates that are still trustworthy, a Dockerfile, exploit script, and dependency configuration are automatically generated and thrown into a strictly restricted sandbox for execution: a read-only file system, 512MB RAM, single CPU, no privilege escalation allowed, and a 120-second timeout. All artifacts are destroyed after use. Only when it runs successfully and is confirmed to be reproducible does it count.

Final results: 190 vulnerabilities confirmed, 144 automatically reproducible, with a reproduction rate of 75.8%, covering more than 30 types such as IDOR, SSRF, path traversal, and injection. Among them, vulnerabilities where "input directly drives results" had the highest reproduction rates — command injection 100%, path traversal 88.9%, authentication bypass 83.3%; while vulnerabilities with complex environmental factors like race conditions and cross-service timing were harder to reproduce automatically.

The total cost of the run was US$1,461.25, averaging less than US$200 per repository. The authors calculated that without the first step of reachability filtering, running all functions would cost around US$23,700 — filtering reduced the cost by over 96%, making running large models to hunt vulnerabilities in real repositories economically viable.

Defenders Must Match the Same Pace

To translate this set of numbers into one sentence: vulnerability hunting + PoC writing, a task that originally relied on a veteran Red Team, is becoming a matter of running a pipeline in continuous integration.

Yet, many teams' security self-testing is still stuck in a "periodic" logic. A penetration test once a year, a sudden push before major drills, and relying on vulnerability scans to generate reports for compliance in normal times.

Vulnerability scans only tell you "there is a vulnerability here," they don't tell you "can this vulnerability be attacked from the outside." Penetration testing is more accurate, but limited by manpower and time, it stops at a certain point, failing to cover the complete attack chain or run daily.

When the offensive side's ability to find vulnerabilities becomes a sustainable, dispatchable, and low-cost pipeline, the defender's rhythm of intermittent self-testing is left far behind. The last time you verified your defenses was three months ago, while the attacker might be using AI to scan your exposure surface again every day.

The way to break this deadlock actually shares the same origin as OpenAnt: adopting an attacker's perspective to continuously and automatically test oneself.

Some teams in the industry are already doing this — turning penetration testing from a "temporary project" into a "continuously running capability" that can be scheduled and automatically generate reports, covering the entire chain from external information gathering to internal lateral movement. The CyriCART automated red teaming system does exactly this: using automated attack modules to continuously launch penetration tests against oneself, seizing the offensive side's automated advantage for the defense.

But "getting in" is only half the problem. After finding a vulnerability and confirming it is exploitable, the question that truly needs answering is another: does your existing defense system actually stop this attack path?

This requires adversarial validation. The CyriBAS intelligent security validation platform uses real APT attack scenarios to conduct adversarial validation on the defense system, analyzing alerts and logs of the entire attack process, locating points of failure in security products, processes, and personnel, and then providing actionable remediation recommendations.

Connecting the two forms a closed loop: the Red Team continuously finds attack paths, the validation platform confirms whether the defense lines can block those paths, and the results feed back to make the next round of testing more accurate. The offensive side has turned vulnerability hunting into a pipeline using AI; if the defensive side still relies on manual work and periodic checks, the gap will only grow wider.

US$1,461, 144 vulnerabilities, 8 real repositories. The true significance of this set of numbers is not "how powerful AI is," but that the cost curve of attacks is collapsing.

When finding vulnerabilities becomes this cheap and this fast, "we tested last quarter" is no longer a reassuring answer.

When was the last time your organization fully verified its defenses? Was it scheduled as a project, or is it already running continuously? Leave a comment to share your current practice.

References

OpenAnt: LLM-Powered Vulnerability Discovery Through Code Decomposition, Adversarial Verification, and Dynamic Testing（Nahum Korda, Gadi Evron, Knostic）→ https://arxiv.org/pdf/2606.19149