Six CVEs in 163 Minutes: An Autonomous Pentest, Reasoning Included

A benchmark tells you how a tool scores. An engagement tells you whether it actually works. We have run the benchmarks: KLUE against four DAST scanners and four SAST tools, on a deliberately-broken practice app, graded on a curve. This is not that. This is one real, in-scope codebase, a running instance, the full source, and a clock. No answer key. No second tries.

Two hours and forty-three minutes later, KLUE walked away with six CVE-worthy vulnerabilities. Not missing headers. Not maybes. A chained second-order SQL injection that took three requests and a defeated safety check to reach. A template injection we rode all the way to an interactive shell. An unauthenticated signature bypass that handed over write access to anyone who asked. A server-side request forgery reaching straight into cloud metadata. And two reflected XSS for good measure. Four rated High, two Moderate, every one now responsibly disclosed, fixed, and assigned a CVE.

And it was not even finished. The run did not stop because KLUE ran out of things to look at. It stopped because the clock ran out, mid-investigation, with live leads still on the table. Six is the floor of what it found in one sitting, not the ceiling.

Here is the slate it walked away with:

What makes this worth reading isn't the count. It's that every one of these came with a trace you can audit. A scanner hands you a finding and a confidence score. KLUE hands you the chain of reasoning that produced it, including the moments it talked itself out of a dead end. The rest of this post is that trace, for the two findings that a signature engine could never have reached.

How the agent actually runs

Before the findings, a quick look at what you are reading in those excerpts, because the format is the methodology. KLUE does not scan. It runs a loop, and every line in the log is one move in that loop: form a hypothesis about where a bug could live, pull exactly the code needed to test it, reason about that code in the open, then either validate and file the finding or kill it with a reason. The tags in the trace map straight onto those moves:

• [STATE] is the current hypothesis, the thing it is actively chasing (Investigating render handler for arbitrary content injection).
• [TOOL] Reading and [TOOL] Searching are retrieval. KLUE does not ingest the whole repository and hope. It reads specific files and greps for specific patterns the way a human reviewer does: header\s*\(\s*['"]Location when it is hunting open redirects, createTemplate|renderFromString when it is hunting template sinks, allow_raw_html when it wants to know whether a dangerous default is on.
• [AGENT] is the reasoning itself, verbatim and unedited in spirit.

Underneath, it keeps books. Candidate findings get numbered as they appear (#203, #204, #206 show up in this run), and cleared areas go into a coverage ledger tagged exhausted with the reason they were dropped, so the agent does not burn budget re-litigating ground it already settled. One tight cycle, start to finish, looks like this:

Two things in there are what separate this from a scanner mechanically, not just in marketing.

First, retrieval is hypothesis-driven, not exhaustive. KLUE pulls the login action because it is chasing a redirect, the link tracker because it is chasing an orphan check. The same file gets read three or four times from different angles as the theory sharpens. It is following data, not enumerating files.

Second, validation is adversarial and it argues both ways. Before anything is filed, the agent builds a proof-of-concept and reasons out a severity, and it talks impact down as readily as up: the two XSS bugs were held at Moderate because they need a click, and a perfectly real open redirect was dropped entirely as not worth a CVE. A static-analysis sweep does run late in the engagement (let me run a quick static analysis scan to catch anything I might have missed), but it is a backstop. The primary engine is a model reading code the way a senior reviewer reads it.

And all of it runs against a clock. The time budget is a first-class control, not an afterthought: it bounds exploration and forces the agent to spend attention where the payoff looks highest, which is exactly why this run ends mid-thought instead of at a tidy stopping point. On this engagement, that budget was 2 hours and 43 minutes. Now the two findings worth slowing down for.

The headline: a SQL injection it had to reason its way into

Most automated SQL injection findings are first-order: a parameter goes straight into a query, you fuzz it with a quote, the database complains, done. This was not that. This was a second-order bug behind a three-step setup, and the only way to it was to understand how the application tracked relationships between its own pages.

KLUE started at the sink. A page-delete API endpoint came down to a handful of lines like these:

// page-delete API handler  (names redacted)
public function deletePage(string $tag)
{
    $page = $this->pageManager->getOne($tag, null, false); // lookup is safely escaped
    $tag  = $page['tag'] ?? $tag;                           // $tag now holds the RAW stored value

    if (!$this->pageManager->isOrphaned($tag)) {            // gate: page must be linked-to
        $this->db->query(
            "DELETE FROM {$this->db->prefix('links')} WHERE to_tag = '$tag'" // unescaped
        );
    }
}

The interesting word is $tag. By the time it reaches that query it no longer comes from the request. It comes from the database:

That is the whole shape of a second-order injection in two sentences: the input is sanitized on the way in, stored faithfully, and trusted on the way out. A fuzzer firing payloads at the delete endpoint sees nothing, because the payload was never in that request. It was planted earlier, somewhere else.

So KLUE had a sink and a theory. Then it walked straight into a wall, and a well-built one. The delete code only runs the vulnerable query if the page is not orphaned: some other page has to link to it first. And linking to this particular page turned out to be hard on purpose:

This is the moment the whole post is about. A scanner that had somehow stumbled onto the sink stops here, because the obvious path is genuinely closed. Honestly, a human reviewer two hours deep might shrug and move on too. KLUE treated the closed door as a constraint to route around, not a verdict, and went hunting for any other mechanism that registers a link between two pages. It found one the grammar does not gate:

With that, the full chain assembled itself:

Then it stopped theorizing and confirmed impact, because a DELETE returns no rows and leaks nothing directly. The answer was timing:

A low-privilege account, three ordinary-looking requests, and the agent has a blind read primitive over the entire database. No single request in that sequence is suspicious. The vulnerability is the order, and the precondition (defeating the orphan check) is a puzzle you can only solve by modeling how the application behaves, not by recognizing a payload. That is the find we are proudest of, and it is the one no scanner in our benchmark would have produced.

From a template field to a shell

The second flagship was a server-side template injection, and here KLUE's value showed up as restraint as much as reach.

The application renders certain administrator-authored fields through a template engine with auto-escaping explicitly disabled. A naive analyzer sees attacker-influenced data flowing into a template render and screams "SSTI" at every one of them. KLUE did the opposite. It first ruled out the tempting-but-wrong version of the bug:

That distinction matters. Reporting attacker-data-into-template as RCE would have been a false positive, and a confident one. KLUE located the actual sink: a stored template field, editable by an administrator, rendered raw, and then reachable through a public, unauthenticated export endpoint. The sink itself is tiny:

// renders a STORED, admin-authored template  (names redacted)
public function renderFromStringNoEscape(string $template, array $data): string
{
    $wrapped = "{% autoescape false %}{$template}{% endautoescape %}"; // escaping OFF
    return $this->twig->createTemplate($wrapped)->render($data);       // arbitrary Twig
}

// call site, reachable on a PUBLIC, unauthenticated JSON-LD endpoint
$out = $this->engine->renderFromStringNoEscape($form['semantic_template'], $data);

So the trust boundary is real: a malicious or compromised admin (or anyone who can land content in that field) plants a payload once, and an anonymous request triggers it.

We validated it the conventional way. A probe of {{ 7 * 7 }} came back as 49: arithmetic evaluated server-side, the canonical SSTI confirmation. From there the template engine's reachable callables were enough to escalate from expression evaluation to operating-system command execution, and we established an interactive shell on the test host. Stored payload, public trigger, full server compromise.

The unauthenticated pair: a signature bypass and an SSRF

The next two findings lived in the same place, the application's federation inbox, and both needed no account at all.

The first is the kind of bug that is invisible unless you read return values like a lawyer. Signature verification was gated on a call to the platform's openssl_verify, used roughly as:

// ActivityPub signature check  (names redacted)
$ok = openssl_verify($signingString, $signature, $publicKeyPem, OPENSSL_ALGO_SHA256);

if (!$ok) {                       // catches 0 (invalid) and false (bad args)...
    throw new SignatureException('invalid signature');
}
// ...but NOT -1 (verify error): execution falls through to processActivity()

That looks airtight until you remember the function has three outcomes, not two:

 1   valid signature      !1   === false  -> exception NOT thrown (correct)
 0   invalid signature    !0   === true   -> exception thrown      (correct)
-1   internal error       !-1  === false  -> exception NOT thrown  (BYPASS)

-1 is truthy, so !(-1) is false, and the throw is skipped. An invalid signature that merely errors sails straight through as if it were valid. And triggering the error is easy: feed the verifier a key of one type (say DSA or EC) while it attempts an RSA digest, and on current runtime versions you get -1 instead of a clean 0. An unauthenticated attacker posts an activity to the public inbox, references an attacker-hosted key designed to provoke the error, and the verification "passes." The payoff is full create, update, and delete over the application's federated entries, no credentials required. KLUE flagged this as a high-impact integrity bug and noted, correctly, that it converts the entire authenticated write surface of that subsystem into an open one.

The second bug sat one step earlier in the very same handler, and it is almost worse for being so simple. To verify a signature you first have to fetch the signer's key, and the code fetched it from a URL taken straight out of the request before any validation happened at all:

// same handler, a few lines earlier  (names redacted)
$params = $this->parseSignatureHeader($request->headers->get('Signature'));
$actor  = $this->httpClient->get($params['keyId'])->toArray(); // fetch BEFORE any check

The keyId is attacker-controlled, so a single unauthenticated POST gets to pick the destination:

POST /api/forms/<id>/actor/inbox
Signature: keyId="http://169.254.169.254/latest/meta-data/iam/security-credentials/<role>", ...

The server dutifully issues a GET to whatever keyId points at. Point it at a cloud metadata service and the application reaches in on your behalf; point it at internal hosts and ports and it becomes a network scanner with a trusted source address. Unauthenticated, no interaction, scope-changing. Classic SSRF, reachable by anyone who can send the platform a POST.

Two XSS, and the discipline to rank them

The remaining two findings were reflected XSS, and they are a good illustration of KLUE not inflating severity to pad a report.

One reflected a revision-timestamp parameter straight into a hidden form field, unescaped:

// page-show handler  (names redacted)
$time = isset($_GET['time']) ? $_GET['time'] : '';
echo '<input type="hidden" name="time" value="' . $time . '" />';

The clever part, which KLUE worked out, is that the same parameter is also used in a database lookup, so a raw payload would normally fail to match any revision and the form would never render. It got around that with the database's lenient datetime coercion: a value that starts with a valid timestamp still resolves to the archived revision, and the trailing "><script>... is reflected anyway.

The other reflected a widget id parameter sanitized only with a tag-stripping function, which removes <tags> but does not escape double quotes:

// widget template  (names redacted)
echo 'data-formid="' . strip_tags($_GET['id']) . '"';

So an attribute-breakout payload like " onmouseover="alert(1) drops an event handler straight into the markup, executed with no authentication and no valid page or form required.

Both are real, both got CVEs, and both were rated Moderate rather than dressed up as something graver. They need a victim to click. KLUE scored them accordingly.

What it chose not to report

The reason that slate is six items and not sixteen is the part of the log we find most telling. Throughout the run KLUE kept finding plausible bugs and then declining to file them, with a stated reason each time:

This is the behavior that distinguishes a useful autonomous tester from a noisy one. Each of those is something a less careful tool would have dumped into a findings list with a medium badge, and each is something a human reviewer would then have to spend time refuting. KLUE refuted them itself, in writing, before they ever reached us. Across the whole engagement, the six issues it did escalate all held up to validation and all received CVEs. That is the precision half of the story: reach without it is just a longer queue of things to dismiss.

Why a scanner does not get here

Step back and look at the second-order SQL injection as a sequence of capabilities, because it is the clean proof of the thesis.

To find it, a tool has to: locate a sink where a database value, not a request value, lands in a query unescaped; understand that the write path's escaping is what makes the read path dangerous; discover that the obvious trigger is blocked by an input grammar; search for and find an alternate, ungated mechanism that satisfies the precondition; assemble those into a three-request chain across two different endpoints; and then, because the sink leaks nothing, fall back to a time-based blind oracle to prove impact. Every step depends on a model of how the application behaves, carried forward from the step before.

That is not a payload library. A signature engine has no rule for "these three benign requests, in this order, with this state in between, are a database read primitive." It cannot plant a row now to exploit it later, and it cannot reason that a closed door means look for a window. The bug was reachable only to something that builds and updates a theory as it goes, which is exactly what reading the trace shows KLUE doing.

We will not pretend the agent did all of it alone. KLUE found the chains, reasoned through the preconditions, and wrote the proofs. A human on our side confirmed the blind database reads and rode the template bug the last few feet into a shell. That is the honest shape of AI-assisted pentesting right now, and it is worth saying plainly: the machine does the expensive part, the part that does not scale with headcount, which is reading every path and holding a theory of the whole application in its head for three hours without getting bored or tired. People still close the loop. The closing is just the cheap part now.

None of this retires deterministic scanners either. For recognizable, first-order issues in continuous integration they are fast, repeatable, and auditable in a way model-driven reasoning is not yet, and reasoning-based testing carries its own open questions around run-to-run consistency and cost. We would rather name those than bury them. But on this engagement, in 163 minutes, one agent found and chained the kind of stateful, multi-step, intent-level vulnerabilities that define real penetration testing, and it left behind a record of its thinking you can check line by line. The clock stopped it. Nothing else did.

Want a run like this against your own codebase? Book a time-boxed engagement at shellvoide.com/book, or reach us at info@shellvoide.com.