Autonomous Hackbots, in depth: how LLM offensive-security agents work, what the evidence shows, and how to contain one

What an autonomous hackbot actually is

Strip away the branding and the architecture is familiar. A hackbot combines four parts: a language model that plans and interprets, a set of tools it can invoke (a shell, an HTTP client, a scanner, a code reader), a memory of what it has tried, and a control loop that keeps calling the model with the latest observation until a stopping condition. This is the ReAct pattern, reason then act, applied to a target rather than a codebase.

The model supplies two things ordinary scanners lack: it can read unstructured context, a vague bug-bounty scope, an error message, a snippet of source, and it can chain steps, using the result of one probe to decide the next. A traditional scanner runs a fixed battery of checks; an agent writes its own next check. That is the capability jump, and it is why the loop, not any single clever prompt, is the unit to reason about.

The honest framing is dual-use. The identical loop serves a defender running an authorized test and an attacker running an unauthorized one. The code in this article is written for the first case and is deliberately limited to the harness that bounds the agent. Nothing here is an exploit.

What the public evidence does and does not show

Four results are cited constantly, so it is worth stating what each actually established. In 2024, Fang and colleagues ("LLM Agents can Autonomously Exploit One-day Vulnerabilities") reported that a GPT-4-based ReAct agent exploited 87% of a 15-item set of one-day vulnerabilities when it was handed the CVE description, versus 0% for the other models and for off-the-shelf scanners they tested. The number that gets dropped in the retelling is the other one: without the CVE description, the same agent exploited only about 7%. The agent was good at operationalizing a known, described flaw, not at discovering unknown ones.

In mid-2025, an autonomous system called XBOW reached the top of HackerOne's United States leaderboard for a period, the first time an automated system ranked at the top among human researchers on a public bug-bounty platform. Treat it as a milestone in volume and triage of relatively standard web findings, and remember that leaderboard submissions still pass through human validation and vendor response. In November 2024, Google's Big Sleep agent was reported to have found a previously unknown, exploitable flaw in a development branch of SQLite, an early example of an agent surfacing a real zero-day before release. And in 2025, DARPA's AI Cyber Challenge concluded, with Team Atlanta's cyber-reasoning system taking first place and a four-million-dollar prize for automatically finding and patching vulnerabilities at scale.

Read together, these say the capability is real and improving, concentrated today in known-flaw exploitation and high-volume triage, and still coupled to human validation at the edges. They do not show a general-purpose autonomous attacker that reliably discovers novel critical bugs in hardened targets unaided. Plan for the trend, not for the hype, and do not quote a single headline percentage without its scope.

Where they are strong and where they break

Hackbots are strongest where the work is broad, shallow, and well-documented: enumerating a large attack surface, recognizing a known misconfiguration, adapting a public proof-of-concept to a slightly different target, and writing a clear report. They are patient and parallel in a way humans are not, which is exactly why they top a volume-based leaderboard.

They break on the things that make the loop expensive or ambiguous. Long, stateful exploit chains drift: the agent forgets a constraint it established twenty steps earlier. Novel logic bugs that need a mental model of the business, not a signature, are still hard. They hallucinate success, reporting a vulnerability from a misread 200 response, which is why human validation remains in the loop. And every step costs tokens and wall-clock time, so an unbounded agent can burn a large budget wandering. For the defender, those weaknesses are also the levers: raise the cost and ambiguity of each step and the economics turn against a hostile agent.

The containment harness

If you run one of these for an authorized test, the agent is the least trustworthy component in your system, so it must act through a boundary you control. Four controls do most of the work: a scope enforcer that refuses any target outside the engagement, a tool allowlist that denies anything not explicitly permitted, a budget that caps tokens, time, and request rate, and an append-only audit log that records every action before it happens. A kill switch ties them together: on any breach, revoke the agent's credentials and stop the loop.

The sketch below is the gate every tool call passes through. It is defensive by construction, it decides whether an action is permitted and records it, and it contains no attack logic of its own.

import time, ipaddress, logging
from urllib.parse import urlparse

audit = logging.getLogger("engagement.audit")  # ship to append-only, write-once storage

class OutOfScope(Exception): ...
class BudgetExceeded(Exception): ...

class Harness:
    def __init__(self, allowed_hosts, allowed_tools, max_requests, max_seconds, token_budget):
        self.allowed_hosts = set(allowed_hosts)      # exact hosts in the signed scope
        self.allowed_tools = set(allowed_tools)      # e.g. {"http_get", "read_file"}
        self.max_requests = max_requests
        self.deadline = time.time() + max_seconds
        self.token_budget = token_budget
        self.requests = 0
        self.tokens = 0
        self.stopped = False

    def _check_budget(self, tokens_used):
        self.tokens += tokens_used
        if self.stopped:
            raise BudgetExceeded("kill switch engaged")
        if time.time() > self.deadline:
            raise BudgetExceeded("time budget exhausted")
        if self.tokens > self.token_budget:
            raise BudgetExceeded("token budget exhausted")

    def _check_scope(self, url):
        host = urlparse(url).hostname or ""
        if host not in self.allowed_hosts:
            raise OutOfScope(host)                   # fail closed: unknown host is denied
        # also deny internal ranges even if a host resolves there (SSRF guard)
        try:
            if ipaddress.ip_address(host).is_private:
                raise OutOfScope(host)
        except ValueError:
            pass

    def guard(self, tool, url, tokens_used):
        """Called before EVERY tool invocation. Returns None or raises."""
        audit.info("attempt tool=%s url=%s tokens=%d", tool, url, self.tokens)
        self._check_budget(tokens_used)
        if tool not in self.allowed_tools:
            raise OutOfScope(f"tool {tool} not allowed")
        if url:
            self._check_scope(url)
        self.requests += 1
        if self.requests > self.max_requests:
            raise BudgetExceeded("request budget exhausted")
        audit.info("allow tool=%s url=%s req=%d", tool, url, self.requests)

    def kill(self, reason):
        self.stopped = True
        audit.warning("KILL reason=%s", reason)      # then revoke the agent's credentials

Three properties make this trustworthy. It fails closed: an unrecognized host or tool is denied, not allowed. It logs the attempt before the decision, so even a denied action leaves a trace. And the budget is checked on every call, so a looping agent cannot outrun its cap. In production you would add idempotency on the audit sink, per-target rate limiting, and run the agent itself inside a sandbox such as the one in LLM sandboxing with network egress pinned to the scope, as in egress filtering.

Running an authorized engagement

The harness is necessary but not sufficient; the engagement around it is what keeps the work lawful and useful. Start from a written, signed scope that names exact hosts, accounts, and time windows, and encode that same scope as data the harness reads, so the boundary in the contract and the boundary in the code are the same object. Testing anything you are not explicitly authorized to test is not a configuration slip; it is potentially a crime, and "the agent did it" is not a defense.

Give the agent its own throwaway identity, never a human's credentials, so revocation is a single clean action. Run against a staging mirror first, where a mistake costs nothing, and only then against production with a tighter budget. Keep a human on the kill switch and review the audit log in near real time; the agent's self-reported success is a lead to verify, not a finding. When it is done, the artifact a defender keeps is the audit trail plus validated findings, routed into the same tracker as any other security work. The append-only log is also what lets you reconstruct what happened if the agent misbehaves; treat it with the rigor described in LLM audit logging.

Defending against hostile hackbots

Flip the lens and the agent is the threat. The good news is that the defenses are the ones you already owe your attack surface, now with more urgency because the probing is cheap and continuous. Reduce surface: fewer exposed endpoints, fewer stale subdomains, fewer forgotten test hosts are fewer places for an agent to enumerate. Patch one-day flaws fast, because the clearest demonstrated capability is operationalizing a described CVE, and the window between disclosure and automated exploitation is shrinking.

On detection, autonomous agents leave a signature humans rarely do: steady, high-volume, broad-but-shallow probing that does not fatigue. Rate limiting, anomaly detection on request patterns, and tar-pitting raise the per-step cost that agents are sensitive to. Assume the attacker also has an agent reading your error messages, so return less in them. And govern your own tool-using agents so they cannot be turned into an inside attacker, the tool-abuse and confused-deputy risks in tool abuse apply directly. The economics favor the defender who makes each step expensive and ambiguous, and punish the one who leaves cheap, well-documented holes.

Failure modes

The failures cluster. Scope leakage is the worst: an agent follows a redirect or a discovered link to a host outside the engagement. The harness must re-check scope on every request, not just the seed URL, and treat a redirect target as a new, untrusted URL. Budget runaway is next: without a hard token and time cap, a confused agent loops expensively; cap on every call and alert well before the ceiling. Hallucinated findings waste responder time and erode trust; require evidence, a captured request and response, for every claimed vulnerability. Audit gaps, logging after the action instead of before, mean the one action you most need to explain is the one that is missing; log the attempt first. And credential blast radius, giving the agent a human's standing access, turns a bug into a breach; use a scoped, disposable identity so the kill switch is total.

What to do next

• Write the engagement scope as signed text first, then encode the exact same hosts, accounts, and time window as the data your harness loads, so contract and code share one boundary.
• Put every tool call behind a gate that fails closed on unknown hosts and tools, checks a token, time, and request budget on every invocation, and logs the attempt before the decision.
• Re-check scope on every request, including redirect and discovered-link targets, and deny private IP ranges to block SSRF-style pivots.
• Give the agent a disposable identity, never a human's credentials, and wire a kill switch that revokes it and halts the loop in one action.
• Require a captured request-and-response as evidence for every finding; treat the agent's self-reported success as a lead to validate, not a result.
• As a defender, shrink your attack surface, patch one-day CVEs fast, and add rate limiting and anomaly detection tuned for steady, high-volume, non-fatiguing probing.