Under the hood of Pliny's T3MP3ST

TL;DR: Anything Pliny ships is worth studying, so I used T3MP3ST to learn what makes an offensive agent effective: a capable model, tight context management, long-term memory, a real execution environment, and an oracle to verify hits.

Pliny, known for his jailbreaks of frontier models, shipped T3MP3ST, a roughly 34,000-line offensive framework that got more than 4,600 stars in the last couple of weeks. They claim to beat XBOW on its own benchmark, so I looked under the hood to understand what differentiates offensive harnesses and what gives a competitive edge.

There's no shortage of offensive frameworks. I found at least eight that are currently maintained.

What is T3MP3ST?

T3MP3ST is an autonomous offensive-security framework that can pen-test a web app, solve a CTF challenge, or find vulnerabilities in open-source repositories. You supply the model via an API key, a CLI, or a local model.

Let's unpack how the web-pentesting arm works.

1. Pointed at a web app, it walks the seven-phase Lockheed Martin kill chain: Reconnaissance, Weaponization, Delivery, Exploitation, Installation, Command-and-Control, and Actions-on-Objectives. Each phase auto-spawns one specialized agent: a recon operator, a vulnerability scanner, an exploiter covering both delivery and exploitation, an infiltrator for installation and lateral movement, a ghost for command-and-control, and an analyst that writes up findings. The agents differ mainly by their system prompt and allow-listed tools, 35 built-in or 102 with external adapters like nmap, sqlmap, metasploit, and hydra.
2. Recon seeds four fixed tasks: DNS enumeration, port scanning, web probing, content discovery. Each task is one agent run of up to 15 turns, up to three agents in parallel, so about 60 model-turns per phase. The result is a single target object holding the URL's services, plus raw findings in a vault.
3. Weaponization seeds three fixed tasks: automated vulnerability scan, web application security testing, network service vulnerability assessment. Each task is one scanner agent run of up to 15 turns, up to three agents in parallel, so about 45 model-turns per phase. Each agent reads the recon-enriched target object, then ends in a JSON findings block that fills in its list of vulnerabilities.
4. Delivery seeds a single task, exploit the confirmed vulnerabilities, run by one exploiter agent for up to 15 turns, no parallelism. It reads the same target object but can only send stateless HTTP requests, with no shell or sandbox to run an exploit. Its findings land there too, and nothing verifies the exploit worked: success just means the agent wrote a finding.
5. After skipping the installation and command-and-control phases, which spawn an infiltrator and ghost that never run, the actions phase spawns an analyst to analyze the accumulated findings, validate severity, identify attack chains, and write the report. One agent, up to 15 turns, no parallelism. The user-facing report is generated by a separate module that reads the vault.

This pentesting arm wasn't benchmarked. The published XBEN result, 90.1% on 104 web challenges, came from a single agent, not the multi-operator kill chain the framework is built around.

What makes an agentic pentesting framework do the job, expand target coverage, and stay cost-efficient?

1. Raw model capability still matters. A frontier model without cyberguardrails, plugged into an offensive framework, can provide a significant advantage. Pliny swapped GLM-5.2 for gpt-5.5 and scored 92 versus 82 on XBEN.
2. Context management feeds each agent only what its current step needs, held in a token-efficient working memory: the target details, the confirmed findings it can build on, and a tried-and-ruled-out ledger that keeps it from re-running dead vectors. The tension: the longer a run goes, the more context grows, driving up token cost and, more importantly, diluting the model's attention.
3. Long-term memory lets the system evolve by distilling lessons from past runs so it does not relearn every target from zero. Google's Co-RedTeam proposes three kinds. Vulnerability patterns record how a symptom became a confirmed bug, along with the false leads that wasted time. Strategies capture exploitation workflows that transfer across targets, successes and dead ends alike. Technical actions save the exact commands that worked or failed as reusable snippets, such as a one-liner for testing SSRF reachability.
4. Execution environment gives the agent a shell and sandbox that hold state across turns, so it can stage a payload, keep a session, and chain one request into the next. Stateless HTTP requests alone cap it at whatever a single blind request can reach.
5. Oracle proves impact from the target's own behavior, with evidence the agent cannot fabricate. It can be an out-of-band callback from the target's IP, a nonce an injected command echoes back, a response body that actually returns /etc/passwd, or a timing delta against a control probe. An agent simply reporting success is not enough.

Sources:

1. T3MP3ST offensive-security framework (elder-plinius) (Last accessed 07-13-2026)
2. XBEN validation benchmarks (XBOW)
3. Cybench CTF benchmark (andyzorigin)
4. PentAGI (vxcontrol)
5. Strix (usestrix)
6. CAI (aliasrobotics)
7. Pentest-Swarm-AI (Armur-Ai)
8. Decepticon (PurpleAILAB)
9. pentestagent (GH05TCREW)
10. PentestGPT (GreyDGL)
11. hackingBuddyGPT (ipa-lab)
12. Co-RedTeam: Orchestrated Security Discovery and Exploitation with LLM Agents (Google, 2026)