Top 5 Insights from [un]prompted II, Day 1
In March, [un]prompted was about what AI could do. At [un]prompted II, it is about what happens when it does. Here are the five insights from day one that matter.
1. Containment and the ability to respond fast is now critical. (Caleb Sima and Alec Eckmann, Hadrian)
Caleb secretly brought in an AI-agent red team vendor for what was supposed to be an external-only test. The agents got in anyway and spread from the AI gateway to Okta, GitHub, Snowflake, and 29 client secrets, and the team had to shut the gateway down. When Caleb asked the vendor to kill it, "they said, 'We did kill it,' and we're like, 'No, you didn't.'"
Caleb's conclusion: "containment and the ability to respond is very, very key here and how to do this in very quick and automated ways." And there will be more of this: "a lot of consulting services are getting access to Daybreak Red and Mythos. How you actually control that is a whole different ballgame."
2. AI can now fix vulnerabilities, but the exposure time doesn't go down. (David Molnar, Meta)
At Meta, more than 80% of new code now comes from AI coding agents. By July, AI was writing the fixes for 40% of critical vulnerabilities, and almost half of those fixes landed without a single change.
But exposure time, from finding a bug to shipping the fix, didn't go down, the same exposure debt we saw building in May. Engineers were drowning in AI-written PRs to review, fixes often went to the wrong person, and every product had its own release process. In the end, Meta cut exposure time the old-fashioned way, by getting VPs to agree on shared goals and having a human ops team chase down misrouted fixes. "Not very AI of us, but it works."
3. AI vulnerability scanners are finding serious flaws in production software, including 38 remote code execution bugs in Windows and Hyper-V. (Taesoo Kim, Microsoft)
The Windows security team has run MDASH, Microsoft's multi-agent scanner, in its development pipeline since April. It has found about 200 high-severity vulnerabilities, including 38 RCEs in Windows and Hyper-V: "we've been discovering about 200 vulnerabilities so far. ... 38 remote code executions on Windows and Hyper-V."
4. AI can find a wormable bug in an app used by 1.5 billion people for $8. (Anna Groza, Calif.io)
Calif.io's agents found a wormable WeChat bug for $8 of Opus. In April we wrote that AI was making attacks 10x cheaper; this is what that looks like in practice. As Anna put it, it's "maybe not great ... that you can worm all of them with a Claude sub." On a separate iOS bug, she gave a model a short goal, walked away, and 12 hours later had a working exploit, even with Apple's newest memory protections switched on.
5. Cryptographically signed audit logs cannot be trusted if AI agents can steal the signing key and rewrite the logs. (Daniel Cuthbert, Santander)
Daniel's team signed agents' logs to preserve integrity. Turns out the agents could chain a few sandbox bugs, grab the signing key, and sign their own altered logs. As Daniel put it, "we had no way of determining that the agents were doing something nefarious."
