Research
Ilya Kabanov

Capability without security: measuring the functionality-security gap in AI-generated code

Google reported that 75% of its new code is now AI-generated and reviewed by an engineer. The Weather Report measured how secure the AI-generated code is and how security measures like model-driven threat modeling and code security review actually improve it.

I recently wrote that despite 14 existing benchmarks and 30+ papers on AI code security, the industry lacks longitudinal data on how agentic code security is actually evolving across new releases of frontier models and agentic coding harnesses. Therefore, instead of introducing another benchmark with no prior-generation baseline, we decided to rerun two published ones on the current frontier cohort.

Highlights:

  • Two existing benchmarks: CyberSecEval (snippet-level) and SusVibes (real repository tasks).
  • Measured Python code functionality and security on Claude Opus 4.8, GPT-5.5, and Gemini 3.1 Pro and 3.5 Flash in their native CLIs.
  • Working code is produced 83% to 95% of the time. The security of that code has increased four to five times over the last year, but still reaches only 24% to 36%.
  • Three main gaps: security-attention, recognition-to-action, and execution.
  • A pre-coding threat-modeling turn lifts secure-and-functional output to 43% to 49%, and a post-hoc security review reaches 47% to 56%.
  • The token cost of those security measures can go up to about five times the cost of writing code alone.
Threat modeling and security review improve code security. Source: The Weather Report.
Threat modeling and security review improve code security. Source: The Weather Report.

Key takeaways:

  1. Frontier models and their agentic coding CLIs made a big jump in coding and security. Code security has improved four to five times in a generation, but you still cannot put their code into production without a security review.
  2. Being the better coder does not make a model write more secure code. Functional capability is a weak predictor of security.
  3. A generic "write secure code" reminder does little for frontier models, and effort-maxing yields no meaningful security gains.
  4. Adding a threat-modeling turn and a security review lifts the secure-and-functional rate by roughly 20 to 30 percentage points. But there is no free lunch, so get ready to burn tokens.
  5. We don't need yet another benchmark. We need a continuously maintained one to really know whether security is keeping up.

Sources:

  1. Capability without security: measuring the functionality-security gap in AI-generated code, The Weather Report (PDF)
  2. Cloud Next '26: Momentum and Innovation at Google Scale, Sundar Pichai, Google
  3. Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models, Bhatt et al.
  4. Is Vibe Coding Safe? Benchmarking Vulnerability of Agent-Generated Code in Real-World Tasks, Zhao et al.