There may not be a clear winner in terms of which AI model is the best or worst for coding, but there are better (and far worse) models for organizations depending on the development environment or framework one codes in.
Software governance firm Secure Code Warrior today unveiled its AI Trust Index, a body of data attempting to quantify the risk established via large language model (LLM)-powered coding tools. AI-assisted development has become exceedingly popular at the organizational level, using tools to generate code, test for vulnerabilities, and audit for general integrity. It is by no means a secret that these tools are expensive, while introducing both vulnerabilities and risk, no matter the efficiency gains.
While this does not mean organizations should stay away from AI coding tools, it does mean one should be aware of the risks and mitigations before jumping into the deep end.
To illustrate some of those risks, the AI Trust Index (introduced as a website) was built on a methodology created with RMIT University and published by Secure Code Warrior. Researchers evaluated 1,760 complete codebases generated by 16 frontier models from vendors including OpenAI, Anthropic, Google, and others.
AI-generated code contained an average of 15 confirmed vulnerabilities per codebase, including 4.3 vulnerabilities rated critical or high severity. More than 27,000 vulnerabilities were identified in total across all codebases.
No Clear AI Coding Winner
Researchers evaluated models based on a 0 to 100 trust score, with 100 meaning the fewest vulnerabilities found in testing the code they generate, relative to other models.
As far as the major vendors are concerned, there was no universal winner. Two Anthropic Claude models (Sonnet 5 and Fable 5) scored best (80.4 and 76.4 respectively), while OpenAI GPT 5 Mini ended up in last place (21.6). Overall, OpenAI, Anthropic, and Google frontier models ended up near the top and bottom of the list.
That said, it turns out that every AI model has a distinct security fingerprint, according to the report: “Across all evaluated frameworks, models consistently exhibited recurring vulnerability patterns rather than random failures. The SCW AI Trust Index shows each model produces a repeatable mix of OWASP vulnerability categories, enabling organizations to anticipate where security weaknesses are most likely to occur.”
The most important detail to note is that the strength of a model’s performance varies across frameworks; Secure Code Warrior found that a model that performs well in one framework (such as Django or React) may offer a mediocre performance in another, or even miss controls altogether. For example, Claude Opus 4.8 got a 100 Trust Index score when tested against Django, but a just garnered a 57.3 in C-Basic and a 28.5 in C#-Basic.
Ultimately, the results showed that the zoomed-out overall trust score matters less than which model is paired with which framework; 11 development frameworks were tested in all.
Researchers also found that there was little to no correlation between model usage cost and security; that the most common security failures resulted from things models forgot to do (authentication checks and input validation) rather than actively dangerous code; and that risk was overall predictive rather than random. Specifically, the model and framework combination made it more or less predictable how many (and which kind of) vulnerabilities researchers would find.
Consider Your Frameworks & Prioritize Training for High-Risk Environments
Secure Code Warrior CEO and co-founder Pieter Danhieux tells Dark Reading that, according to the data, the highest-scoring frameworks were 40 times less risky than the lowest-scoring ones. JavaScript and Java EE/JSP carried much of the exposure, while C# and Java Spring “barely registered.”
“You can’t personally audit (nor override) every team’s framework choice, but you can require that any team building in a high-risk environment have a security gate and training requirements in place before it ships, the same way you’d require extra scrutiny for any other high-risk system in your environment,” he says.
He also recommends that organizations prioritize their training investments around the actual universal situations, rather than trying to cover every situation a developer may run into.
“We found 17 CWEs that showed up in every one of the 16 models we tested. That’s too long a list to mandate all at once, so we’d tell a CISO to prioritize the five that carry the most volume: sensitive data leaking into logs, cross-site scripting, hard-coded credentials, predictable session tokens and reset codes, and path traversal,” he says. “None of these is a new, insurmountable threat vector. AppSec teams have had playbooks for all five for years. The job now is making sure those playbooks get pointed at AI-generated code with the same rigor as anything else, which is a requirement you can set from the top without writing a line of code yourself.”
