When it comes to understanding the OpenAI agent attacks on Hugging Face, and the subsequent disclosures from other models that had the same rogue agent problems, Rich Mogull, chief analyst of the Cloud Security Alliance, is a sure bet to help break through the hype. Mogull sat down with Dark Reading’s Becky Bracken at the News Desk to unpack the details surrounding the AI agent escapes, and critically, what it’s going to take for cyber defenders to win in this new goal-seeking AI agent world.
Mogull digs into specific strategies aimed at stopping rogue offensive AI attacks, and how defenders can start to implement them today. He also took time to explain the distinction between frontier, proprietary, and open-weight AI models, and the national security and international business implications of rising open-weight Chinese models. If US policy bans emerging models from China, will the US fall behind in research? It’s an ongoing debate. The CSA, Mogull stresses, recommends including these open-weight models in any incident-response plan.
These new “industrial accidents,” as Mogull characterizes this new flavor of offensive AI agent cyberattack, are here to stay. What happens next is still an open question.
For all of our Dark Reading News Desk videos, please check out our YouTube channel, and our curated video articles.
Dark Reading News Desk With Becky Bracken & Rich Mogull: Full Transcript
This transcript has been edited for clarity, readability, and length by Informa TechTarget’s internal AI assistant and human editors. For the full experience, please watch the video.
Dark Reading’s Becky Bracken: Hello, everybody, and welcome to the Dark Reading News Desk. We are coming to you here from Black Hat USA 2026. My name is Becky Bracken. I’m a senior editor with Dark Reading and I am very enthusiastic about our next guest, Rich Mogull from the Cloud Security Alliance. He and I just had a recent conversation on our Dark Reading Confidential podcast about the OpenAI Hugging Face debacle. And now he’s back, so we can continue that conversation and really talk about the developments that have happened since. Welcome.
Rich Mogull: Thank you. Thanks for having me.
DR’s Becky Bracken: Thank you so much for being here. OK. So when we last talked, we thought it was just OpenAI. We thought it was just Hugging Face, but things sort of exploded since then. Yes?
Rich Mogull: I don’t even know if I can keep up with the developments. I was getting news reports today of other new ones. So, when we talked last, we knew about OpenAI/Hugging Face, and we had a lot of information from Hugging Face that had been released. OpenAI yesterday told their side of the story.
But in that intermittent time period, Anthropic said that they had three situations that were pretty similar. And then we had the UK’s AI Cybersecurity Institute in there testing all the models, which basically cheated on the tests. I think I read that Meta just had the similar issue, but I think Meta, maybe they just want the attention.
DR’s Becky Bracken: You think so?
Rich Mogull: Yeah.
DR’s Becky Bracken: OK. What do you make of all of this? What am I looking at, really, at this point in time, from a cyber defender perspective?
Rich Mogull: I think we’ve seen an industry that really wants themselves to be regulated and controlled because they are not doing fundamentals in terms of safety around these incredibly powerful tools. Now, we’ve been really lucky so far. We haven’t had certain kinds of destructive actions: they’re trying to cheat on tests, they’re not trying to take down systems or anything along those lines. But, they need better safety protocols.
Apparently [the cheating] is endemic. And that’s hard because these are pretty powerful and unpredictable technologies. But still you got to do your work. I’m classifying these right now as industrial accidents.
DR’s Becky Bracken: I noticed that, and tell me tell me about why that is. Because that’s a really interesting point, I think.
Rich Mogull: Well, we have a lot of safety regulations because the laws are written in blood. In a lot of our industries there are long histories of people not paying attention, not following safety protocols, not even having safety protocols. And as a result, innocent bystanders get hurt and employees get hurt. We’re really seeing a similar situation here. And I don’t think it’s something we’ve had to deal with in technology in the same way before.
If Microsoft or Amazon goes down when they have outages, that’s a power outage or something else. An industrial accident is where you have a blast radius and where you are affecting others around you. And that’s what we’re seeing here.
DR’s Becky Bracken: And there has been a lot of criticism about OpenAI in particular, and their lack of security controls. So let’s talk about what good would look like, in these instances with these models.
Rich Mogull: If you’re going to go gloves-off with the models, you need real sandboxing, and you also need to have much more in-depth monitoring than some of the things that they talked about in that OpenAI session. I know good people working over there who are top-notch security people. And it’s time to take a step back. Let’s put in safety protocols, different levels of monitoring, different levels of sandboxing, now that we know what these are capable of.
It’s just fascinating from the creative side. [OpenAI’s rogue model] was inventing its own languages, and doing dead drops, and moving files. OpenAI shut that down, and then they were using directory names to communicate with each other. It is just wild what the agents were doing.
DR’s Becky Bracken: I think that is an important point to make, that they’re creative. They’re not just following a binary path.
Rich Mogull: No. But it’s not creative like a human, because I do not want to anthropomorphize. I think way too many people do that.
DR’s Becky Bracken: And I take that to heart. You said that during the podcast, and I catch myself.
Rich Mogull: But it’s the million monkeys at a typewriter. It will try everything it can, and then once it finds something that works, it’s going to redirect to there.
From what we know from Anthropic, where some of that testing was occurring at a third party, where they were supposedly responsible for making sure that there was a sandbox and there wasn’t … we’re waiting to hear the news, the actual stories, on that side of it. But we now know that there’s a risk. We now know that these things can expand beyond what people think. And someone’s got to hit the big red button in terms of this kind of testing until we can say, Hey, we I think we do have the tools to be able to test those safely.
[Models] are not magic. They’re not yet inventing new vulnerability classes. They will hammer away until they find a solution to a problem. For alignment purposes, they’re going to try and do what the humans have directed them to do, there’s no consciousness or anything. It’s just it’s a different kind of algorithm and such.
DR’s Becky Bracken: So let’s take a few minutes to talk about, what is this ongoing friction between weighted models frontier. What is the issue?
Rich Mogull: So we have foundation models, frontier models, open-weight models. So the foundation models are generally from the big providers (Claude and and OpenAI have foundation models; so do Amazon and Microsoft). And it may not be the latest and greatest, but it’s solid.
DR’s Becky Bracken: Reliable kind of tried and true.
Rich Mogull: Well they’re sort of deterministic. They’re not all reliable.
A frontier model becomes a foundation model as it matures. The frontier models are the ones pushing the edge, so some of the testing models that OpenAI talked about. The model was given impossible problems because that’s a very legit way from a scientific standpoint to see what it can actually do. It’s the Kobayashi Maru [Starfleet Academy training exercise] for the Star Trek fans.
DR’s Becky Bracken: There we go. OK.
Rich Mogull: And then we have open-weight models.
So frontier models run in OpenAI, Anthropic, Google — their data centers. Open-weight models can technically be run anywhere. So the model itself is released and you get to adjust some of the weighting on it. That’s where that term “open-weight” comes in. These are becoming very powerful models. And in some cases are so big you can’t effectively run it. You’re not running that on your Mac mini at home, no matter how much RAM you have. But you can run it in Hugging Face or Amazon or wherever if you want to pay.
Of course you’re going to have to pay for that infrastructure. And then of course they have the versions of it where you treat it as if it’s a foundation or frontier model as well. So here’s the debate. There’s very few open-weight models coming out of the US or Europe. Most of these big, powerful open-weight models are coming out of China. If you can run it yourself, you can do retraining and fine-tuning and stuff, and effectively disable all the security guardrails that are built into the model itself. A lot of what the frontier models do, a lot of the safety features that they have in place are in the harness, it’s not necessarily in the model [where they can be disabled].
Now, there’s a business side of this and an international economics side of this that goes well beyond those of us in cybersecurity, which is, if those open-weight models become as capable of ours, we can run them cheaper on lower-level hardware. Then it becomes a financial challenge to the frontier-model providers. So it’s an arms race.
Now, the next question is, why did we at CSA with our with our partners recommend including open-weight models into incident-response processes?
DR’s Becky Bracken: That was my next question.
Rich Mogull: So that is an issue because of what we just saw, which is the refusal of the safety guardrails. So the major model providers do have cyber programs and they’re called slightly different things. You can get additional verification on both Anthropic and on OpenAI. I use both of those, but I’m still not at the highest level like Mythos, where the models have reduced safeties to allow you to do things like reverse-engineer malware.
I don’t know what level Hugging Face had in terms of their [open-weight model] access when they went to use it for incident response. They got refusals from [frontier models] on what they were trying to do. I have research, friends. You can do one thing one day and get a refusal the other day.
So if I have an open-weight model I’ve trained, I know that I am controlling the refusals and that it will behave predictably even if it’s not the latest-greatest. I actually don’t care if it’s an open weight model. When I’m teaching incident responders, it’s more about having consistent behavior, knowing what the limits and capabilities of that are.
DR’s Becky Bracken: So you can have a test with the same controls.
Rich Mogull: Yes. That matters to me than if it’s open-weight or frontier, or how proprietary it is. I need consistent behavior and that I can use it to get the job done.
DR’s Becky Bracken: Thank you so much again. Once again, you’ve made me smarter in a very short amount of time. So thank you so much for stopping by and sharing your expertise on this subject.
Rich Mogull: Thanks. I love talking about this stuff, so thanks for having me again.
