Is open source the answer to rogue AI agents? Nvidia's new alliance says yes

Follow ZDNET: Add us as a preferred source on Google.

AI agents are behind a steady string of security incidents this year. Nvidia thinks a new partnership built on open-source software is the answer.

On Monday, the company announced the Open Secure AI Alliance, which it said "will work to remediate and disclose vulnerabilities using open technologies." The announcement is something of a response to Project Glasswing , the security alliance Anthropic spearheaded around its highly capable Mythos 5 model. Nvidia's initiative aims to further democratize AI security tools by focusing on open-source software rather than reserving access to a proprietary model for a few major companies.

Also: OpenAI's attack agent did exactly what it was told - just more relentlessly than expected

Nvidia said the Alliance was spurred by last week's Hugging Face incident , in which an OpenAI agent escaped a testing environment and infiltrated Hugging Face, stealing credentials and demonstrating what OpenAI said was an "unprecedented" outcome. Nvidia argued in its announcement that because AI tools themselves have become an effective way to remedy AI attacks, open-source security tools must be a reliable public good.

"When closed AI tools -- unable to distinguish attackers from defenders -- blocked essential forensic analysis, Hugging Face ran the open-weight GLM 5.2 model on its own infrastructure to analyze more than 17,000 actions and contain the intrusion," Nvidia noted.

Cloudflare, CrowdStrike, Adobe, IBM, the Linux Foundation, Microsoft, and ex-OpenAI executive Mira Murati's startup Thinking Machines Lab are among the first participants in the Alliance. Nvidia said its contribution to the effort will include new research on agent harnesses -- the infrastructure that turns an LLM into an agent by helping it act autonomously -- as well as open models, weights, and data.

In the announcement, Nvidia specifically argued for open models (alongside closed models) as a solution to security incidents, countering the dominant narrative that open-source AI can be more easily hijacked.

Also: How AI has suddenly become much more useful to open-source developers

"Some argue that open models are inherently less safe because they can be misused for cyberattacks or modified to remove guardrails," the company said. "Those risks are real, but they do not disappear in closed systems, and simply keeping weights closed does not prevent determined attackers from seeking or exploiting powerful AI."