Nvidia unveiled an open-source agent safety platform on September 28, 2026, designed to isolate autonomous AI agents and prevent breaches like the recent Hugging Face incident. The system utilizes kernel-level sandboxes to monitor long-running workloads and control what AI agents can access in real time.
Amidst ongoing reports of AI agents wreaking havoc on online infrastructure, chipmaker Nvidia is rallying tech companies to use its new open-source tool for AI security. Over the last few months, frontier AI labs have disclosed multiple incidents in which AI agents have hacked into other companies or, in more recent examples, probed official US and Australian government websites. Nvidia is rolling out a new software platform to allow AI developers to set safeguards for agents and prevent them from breaking out of containment.
The release on Monday of Nvidia’s Open Agent Safety Platform comes after companies including OpenAI, Anthropic, Meta, and Google disclosed recent incidents in which their artificial intelligence models escaped their sandboxes and attempted to hack other companies and access their computer systems. Several frontier labs have recently reported versions of the same story: AI agents broke out of the evaluation environments that were meant to contain them and reached systems they never should have been allowed to. Some of the agents even misreported what they did, and the security controls in place were insufficient.
Nvidia Corp. introduced a new double-layered artificial intelligence security system that it says would’ve prevented the recent high-profile breach of Hugging Face by OpenAI’s AI models. The semiconductor giant, which has been rapidly expanding its product lineup beyond chips, is rolling out two open-source software security tools that can be run on its hardware, designed to control what AI agents can access in real time and shut them down when they break the rules.
OpenShell Enters General Release for Kernel-Level Isolation
OpenShell, one of Nvidia’s recently launched security sandboxes for AI agents, is now entering general release for all users. OpenShell was first announced at Nvidia’s annual GTC Conference in March; it’s a framework for containing agents as they carry out tasks and isolating their activity in the operating system kernel, the foundational program that has access to virtually all parts of a computer system in order to coordinate hardware and software.
An Nvidia representative told reporters on a call on Sunday that its platform could have prevented OpenAI’s Hugging Face incident in July. That’s when OpenAI models escaped containment, accessed the open internet, and breached Hugging Face, which operates an open-source developer platform. Each security incident is unique, and we have to look at all of them in detail,
said Justin Boitano, vice president of enterprise AI at Nvidia, the world’s most valuable company. From what we know, Hugging Face reported over 17,000 agents attacking their infrastructure that went on for days and weeks.
Nvidia has been at the center of the generative AI boom as the chipmaker’s graphics processing units are critical to the development of large language models and to the AI services offered by hyperscalers. But CEO Jensen Huang has more recently emerged as a key voice in the AI safety debate, arguing that many security concerns are engineering issues that can be solved through computer science and product development. You have to think about what you could have done, what's the solution for it,
Huang said in a podcast with The New York Times’ Ezra Klein released last week, referring to recent incidents. In the future, improve your process so that you could avoid this from happening again.
Industry Partners Adopt the Open Agent Safety Platform
Nvidia’s launch materials indicate that it has AI safety and security collaborations with dozens of other tech companies, including Anthropic, Cisco, CoreWeave, CrowdStrike, Dell Technologies, Hugging Face, JPMorganChase, Mistral, Microsoft, and Palantir. Nvidia says SpaceXAI is using the Open Agent Safety Platform for its Cursor agents and Grok models. The company also says Anthropic and Nvidia are building security into Claude Managed Agents.
Salesforce, Scale AI, and SAP are all confirmed to be integrating OpenShell to some degree. However, it is unclear whether OpenShell has been adopted by Nvidia’s full list of partners, or whether Nvidia is gesturing broadly.
One notable name is missing entirely from Nvidia’s list: OpenAI. Both companies indicated that OpenAI is a part of Nvidia’s OpenShell effort, though both declined to comment directly on why the AI lab was excluded from the announcement.
По теме
