Nvidia debuts system designed to control AI agents

SECURITY: Nvidia CEO Jensen Huang has repeatedly downplayed the risk of AI slipping out of human control, saying that safety concerns are an engineering challenge

Nvidia Corp introduced a new double-layered artificial intelligence (AI) security system that it says would have prevented the recent high-profile breach of Hugging Face by OpenAI’s AI models.

The semiconductor giant, which has been rapidly expanding its product lineup beyond chips, is rolling out two open-source software security tools that can be run on its hardware. They are designed to control what AI agents can access in real time and shut them down when they break the rules.

If cutting-edge labs had been using this technology to evaluate their AI models early on, it the Hugging Face attack could have been averted, Nvidia vice president of enterprise AI Justin Boitano said during a briefing with reporters ahead of yesterday’s announcement.

People visit booths at the Al In 2026 conference in Montreal, Canada, on Sept. 16.

Photo: AFP

“From what we know, this new security platform could have stopped the breach,” he said.

Misconduct by autonomous agents, including the Hugging Face incident in July, has roiled the AI industry and led to calls to slow down work on the technology. With the new product — dubbed the Open Agent Safety Platform — Nvidia is offering a way to prevent breaches without curbing AI development.

Nvidia chief executive officer Jensen Huang (黃仁勳) has repeatedly downplayed the risk of AI slipping out of human control.

In the past few days, Huang has cast safety concerns as an engineering challenge, rather than something that requires more regulation or global coordination. His engineering solution to the AI safety problem has two parts. OpenShell, a software product that Nvidia already previewed at its hallmark technology-focused conference in March, can run on Nvidia’s Vera central processing units. It enables users to set rules for what AI agents can access and enforce them in real time. The software is open source, meaning it can be used and adapted freely.

Meanwhile, Nvidia Sentry is a new product that can run on the chipmaker’s BlueField data processing units. It is designed to provide an extra layer of AI monitoring that polices agents and intervenes to isolate any that act suspiciously, the company said.

“We believe this added security layer will allow the industry to test even the most advanced AI systems safely,” Boitano said of the Sentry product. “It can quarantine a suspicious agent in milliseconds.”

OpenAI’s recent incidents — including a breach of an Australian government system, as well as attempts to access dozens of US government and university Web sites — happened when its models escaped testing environments that were supposed to be secure.

As the problems proliferated, OpenAI on Friday said that it would pause training of its most capable AI models. In July, Anthropic PBC also disclosed that its agents broke out of what was supposed to be an isolated testing space.