Nvidia releases AI safety software it says could have stopped Hugging Face hack

Shell uses hardware features on Nvidia's central processor chips to contain agents. Nvidia said it is also working with Arm Holdings  and Intel to ensure the system ​also works on their central processors. Nvidia is launching the tools with dozens of partners, ​including Anthropic.

Justin Boitano, vice president and general manager of enterprise computing at Nvidia, said the tools ‌would ⁠have stopped the Hugging Face attack disclosed this summer.

Read More:China says dialogue with US ‘important pathway’ for advancing AI“From what we know, this new security platform could have stopped the breach if it was being used in frontier labs for model evaluation early on,” Boitano said during a media briefing.

“We’re advancing this openly, ​and we want to ​engage everybody to ⁠work with us.”Another system called Sentry uses a separate Nvidia chip in tandem with Open. Shell to cut off a rogue agent if ​it tries to escape its container on a central processor.

The ​Nvidia tools ⁠use mathematical formulas to detect when agents are trying to use workarounds, such as when an agent might "spawn" several "sub-agents" in an attempt to circumvent efforts to block the main agent, ⁠said Ali ​Golshan, senior director of AI software at Nvidia.

"This ​is really agentic behavior that we're talking about, which is fleets of agents and how they operate together," ​Golshan said during a briefing.