Nvidia's big bet on physical AI aims for safer robotaxis, humanoid robots

On the software side, the Halos operating system was designed to enable constant monitoring of “every hardware block” and “every software library” to swiftly spot any failures, Goel explained. The system also isolates safety-critical computing workloads to avoid any potential interference.

The system also includes the Nvidia Holoscan Sensor Bridge that connects sensor data with safety-related computer processing in a way that can easily identify corrupted data. This can be embedded in individual hardware components such as a microcontroller or Field Programmable Gate Array.

Last but not least, the Halos package also includes Nvidia’s simulations for testing robots in virtual environments, and an inspection lab program that allows robotics companies and other partners to get quick feedback on any safety “artifacts” that arise during robotic development.

However, adapting Nvidia Halos from autonomous vehicles to robotics required accommodating many different definitions of safety. The definition of functional safety for autonomous driving generally remains the same across different automotive companies and countries, Goel explained. But a robotic vacuum cleaning a hallway will have very different safety considerations compared to a robotic forklift handling heavy payloads in a warehouse loading docks.

“We had to reimagine and do a lot of foundational work to build a platform that is programmable, that gives all the hooks to the developers to actually define custom safety functions, but at the same time not break the underlying infrastructure that we built,” Goel told Ars. “Because giving flexibility can come at the cost of losing some control over the stack.”

Custom safety functions are also necessary because robots may operate in complex, diverse environments.

“If you are a robot coming down an aisle at six miles per hour, your senses do not see what are the blind spots and what is coming around the corner,” Gold said. “What the robots do is almost come to a halt before the turn, because they really don’t know what’s happening in all these factories, warehouses and hospitals—they’re unstructured environments and things can come from anywhere.”