Nvidia announced today that it is releasing a family of foundational AI models called Cosmos that can be used to train humanoids, industrial robots and self-driving cars. While language models learn how to generate text by training on large volumes of books, articles and social media posts, Cosmos is designed to generate images and 3D models of the physical world.
During a keynote speech at the annual CES conference in Las Vegas, Nvidia CEO Jensen Huang showed examples of Cosmos being used to simulate activities inside warehouses. The universe was trained on 20 million hours of real footage of “people walking, waving, manipulating things,” Jensen said. “It’s not about producing creative content, but teaching AI to understand the physical world.”
Researchers and startups hope that these kinds of basic models can happen Give robots used in factories and homes More advanced capabilities. Cosmos, for example, can generate realistic video footage of boxes falling from shelves inside a warehouse, which can be used to train a robot to recognize accidents. Users can also fine-tune the models using their own data.
Nvidia says many companies are already using Cosmos, including humanoid robot startups Agility and Figure AI, as well as self-driving car companies like Uber, Wabi and Wave.
Nvidia also announced software designed to help different types of robots learn to perform new tasks more efficiently. The new feature is part of Nvidia’s existing Isaac robot simulation platform that will allow robot builders to take many examples of a demanding task, such as understanding a specific object, and generating large amounts of synthetic training data.
Nvidia hopes Cosmos and Isaac will appeal to companies looking to build and use humanoid robots. Jensen was joined on stage at CES by life-size images of 14 different humanoid robots developed by companies including Tesla, Boston Dynamics, Agility, and Figure.
