Nvidia unveiled Cosmos 3 on June 1, 2026, an open AI world model for robots and autonomous vehicles to understand and predict physical environments. The company trained Cosmos 3 on 20 trillion tokens of multimodal data, including 400 million real and synthetic videos. Nvidia released two versions: a 'super' model for high physics accuracy and a 'nano' model generating results in fractions of a second. Initial partners include Agile Robots, Black Forest Labs and Runway.
Cosmos 3 world model launch
- ▪Nvidia unveiled Cosmos 3 on June 1, 2026
- ▪Cosmos 3 is an open AI world model designed to help robots, autonomous vehicles and other physical systems better understand and predict real-world environments
- ▪Nvidia said an 'edge' model of Cosmos 3 that can run locally is coming soon
- ▪Nvidia is releasing two versions of Cosmos 3 immediately: a 'super' model for tasks requiring high physics accuracy and a 'nano' model that can generate results in fractions of a second
Multimodal training data scale
- ▪Cosmos 3's training data included ambient audio, text and action data from humans and robots
- ▪Cosmos 3's training data included nearly a billion images
- ▪Cosmos 3's training data included 400 million real and synthetic videos
- ▪Nvidia trained Cosmos 3 on 20 trillion tokens of multimodal data
Autonomous action generation capabilities
- ▪Cosmos 3 is meant to model how machines move, not just how scenes look, according to Ming-Yu Liu, VP of Nvidia's Cosmos Lab
- ▪Developers can use Cosmos 3 to simulate actions in physical environments, then build task-specific models for robots and other machines on top of it
- ▪Cosmos 3 is designed to generate action data such as robot joint angles, gripper positions and trajectories that can help train machines to navigate and manipulate the physical world
- ▪Nvidia says Cosmos 3 can generate rare or dangerous scenarios such as robot collisions or unusual road events that are difficult, expensive or unsafe to capture repeatedly
Open model strategy
- ▪Nvidia is establishing a coalition of companies supporting Cosmos 3
- ▪Cosmos 3 is an open model, similar to Nvidia's early Nemotron family
- ▪Hot startups in the world model area include Fei-Fei Li's World Labs and Yann LeCun's AMI Labs
- ▪Cosmos 3's open model design makes it easier for hardware makers to customize Cosmos 3 to their needs and ensure that future versions more closely align to the needs of the industry, according to Ming-Yu Liu
- ▪Initial partners supporting Cosmos 3 include Agile Robots, Black Forest Labs and Runway
Physical AI applications
- ▪Cosmos 3's 'super' model is designed for tasks requiring high physics accuracy, such as training robots and autonomous vehicles
- ▪Nvidia is continuing its move beyond chips into AI models and software, positioning itself to become a foundational platform for physical AI development
- ▪Ming-Yu Liu said a world model wants to help physical agents become more generalizable by understanding the world and how it works so they can make a plan
Story comments
Loading comments…