NVIDIA shipped Cosmos 3 today: an open physical-AI foundation model that outputs joint angles and gripper positions natively, meaning policy training no longer routes through a separate head bridging video prediction to action. The architecture is a Mixture-of-Transformers with two towers, a vision-language reasoner that interprets motion, object interactions and physical context, and a generator that produces future observations and action sequences via diffusion. Two sizes ship today: Cosmos 3 Nano (16B parameters, RTX PRO 6000-class workstation) and Cosmos 3 Super (64B, Hopper/Blackwell datacenter). An Edge variant is coming. Weights are on HuggingFace under OpenMDW 1.1, the Linux Foundation's permissive license, train, modify, redistribute.
The Mixture-of-Transformers (MoT) detail worth pausing on: information flows unidirectionally from reasoner to generator. The reasoner is an autoregressive VLM that can be called independently; the generator activates both towers. That's not a fully joint single-system architecture, it's reasoner-then-generator with the reasoner standing alone. The action head is native: numerical action data (joint angles, gripper positions) sits alongside text, image, video and ambient-sound outputs. NVIDIA reports Cosmos 3 topping VANTAGE-Bench (32B and 8B tiers), PAI-Bench, R-Bench, Physics-IQ and RoboLab among open weights. No numerical comparisons against GR00T N1 or Gemini Robotics surface in the launch materials, and the contamination story for each leaderboard isn't disclosed. Quantization options (BF16, FP8, NVFP4) and six open synthetic data generation datasets ship alongside for post-training. The technical report lives at research.nvidia.com/labs/cosmos-lab/cosmos3.
The Cosmos Coalition is the ecosystem signal: Agile Robots, Black Forest Labs, Generalist, LTX, Runway and Skild AI line up as co-builders. That tent stretches from humanoid hardware (Agile, Skild) through media generation (BFL, Runway, LTX) to a foundation-model layer (Generalist). NVIDIA's bet is that the same MoT substrate serves both worlds, synthetic data for robot policies and high-fidelity video generation for media. For builders skeptical of foundation-model overlays on top of robotic perception, people building from primitives like predictive coding, SLAM and ROS2 vision stacks, the unidirectional MoT is actually closer to your camp than the marketing suggests. Separating "interpret the scene" from "generate the next state" is the right decomposition; the reasoner being independently callable means you can use it as a perception head and bring your own controller. OpenMDW 1.1 makes that legally clean.
Monday morning, if you're training robot policies: Cosmos 3 Super is now your strongest open baseline for synthetic data generation, ahead of pure video-prediction models because the action head is native. If you're shipping inference at the edge or on workstation, Cosmos 3 Nano (16B) is the practical target, wait or watch for Edge. If you're building from primitives and don't want a foundation-model overlay, the reasoner is callable on its own under a permissive license, treat it as a perception baseline, not a black box. And before betting infrastructure on the leaderboard claims, run your own harness on a workload that matches yours: the benchmarks NVIDIA tops here are also benchmarks NVIDIA helped define.