New NVIDIA Cosmos 3 Open Models for Interactive World Simulation

Physical AI needs worlds that react. NVIDIA's Cosmos platform just added four open-weight Cosmos 3 Nano post-trains aimed at interactive simulation: Transfer-Auto for multi-view driving with weather and lighting control, Sim-Auto for streaming camera and LiDAR, Sim-Transfer for turning simulator controls into photoreal video, and Sim-Bimanual for action-conditioned robot video.

Each has 16 billion trainable parameters and ships under OpenMDW 1.1 for commercial and non-commercial use. Recipes cover bidirectional adaptation, teacher forcing, consistency distillation, and self-forcing so errors in generated history get trained against.

This is infrastructure for people building driving and robot policies who cannot film every edge case on real roads and factory floors.

We're constructive on open world models that researchers can actually fine-tune. Closed sim stacks lock progress inside a few labs. Cosmos 3's bet is complementary checkpoints: offline multi-sensor driving data, fast causal streaming for closed-loop rollouts, bridging classical physics simulators to RGB, and two-arm robot interaction for YAM and AgiBot-style embodiments.

The technical shape is a Mixture-of-Transformers backbone linking text and visual generators, predicting short chunks from prompts, controls, and history, with distillation down to a handful of denoising steps for streaming. Self-forcing matters because a slightly wrong object becomes tomorrow's context — training has to feel that drift. NVIDIA is explicit that fidelity for policy learning is still an open measurement problem, not a solved product.

I want this used to stress-test autonomy and manipulation before metal moves. The downside if synthetic comfort outruns validation: teams ship policies that look great in Cosmos rollouts and fail on the first odd wet night or novel gripper slip. Open weights help the field compare; they do not excuse skipping real-world held-out eval. Useful physical AI will mix these generators with stubborn measurement — ToyBlocks energy, not just prettier video.

Cosmos sits inside NVIDIA's broader push to supply the simulation and synthetic-data layer under robotics and AV stacks. Hugging Face hosting and open skills lower the setup cost for teams that already live in that ecosystem.