Lyra 2.0

What It Does

Lyra 2.0 is NVIDIA's advanced system for generating persistent, explorable 3D worlds at scale. It combines cutting-edge video generation with feed-forward 3D reconstruction to create camera-controlled walkthrough videos that can be lifted into interactive 3D environments. The system enables users to generate simulation-ready 3D scenes that can be exported to physics engines like Isaac Sim for robotics applications and embodied AI simulation.

Key Features & Capabilities

• Persistent 3D Generation: Creates long-horizon, 3D-consistent video trajectories using novel spatial memory and temporal drift correction techniques
• Interactive GUI: Real-time visualization of accumulated point clouds with user-controlled camera trajectory planning to explore and revisit scenes
• Dual-Format Output: Exports both 3D Gaussian Splats and polygon meshes ready for real-time rendering
• Spatial Memory System: Maintains per-frame 3D geometry to prevent spatial forgetting during extended exploration
• Self-Augmented Training: Teaches the model to correct temporal drift rather than propagate synthesis errors over time
• Physics Engine Integration: Direct export to Isaac Sim for physically grounded robot navigation and interaction
• 4-Step DMD Distillation: Optimized LoRA available for faster generation

Who It's For

Lyra 2.0 targets researchers, developers, and studios working in 3D content creation, robotics simulation, virtual production, game development, and embodied AI. It's designed for users who need high-quality, explorable 3D environments without manual modeling—from VFX professionals creating virtual sets to robotics researchers building training simulations.

Typical Workflows

Users start with an input image and optional text prompt, then interactively navigate through the scene using the 3D explorer GUI. As they move, Lyra 2.0 progressively generates new video segments, retrieves relevant history frames from spatial memory, and establishes dense 3D correspondences. The system continuously lifts generated frames into point clouds that feed back into the exploration loop. Once exploration is complete, the accumulated 3D data can be reconstructed as Gaussian Splats or meshes and exported to downstream applications.

What Makes It Different

Unlike conventional video generation models that degrade quickly over long sequences, Lyra 2.0 solves two fundamental challenges: spatial forgetting (hallucinating previously observed regions) and temporal drifting (accumulating synthesis errors). Its geometry-guided information routing and self-correcting training approach enable substantially longer, more consistent 3D-coherent video generation than existing methods, making it uniquely suited for creating large-scale explorable worlds.