Dynamic 3D Scene Generation with Video Diffusion Models
Abstract
Generative models capable of synthesizing and reconstructing 3D and 4D scenes are becoming central tools for virtual and augmented reality, computer graphics, simulation, and embodied AI. These models aim to infer geometry, appearance, and motion from minimal inputs such as text, images, or monocular video, enabling new forms of controllable scene generation and large-scale world modeling. However, achieving high visual fidelity while maintaining accurate 3D structure, realistic dynamics, and efficient controllability presents fundamental computational challenges. This thesis introduces a set of algorithmic approaches that address these challenges and advance 4D scene generation into a reliable and scalable methodology. Specifically, the thesis develops several complementary strategies for improving controllability, consistency, and reconstruction in 4D generative modeling. These include distillation-based methods that combine multiple diffusion priors to jointly model appearance, geometry, and dynamics; trajectory-conditioned approaches that enable user-specified motion within complex scenes; camera-aware video generation techniques that introduce explicit viewpoint control into spatiotemporal transformers; analysis-driven conditioning schemes that enhance the accuracy of camera control; and feed-forward reconstruction pipelines that distill video diffusion models into explicit 3D representations. Together, these methods reduce reliance on heavy optimization, improve motion…
Read the paper
Related studies
- Learning from First-Generation Qualitative Approaches in the IS Discipline: An Evolutionary View and Some Implications for Authors and Evaluators (PART 1/2)
- Commentary on the clinical and preclinical dosage limits of interstitially administered magnetic fluids for therapeutic hyperthermia based on current practice and efficacy models
- Prosthesis: New Technological Opportunities and Innovative Biomedical Devices
- The history and future of neural modeling for cochlear implants
- Ryanodine receptors: waking up from refractoriness
- Novel Transgenic Rabbit Model Sheds Light on the Puzzling Role of Matrix Metalloproteinase-12 in Atherosclerosis
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.