Autoregressive Image to 3D Generation via Next Symmetric View Prediction
Abstract
Autoregressive models via next token prediction have emerged as a promising way towards unified models across multiple modalities for general artificial intelligence. Nevertheless, exploiting this idea to achieve single image to 3D generation remains a challenging problem. In this paper, we introduce a new Autoregressive 3D diffusion model (dubbed as AR3D), that novelly frames autoregressive modeling on multi-view images of 3D content as "next symmetric view prediction." This methodology first triggers 3D autoregressive modeling by remoulding video diffusion model with unidirectional next-view prediction, where the next-view images are iteratively generated conditioned on the past views and input image. Furthermore, a unique autoregressive scheme via next symmetric view prediction is introduced to simultaneously predict each pair of views under the view-order symmetry at each autoregressive step. Such design captures the view-order symmetry of 360$^{\circ }$ orbital multi-view images and thus encourages strong geometric and appearance consistency among adjacent views in a near-to-far order. Extensive experiments demonstrate that our proposed AR3D achieves strong performance on both novel view synthesis and single view reconstruction tasks for image-to-3D generation.
Read the paper
Related studies
- Learning from First-Generation Qualitative Approaches in the IS Discipline: An Evolutionary View and Some Implications for Authors and Evaluators (PART 1/2)
- Commentary on the clinical and preclinical dosage limits of interstitially administered magnetic fluids for therapeutic hyperthermia based on current practice and efficacy models
- Prosthesis: New Technological Opportunities and Innovative Biomedical Devices
- The history and future of neural modeling for cochlear implants
- Ryanodine receptors: waking up from refractoriness
- Novel Transgenic Rabbit Model Sheds Light on the Puzzling Role of Matrix Metalloproteinase-12 in Atherosclerosis
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.