2D priors for 3D generation
Ben Poole
(Google Brain)
Please LOG IN to view the video.
Date: February 8, 2023
Description:
Large scale datasets of images with text descriptions have enabled powerful models that represent and generate pixels. But progress in 3D generation has been slow due to the lack of 3D data and efficient architectures. In this talk, I’ll present DreamFields and DreamFusion: two approaches that enable 3D generation from 2D priors using no 3D data. By turning 2D priors into loss functions, we can optimize 3D models (NeRFs) from scratch via gradient descent. These methods enable high-quality generation of 3D objects from diverse text prompts. Finally, I’ll discuss a fundamental problem with our approach and how new pixel-space priors like Imagen Video can unlock new 3D capabilities.
Further Information:
Ben Poole is a research scientist at Google Brain in San Francisco working on deep generative models for images, video, and 3D. He completed his PhD at Stanford University advised by Surya Ganguli in the Neural Dynamics and Computation lab. His thesis was on computational tools to develop a better understanding of both biological and artificial neural networks. He’s worked at DeepMind, Google Research, Intel Research Pittsburgh, and the NYU Center for Neural Science.
Created: Thursday, February 9th, 2023