Computational photography at the point of capture on mobile cameras

Marc Levoy

(Stanford and Adobe)

Please LOG IN to view the video.

Date: March 30, 2022


The ability to capture bursts of images in rapid succession with varying camera settings, and to process them quickly on device, has revolutionized photography on cell phones, as well as disrupting the camera industry. I will first summarize computational photography technologies on Google’s Pixel smartphones: high dynamic range photography (“HDR+”), simulated shallow depth-of-field (“Portrait mode”), multi-frame super-resolution (“Super Res Zoom”), and photography in very low light (“Night Sight”). My goal at Adobe is to build tools, based on computational photography, for serious photographers and creative professionals. In this spirit, I will enumerate some technologies that have been explored by the research community, and which with some more effort could be made robust and efficient enough to deploy on mobile devices, and controllable enough to be useful as a creative tool. Our ultimate goal is to turn picture-taking into an interactive collaboration between camera and photographer, mediated by computational photography and machine learning.

Further Information:

Marc Levoy is the VMware Founders Professor of Computer Science (Emeritus) at Stanford University and a Vice President and Fellow at Adobe. In previous lives he worked on computer-assisted cartoon animation (1970s), volume rendering (1980s), 3D scanning (1990s), light field imaging (2000s), and computational photography (2010s). At Stanford he taught computer graphics, digital photography, and the science of art. At Google he launched Street View, co-designed the library book scanner, and led the team that created HDR+, Portrait Mode, and Night Sight for Pixel smartphones. These phones won DPReview’s Innovation of the Year (2017 and 2018) and Smartphone Camera of the Year (2019), and Mobile World Congress’s Disruptive Innovation Award (2019). Levoy’s awards include Cornell University Charles Goodwin Sands Medal for best undergraduate thesis (1976), ACM SIGGRAPH Computer Graphics Achievement Award (1996), ACM Fellow (2007), and National Academy of Engineering (2022).

Created: Sunday, April 3rd, 2022