“Photographic Image Priors in the Era of Machine Learning”
Eero Simoncelli
(NYU)
Please LOG IN to view the video.
Date: May 12, 2021
Description:
Prior probability models are a central component of the statistical formulation of inverse problems, but density estimation is a notoriously difficult problem for high dimensional signals such as photographic images. Machine learning methods have produced impressive solutions for many inverse problems, greatly surpassing those achievable with simple prior models, but these are often not well understood and don’t generalize well beyond their training context. About a decade ago, a new approach known as “plug-and-play” was proposed, in which a denoiser is used as an algorithmic component for imposing prior information. I’ll describe our progress in understanding and using this implicit prior. We derive a surprisingly simple algorithm for drawing high-probability samples from the implicit prior embedded within a CNN trained to perform blind (i.e., unknown noise level) least-squares Gaussian denoising. A generalization of this algorithm to constrained sampling provides a method for solving *any* linear inverse problem, with no additional training, and no further distributional assumptions. We demonstrate this general form of transfer learning in multiple applications, using the same algorithm to produce state-of-the-art solutions for deblurring, super-resolution, and compressive sensing. I’ll also discuss extensions to visualizing information capture in foveated visual systems. This is joint work with Zahra Kadkhodaie, Sreyas Mohan, and Carlos Fernandez-Granda.
Further Information:
Eero Simoncelli is Silver Professor at New York University and the Director of the Center for Computational Neuroscience at the Flatiron Institute of the Simons Foundation. He is a Fellow of the Institute of Electrical and Electronics Engineers and was a Howard Hughes Medical Institute Investigator from 2000-2020 Simoncelli received his B.S. in physics (summa cum laude) in 1984 from Harvard University, studied applied mathematics at Cambridge University for a year and a half, and then received his M.S. in 1988 and his Ph.D. in 1993, both in electrical engineering from Massachusetts Institute of Technology. He was an assistant professor of computer and information science at the University of Pennsylvania from 1993 until 1996. He moved to New York University in September 1996, where he is currently a professor of neural science, mathematics, data science and psychology. His research interests span a wide range of topics in the representation and analysis of visual images and sounds in both machine and biological vision systems. In addition to his role as scientific director, Simoncelli remains an Investigator with the Simons Collaboration on the Global Brain.
Created: Friday, May 14th, 2021