“Learning-based 3D Computer-Generated Holography”
Liang Shi
(MIT)
Please LOG IN to view the video.
Date: June 2, 2021
Description:
Computer generated holography (CGH) is fundamental to applications such as biosensing, volumetric display, optical/acoustic tweezer, security and many others that require spatial control of intricate optical or acoustic fields. For near-eye displays, CGH provides the opportunity to support true 3D projection in a sunglass-like display. Yet, the conventional approach to compute a true 3D hologram via physical simulation of diffraction and inference is slow and unaware of occlusion. Moreover, experimental results are often inferior to simulations due to non-idealized optical systems, non-linear and non-uniform SLM responses, and image degradation caused by complex to phase-only conversion. These computational and hardware-imposed challenges together limit the interactiveness and realism of the ultimate immersive experience. In this talk, I will describe techniques to mitigate these challenges, including physical simulation algorithms that handle occlusion for RGB-D and more advanced 3D input, methods to create large-scale 3D hologram datasets, training of CNNs to speed up complex and phase-only hologram synthesis, and approaches to compensate hardware limitations. Together, the resulted system can synthesis and display photorealistic 3D holograms in real-time using a single consumer-grade GPU and run interactively on an iPhone leveraging the Neural Engine. I will further discuss possible extensions that could be built top of the proposed system to support foveated rendering, static pupil expansion, view-dependent effect and other features.
Further Information:
Liang Shi is a PhD student in Prof. Wojciech Matusik’s Computational Fabrication and Design Group at MIT CSAIL. He received his B.E. from Beihang University, M.Sc. from Stanford University, and was a member of Prof. Gordon Wetzstein’s Computational Imaging Lab. He was a research intern at NVIDIA, Adobe, Facebook Reality Lab. His current research interests include computational display, fabrication, and appearance modeling.
Created: Thursday, June 3rd, 2021