Project Starline: A high-fidelity telepresence system

Jason Lawrence

(Google)

Please LOG IN to view the video.

Date: January 12, 2022

Description:

 We present a real-time bidirectional communication system that lets two people, separated by distance, experience a face-to-face conversation as if they were copresent. It is the first telepresence system that is demonstrably better than 2D videoconferencing, as measured using participant ratings (e.g., presence, attentiveness, reaction-gauging, engagement), meeting recall, and observed nonverbal behaviors (e.g., head nods, eyebrow movements). This milestone is reached by maximizing audiovisual fidelity and the sense of copresence in all design elements, including physical layout, lighting, face tracking, multi-view capture, microphone array, multi-stream compression, loudspeaker output, and lenticular display. Our system achieves key 3D audiovisual cues (stereopsis, motion parallax, and spatialized audio) and enables the full range of communication cues (eye contact, hand gestures, and body language), yet does not require special glasses or body-worn microphones/headphones. The system consists of a head-tracked autostereoscopic display, high-resolution 3D capture and rendering subsystems, and network transmission using compressed color and depth video streams. Other contributions include a novel image-based geometry fusion algorithm, free-space dereverberation, and talker localization.

Further Information:

• Search Talks

  Search for:

• October 2023

  M	T	W	T	F	S	S
  « Aug
  						1
  2	3	4	5	6	7	8
  9	10	11	12	13	14	15
  16	17	18	19	20	21	22
  23	24	25	26	27	28	29
  30	31

• Sections

  Welcome to Stanford Talks! Technical Support CNI     CNI Inaugural Event     CNI Neuroimaging Tutorials     CNI Neuroimaging Seminars     CNI 5th Year Event Center for Mind, Brain and Computation (MBC)     MBC Symposium: How the Brain Makes Meaning     MBC Symposium: Computational Mechanisms of Learning and Memory     MBC Symposium: Distributed Computation through Neural Dynamics     MBC Symposium: The Role of Sparsity in Neural Computation and the Natural World     MBC Symposium: Deep Learning     MBC Graduate Training Seminar Series     MBC Symposium: Retina - Pathways, Processing, and Prostheses     MBC Symposium: Network dynamics: Theory and Experiment Psychology     Psychology Colloquium Series     PSYCH: Awards and Honors     PSYCH 216A: Statistics and Data Analysis in Matlab     PSYCH 221: Applied Vision and Image Systems     PSYCH: Special Lectures     PSYCH: Hilgard Scholar Lecture Series     PSYCH 204A: Introduction to Human Neuroimaging Using MRI     PSYCH 212: Graduate Social Psychology     Vision Science Society Symposia: The Visual White-Matter Matters     Psychology One Annual Conference SCIEN     SCIEN Symposium on Biomedical Imaging     SCIEN Colloquium Series     SCIEN: Workshop on 3D Imaging     SCIEN: Workshop on Light Field Imaging     SCIEN: Workshop on Cinematic VR and Immersive Storytelling     SCIEN: VR in China     SCIEN Industry Affiliates Meetings     SCIEN Workshop on Augmented and Mixed Reality     SCIEN: Workshop on Medical VR/AR     SCIEN Workshop on the Future of Medical Imaging: Sensing, Learning and Visualization Stanford Law School     SIGNS & CLB: Discussion Series     Law and Memory Conference     Law and Policy of the Developing Brain: Neuroscience from Womb to Death     CLB: Podcast Series CTL: Center for Teaching and Learning     CTL: Award Winning Teachers on Teaching Recent Talks SCIEN LOGIN Stanford Login Login Stanford Neurosciences     Neural Engineering Workshop     Stanford Brain Rejuvenation Project     SNI Seminar Series     Neurosciences Institute Seed Grant Mini-Symposium FAQ Speaker Release Service Inquiry Tutorials     Neuroimaging Tutorials     Statistics Tutorials     Image Systems Tutorials     Programming Tutorials     ISETBIO Tutorials SystemX Alliance     SystemX Alliance: Seminar Series     SystemX Alliance Workshop: Computing, April 2015     SystemX Alliance Conference: Spring 2015     SystemX Alliance Conference: Fall 2015     SystemX Alliance: Workshop on Bio Interfaces Statistics     Statistics Seminars

• Recent Talks

  • Prof. George Drettakis and Georgios Kopanas (INRIA): “Geometry-Guided Neural Rendering for Real and Synthetic Scenes” (08/15/2023)
  • Professor Kaan Akşit (Univ College, London): “Could Holographic Displays be the key to achieving Realism?” (08/15/2023)
  • Professor Peter Wonka (KAUST): “Data Structures for Generative Modeling of 3D Shapes” (06/16/2023)
  • Professor Jung-Hoon Park ( UNIST, Korea): “Shaping Light for Bio-Imaging” (06/01/2023)
  • Francois Chaubard (CEO at Focal Systems): “Lesson in Productizing Deep Learning Computer Vision: Automating Brick & Mortar Retail with State of the Art Computer Vision” (05/26/2023)
  • Dr. Christoph Leuze (Stanford): “Headset design choices for augmented reality applications in industry and medicine.” (05/19/2023)
  • Protected: Professor Rajesh Menon (University of Utah): “Non-anthropocentric Imaging with & without optics” (05/16/2023)
  • Protected: Professor Na Ji (University of California at Berkeley): “Imaging the brain at high spatiotemporal resolution” (05/05/2023)
  • Protected: Professor Ioannis Gkioulekas (Carnegie Mellon University): “Interferometric computational imaging” (05/02/2023)
  • Protected: Professor Yuhao Zhu (Univ of Rochester): “Harnessing the Computer Science-Vision Science Symbiosis” (04/21/2023)

• Topics

  3D augmented reality brain brain imaging camera CLB CNI CNS Cognitive Neuroscience computational imaging computer vision computing deep-learning digital imaging fMRI image sensor ipython law learning light field imaging machine learning MBC medical imaging medical technology memory microscopy MRI MR Methods neural circuitry neural coding neural computation neural network Neuroscience photography podcast programming psychology python retina scien social psychology statistics tutorial virtual reality vision

• More talks

  • NVIDIA GPU Technology Conference 2015
  • NeuWriteWest - podcast series
  • Janelia’s public lecture series
  • Allen Institute for Brain Science Annual Symposium
  • EE distinguished lecture series (Stanford)
  • National Academy of Sciences - 2014 Talks

• Share…

  • Google+
  • Facebook
  • LinkedIn
  • Twitter
  • Tumblr

© Stanford Talks
Stanford University | Terms of Use | Copyright Complaints