Keynotes

Featured Speakers:

Monday, August 20:  Jean-Marc Jot, Distinguished Fellow, Magic Leap

Tuesday, August 21:  Ivan Tashev, Microsoft Research Labs - Redmond

Wednesday, August 22:  Ravish Mehra, Lead Research Scientist, Facebook Reality Labs

KEYNOTE ADDRESS #1

Unleashing Wearable 3D Audio

Dr. Jean-Marc Jot, Distinguished Fellow, Magic Leap

New virtual and augmented reality experiences must render multiple digital objects and support navigation though complex scenes for a physically mobile user wearing a head-mounted audio-visual display device. These applications challenge the state of the art in binaural interactive audio rendering technology. Fortunately, practical algorithmic solutions can be designed by exploiting and expanding our knowledge of human perception and its properties. Faithful 6-degree-of-freedom interactive graphics and audio is becoming possible with modern mobile hardware architectures and rendering algorithms optimized to retain the perceptually relevant cues necessary for audio/visual and real/virtual congruence. In this talk, we survey practical approaches for computationally efficient interactive binaural audio spatialization of multiple virtual sound sources, including near-field localization and spatially extended objects. We address the effects of acoustic reverberation, reflectors and obstacles in the virtual and physical environments, driven by combinations of pre-computed and real-time propagation models. We approach some additional challenges and opportunities brought about by the concurrent experience of physical and digital audio objects in mixed-reality applications.

Dr. Jean-Marc Jot is a Distinguished Fellow at Magic Leap, where he is responsible for audio processing technology innovation. Prior to Magic Leap, he led audio processing R&D at DTS. Previously, with Creative Labs, he led the design and development of SoundBlaster audio processing algorithms and architectures, including OpenAL and EAX technologies for game 3D audio authoring and rendering. Before relocating to California in the late nineties, he conducted research at IRCAM in Paris, where he designed the original IRCAM Spat software suite for spatial audio creation, performance, and virtual reality.

KEYNOTE ADDRESS #2

Capture, representation, and rendering of 3D audio for virtual and augmented reality

Dr. Ivan Tashev, Microsoft Research Labs - Redmond

The design of virtual and augmented reality devices requires realistic rendering of 3D audio, which creates the need for technologies and algorithms in the entire chain of capture, processing, and representation of spatial audio.  In this talk, we will discuss techniques for capturing 3D audio using specialized microphone arrays and its representation with high order ambisonics or as sound objects and diffuse component.  Rendering of spatial audio can be done using headphones with generic or personalized HRTF's, or through a set of loudspeakers.  We will discuss the advantages and disadvantages of both approaches and share some of our experience in generating personalized HRTF's.

Dr. Ivan Tashev received his Master’s degree in Electronic Engineering (1984) and PhD in Computer Science (1990) from the Technical University of Sofia, Bulgaria. He was Assistant Professor in the same university when he joined Microsoft in 1998. Currently Dr. Tashev is a Partner Architect and leads the Audio and Acoustics Research Group in Microsoft Research Labs in Redmond, WA, USA. He has published two books as a sole author and has written chapters in two other books, authored more than 100 scientific papers, and is listed as inventor of 40 granted US patents. Dr. Tashev created audio processing technologies incorporated in Windows, Microsoft Auto Platform, and the Round Table device. He served as the audio architect for Kinect for Xbox and for Microsoft HoloLens. His team designed the algorithms for all four audio subsystems in HoloLens. Ivan Tashev is a member of AES and served as AES Pacific Northwest Committee member (2006-2016), is a senior member of IEEE Signal Processing Society, and serves as member or affiliated member of the Audio and Acoustics Technical Committee and Industrial DSP Standing Committee of this society. Dr. Tashev is affiliate professor in the Department of Electrical Engineering of the University of Washington in Seattle.

KEYNOTE ADDRESS #3 

Sound Propagation for Virtual and Augmented Reality

Dr. Ravish Mehra, Lead Research Scientist, Facebook Reality Labs

Sound propagation is the process of transmission of sound as it is emitted by the source, interacts with the environment and reaches the listener. In the field of VR, realistic sound propagation helps immerse the user in the virtual environment and create a sense of presence. In augmented reality, sound propagation can be used to reinforce the physicality of virtual objects in AR environments by producing sound events that are synchronized with the real world. In this talk, I will discuss sound propagation for interactive applications in VR and AR. I will give an overview of the field of sound propagation and the class of techniques that have been developed over the last three decades. Lastly, I will present a novel sound propagation technique developed at the Oculus Research Lab that has been optimized for generating high-quality acoustics for VR and AR applications.

Dr. Ravish Mehra is the Lead Research Scientist at Facebook Reality Labs where he is responsible for developing novel techniques to push the state-of-the-art for audio in VR and AR. He completed his PhD in Computer Science at the University of North Carolina at Chapel Hill in the field of acoustics and spatial audio. In his doctoral work, he worked on novel physically-based simulation techniques for simulating complex acoustic phenomena arising out of propagation of sound waves in large environments.  His research interests span the fields of audio, acoustics, signal processing, and virtual and augmented reality. Dr. Mehra's work in acoustics and spatial audio has generated considerable interest in the audio community, and his sound propagation and spatial sound system has been integrated into virtual reality systems (Oculus HMD), with demonstrated benefits.