Brian Schmidt, DigiPen Institute of Technology and Brian Schmidt Studios, LLC, Zooms the PNW May 2020 meeting on 3-D audio.
The PNW Section's 2nd virtual meeting of the pandemic era featured Brian Schmidt of DigiPen and Brian Schmidt Studios, LLC, speaking about 3-D audio via Zoom. The human hearing aspects were discussed thoroughly, as well as the approaches to implementing 3-D audio. About 75 persons attended (25 AES members), several from outside of the Section and even other countries. The odd time was chosen to be more convenient for other time zones.
PNW Chair Greg Dixon opened the meeting, and announced the Section election slate for the June Annual Section Business meeting. Committeeperson Dan Mortensen then announced a new Section Zoom project, dubbed Tea Time Tiny Topics, a more informal show-and-tell meeting on the next four Saturdays. Interested persons may find details on the PNW homepage.
Brian Schmidt has been creating music, sound, and audio technology for games since 1987, and has a credits for over 130 games for companies such as Sony, Electronic Arts, Capcom, Sega, Microsoft, Zynga, Namco, and many others. Brian spent 10 years at Microsoft as the primary audio architect for the Xbox team, where he was responsible for technologies such as XMA and XACT, and created the boot sound for the original Xbox. He received undergraduate degrees in music and computer science from Northwestern University. He went on to complete his master's degree in computer applications in music in 1987, and portions of his thesis work appeared in Computer Music Journal. Brian also presented his thesis work, by invitation, to the AES special conference on audio technology. Brian is a frequent guest speaker, as well as the founder and executive director of GameSoundCon, the largest professional conference on game music and sound design. Currently, he is a Senior Lecturer at DigiPen Institute of Technology in Redmond WA USA and also runs Brian Schmidt Studios, LLC.
Brian began with a thorough discussion of the human hearing system. People have major limitations localizing sounds in 3-D. A comparison to the human visual system was made; only a small portion of our eyesight has really high resolution, and in a way, our hearing also has comparable problems.
Human hearing physiology was covered in detailed, including the construction of the inner ear. It seems we have decent capabilities with respect to figuring amplitude & frequency, their change over time, and differences from hearing with two ears. We are not great at precise localization, a bit like our peripheral vision. Our azimuth detection is best in front, not bad behind, sides not as good, and our vertical pretty bad. He described our sensing as the Big 3 + 1:
IID-interaural intensity difference
ITD-interaural time difference
HRTF-head related transfer function
as well as the reverberant field
and described each in detail. Head movement helps a lot, so head tracking helps Virtual Reality devices. Human audio distance detection was discussed, as well as many other factors that influence 3-D localization including low pass filtering in air, reverb changes, direct to reverberant changes, the first reflection you hear, expectations (you expect a bird sound to be above), even training.
For 3-D sound designers, the Big 3+1 isn't enough, and you have to consider things like human expectations, familiarity of source material, relationship to visuals, context and the state of mind of the user. Human 3-D hearing is imprecise and while tech can simulate the physics, much of human localization can't be solved with more DSP.
A vigorous Q&A session followed, ranging from HRTF topics to recording and playback systems.
Next, Lawrence Schwedler (PNW Treasurer and AVAR Conference Co-Chair) announced the AES AVAR 2020 Conference August 17, 2020. The Audio for Virtual and Augmented Reality Conference will be - virtual - this year, with a full slate but minimal cost.
Finally, everyone in attendance was invited to unmute and introduce themselves.
