DeepEarNet: Individualizing Spatial Audio with Photography, Ear Shape Modeling, and Neural Networks

Kaneko S, Suenaga T, Sekine S. DeepEarNet: Individualizing Spatial Audio with Photography, Ear Shape Modeling, and Neural Networks. In: AES Conference: 2016 AES International Conference on Audio for Virtual and Augmented Reality. Audio Engineering Society; 2016. Paper 6-3. Available from: https://aes.org/publications/elibrary-page/?id=18509

@inproceedings{Kaneko2016_18509,
  author = {Kaneko, Shoken and Suenaga, Tsukasa and Sekine, Satoshi},
  title = {{DeepEarNet: Individualizing Spatial Audio with Photography, Ear Shape Modeling, and Neural Networks}},
  booktitle = {AES Conference: 2016 AES International Conference on Audio for Virtual and Augmented Reality},
  note = {Paper 6-3},
  year = {2016},
  month = sep,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=18509}
}

TY  - CPAPER
TI  - DeepEarNet: Individualizing Spatial Audio with Photography, Ear Shape Modeling, and Neural Networks
AU  - Kaneko, Shoken
AU  - Suenaga, Tsukasa
AU  - Sekine, Satoshi
T2  - AES Conference: 2016 AES International Conference on Audio for Virtual and Augmented Reality
M1  - Paper 6-3
PY  - 2016
DA  - 2016/09/06
UR  - https://aes.org/publications/elibrary-page/?id=18509
PB  - Audio Engineering Society
LA  - en
AB  - Individualizing spatial audio is of crucial importance for high-quality virtual and augmented reality audio. In this paper we propose a method for individualizing spatial audio by combining the recently proposed ear shape modeling technique with computer vision and machine learning. We use a convolutional neural network to obtain estimates of the ear shape model parameters from stereo photographs of the user ear. The individualized ear shape and its associated individualized head-related transfer function (HRTF) can be calculated from the obtained parameters based on the ear shape model and numerical acoustic simulations. Preliminary experiments, evaluating the shapes of the estimated individual ears, proved the effect of individualization.
ER  -