Audio-Driven Talking Face Generation: A Review

Liu S. Audio-Driven Talking Face Generation: A Review. J Audio Eng Soc. 2023;71(7/8):408-419. doi:10.17743/jaes.2022.0081

@article{Liu2023_22146,
  author = {Liu, Shiguang},
  title = {{Audio-Driven Talking Face Generation: A Review}},
  journal = {Journal of the Audio Engineering Society},
  volume = {71},
  number = {7/8},
  pages = {408--419},
  year = {2023},
  month = jul,
  publisher = {Audio Engineering Society},
  doi = {10.17743/jaes.2022.0081},
  url = {https://doi.org/10.17743/jaes.2022.0081}
}

TY  - JOUR
TI  - Audio-Driven Talking Face Generation: A Review
AU  - Liu, Shiguang
T2  - Journal of the Audio Engineering Society
J2  - J. Audio Eng. Soc.
VL  - 71
IS  - 7/8
SP  - 408
EP  - 419
PY  - 2023
DA  - 2023/07/06
DO  - 10.17743/jaes.2022.0081
UR  - https://doi.org/10.17743/jaes.2022.0081
PB  - Audio Engineering Society
LA  - en
AB  - Given a face image and a speech audio, talking face generation refers to synthesizing a face video speaking the given speech. It has wide applications in movie dubbing, teleconference, virtual assistant, etc. This paper gives an overview of research progress on talking face generation in recent years. The author first reviews traditional talking face generation methods. Then, deep learning talking face generation methods, including talking face synthesis for a specific identity and talking face synthesis for an arbitrary identity, are summarized. The author then surveys recent detail-aware talking face generation methods, including noise based approaches, eye conversion based approaches, and facial anatomy based approaches. Next, the author surveys the talking head generation methods, such as video/image driven talking head generation, pose information--driven talking head generation, and audio-driven talking head generation. Finally, some future directions for talking face generation are highlighted.
ER  -