Exploring the User Experience of AI-Assisted Sound Searching Systems for Creative Workflows

Liu H, Deacon T, Wang W, Paradis M, Plumbley MD. Exploring the User Experience of AI-Assisted Sound Searching Systems for Creative Workflows. In: Convention Paper. Audio Engineering Society; 2025. Paper 10233. Available from: https://aes.org/publications/elibrary-page/?id=23077

@inproceedings{Liu2025_23077,
  author = {Liu, Haohe and Deacon, Thomas and Wang, Wenwu and Paradis, Matt and Plumbley, Mark D.},
  title = {{Exploring the User Experience of AI-Assisted Sound Searching Systems for Creative Workflows}},
  note = {Paper 10233},
  year = {2025},
  month = oct,
  publisher = {Audio Engineering Society},
  url = {https://aes.org/publications/elibrary-page/?id=23077}
}

TY  - CPAPER
TI  - Exploring the User Experience of AI-Assisted Sound Searching Systems for Creative Workflows
AU  - Liu, Haohe
AU  - Deacon, Thomas
AU  - Wang, Wenwu
AU  - Paradis, Matt
AU  - Plumbley, Mark D.
M1  - Paper 10233
PY  - 2025
DA  - 2025/10/14
UR  - https://aes.org/publications/elibrary-page/?id=23077
PB  - Audio Engineering Society
LA  - en
AB  - Locating the right sound effect efficiently is an important yet challenging topic for audio production. Most current sound-searching systems rely on pre-annotated audio labels created by humans, which can be time-consuming to produce and prone to inaccuracies, limiting the efficiency of audio production. Recent works on text and audio multimodal neural networks have led to the development of contrastive language-audio pretraining (CLAP), which learns a shared embedding space for text descriptions and audio samples. Using this idea, we built a CLAP-based sound searching system (CLAP-Search) that does not rely on human annotations. To evaluate the effectiveness of CLAP-Search, we conducted comparative experiments with a widely used sound effect searching platform, the BBC Sound Effect Library. Our study evaluates user performance, cognitive load, and satisfaction through ecologically valid tasks based on professional sound-searching workflows. Our result shows that CLAP-Search demonstrated significantly enhanced productivity and reduced frustration while maintaining comparable cognitive demands. We also qualitatively analyzed the participants feedback, which offered valuable perspectives on the design of future AI-assisted sound search systems.
ER  -