Conditioned Source Separation by Attentively Aggregating Frequency Transformations With Self-Conditioning

Choi W, Jeong YS, Kim J, Chung J, Jung S, Reiss JD. Conditioned Source Separation by Attentively Aggregating Frequency Transformations With Self-Conditioning. J Audio Eng Soc. 2022;70(9):661-673. doi:10.17743/jaes.2022.0030

@article{Choi2022_21880,
  author = {Choi, Woosung and Jeong, Yeong-Seok and Kim, Jinsung and Chung, Jaehwa and Jung, Soonyoung and Reiss, Joshua D.},
  title = {{Conditioned Source Separation by Attentively Aggregating Frequency Transformations With Self-Conditioning}},
  journal = {Journal of the Audio Engineering Society},
  volume = {70},
  number = {9},
  pages = {661--673},
  year = {2022},
  month = sep,
  publisher = {Audio Engineering Society},
  doi = {10.17743/jaes.2022.0030},
  url = {https://doi.org/10.17743/jaes.2022.0030}
}

TY  - JOUR
TI  - Conditioned Source Separation by Attentively Aggregating Frequency Transformations With Self-Conditioning
AU  - Choi, Woosung
AU  - Jeong, Yeong-Seok
AU  - Kim, Jinsung
AU  - Chung, Jaehwa
AU  - Jung, Soonyoung
AU  - Reiss, Joshua D.
T2  - Journal of the Audio Engineering Society
J2  - J. Audio Eng. Soc.
VL  - 70
IS  - 9
SP  - 661
EP  - 673
PY  - 2022
DA  - 2022/09/06
DO  - 10.17743/jaes.2022.0030
UR  - https://doi.org/10.17743/jaes.2022.0030
PB  - Audio Engineering Society
LA  - en
AB  - Label-conditioned source separation extracts the target source, specified by an input symbol, from an input mixture track. A recently proposed label-conditioned source separation model called Latent Source Attentive Frequency Transformation (LaSAFT)--Gated Point-Wise Convolutional Modulation (GPoCM)--Net introduced a block for latent source analysis called LaSAFT. Employing LaSAFT blocks, it established state-of-the-art performance on several tasks of the MUSDB18 benchmark. This paper enhances the LaSAFT block by exploiting a self-conditioning method. Whereas the existing method only cares about the symbolic relationships between the target source symbol and latent sources, ignoring audio content, the new approach also considers audio content. The enhanced block computes the attention mask conditioning on the label and the input audio feature map. Here, it is shown that the conditioned U-Net employing the enhanced LaSAFT blocks outperforms the previous model. It is also shown that the present model performs the audio-query--based separation with a slight modification.
ER  -