Audio Source Separation Using Variational Autoencoders and Weak Class Supervision

This repository contains supplementary material for the paper, available on arXiv and IEEE Xplore.

Demo

You can listen to some examples of our results.

Source code

• src/mix.py: Generates the SC09 mixture dataset.
• src/train.py: Training script.
• src/test.py: Testing script. It computes the BSS_EVAL metrics.
• src/experiment.py: Automates the mix-train-test scripts.
• src/demo.py: Generates the demo page.
• notebooks/ResultAnalysis.ipynb: Analysis of the experiment results.

Requirements

This project requires Python 3 with the packages pinned in src/requirements.txt (PyTorch, SciPy, librosa, mir_eval, matplotlib, Jinja2):

pip install -r src/requirements.txt

Dataset

SC09 is the subset of spoken digits ("zero"–"nine") from the Speech Commands Dataset v0.02:
Download

Attribution

Speech Commands Dataset (Google; CC BY 4.0) — Pete Warden, "Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition" (2018). The audio samples on the demo page are derived from this dataset and have been modified (mixed, separated, and processed) for this work.

Usage

The mix → train → test pipeline is driven by experiment.py (also see src/run.sh):

cd src
python experiment.py --sc-root ~/datasets/speech_commands --experiments-root ~/experiments --run compare_ae_vae_source_label

Citation (BibTeX)

If you find this repository useful, please cite our work:

@article{karamatli2019audio,
  title={Audio source separation using variational autoencoders and weak class supervision},
  author={Karamatl{\i}, Ertu{\u{g}} and Cemgil, Ali Taylan and K{\i}rb{\i}z, Serap},
  journal={IEEE Signal Processing Letters},
  volume={26},
  number={9},
  pages={1349--1353},
  year={2019},
  doi={10.1109/LSP.2019.2929440}
}