Sound Source Separation

Tools for multi-channel sound source separation and speech enhancement

News

  • I have published my new paper entitled
    "Fast Multichannel Nonnegative Matrix Factorization with Directivity-Aware Jointly-Diagonalizable Spatial Covariance Matrices for Blind Source Separation", IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2020.
    This paper is about the extension of FastMNMF, called FastMNMF2.

  • I added new initialization method for FastMNMF called gradual initialization proposed in the paper above.

    python3 FastMNMF.py [input_filename] --init_SCM gradual
  • I added new option "n_bit" to jointly-diagonalizable full-rank models (e.g. FastMNMF) and rank-1 models. You can set the number of bits to 32 or 64 (32-> float32 and complex64, 64->float64 and complex128, default is 64). By using 32, you can reduce computational cost and memory usage in exchange for the performance. Especially when the number of microphones is large, the performance is likely to degrade. Moreover, when you are using simulated signals without reverberation, since the mixture SCM is likely to be rank-deficient, please add some noise to the simulated signals (as in FastMNMF.py).

  • I removed "CupyLibrary" directory because now cupy.linalg.inv supports complex values. Please use the latest version of Cupy.

FullRank Model

FullRank_Model includes 3 methods called Full-rank Spatial Covariance Analysis (FCA), Multichannel Nonnegative Matrix Factorization(MNMF), MNMF with a deep prior (MNMF-DP).
These methods are based on full-rank spatial model.

  • FCA is a method for general source separation. In fact, it can be available only for speech enhancement because of the strong initial value dependency.
  • MNMF is a general source separation method which integrate NMF-based source model into FCA.
  • MNMF-DP is a method which integrates deep speech prior into MNMF, and is only for speech enhancement.

Jointly Diagonalizable FullRank Model

Jointly_Diagonalizable_FullRank_Model includes 3 methods called FastFCA, FastMNMF, and FastMNMF-DP (FastMNMF with a deep prior) with iterative-projection (IP) method.
These methods are based on the jointly diagonalizable full-rank spatial model, and they are extension of FCA, MNMF, MNMF-DP, respectively.

  • FastFCA is a method for general source separation. In fact, it can be available only for speech enhancement because of the strong initial value dependency.
  • FastMNMF is a general source separation method which integrate NMF-based source model into FastFCA.
  • FastMNMF-DP is a method which integrates deep speech prior into FastMNMF, and is only for speech enhancement.

Rank-1 Model

Rank1_Model includes 2 methods called Independent Low-Rank Matrix Analysis (ILRMA) and ILRMA with a deep prior (ILRMA-DP).
These methods are based on rank-1 spatial model.

  • ILRMA is a general source separation method which integrate NMF-based source model into rank-1 spatial model.
  • ILRMA-DP is a method which integrates deep speech prior into ILRMA, and is only for speech enhancement.

Requirement

  • Tested on Python3.7
  • numpy
  • pickle
  • librosa
  • soundfile
  • progressbar2
  • chainer (7.0.0 was tested) (for MNMF-DP, FastMNMF-DP, ILRMA-DP)
  • cupy (7.8.0 was tested) (for GPU accelaration)

Usage

python3 FastMNMF.py [input_filename] --gpu [gpu_id]

Input is the multichannel observed signals.
If gpu_id < 0, CPU is used, and cupy is not necessary.

Citation

If you use the code of "Jointly Diagonalizable FullRank Model" in a research project, please cite the following paper:

If you use the code of "FullRank Model" in a research project, please cite the following paper:

Soundsourceseparation

The code for multi-channel speech enhancement and source separation such as MNMF, MNMF_DP, ILRMA, ILRMA_DP, FastMNMF, FastMNMF_DP, FCA, FastFCA

Soundsourceseparation Info

⭐ Stars 47
🔗 Source Code github.com
🕒 Last Update a year ago
🕒 Created 2 years ago
🐞 Open Issues 0
➗ Star-Issue Ratio Infinity
😎 Author sekiguchi92