26,648 research outputs found
Deep Remix: Remixing Musical Mixtures Using a Convolutional Deep Neural Network
Audio source separation is a difficult machine learning problem and
performance is measured by comparing extracted signals with the component
source signals. However, if separation is motivated by the ultimate goal of
re-mixing then complete separation is not necessary and hence separation
difficulty and separation quality are dependent on the nature of the re-mix.
Here, we use a convolutional deep neural network (DNN), trained to estimate
'ideal' binary masks for separating voice from music, to perform re-mixing of
the vocal balance by operating directly on the individual magnitude components
of the musical mixture spectrogram. Our results demonstrate that small changes
in vocal gain may be applied with very little distortion to the ultimate
re-mix. Our method may be useful for re-mixing existing mixes
Deep clustering: Discriminative embeddings for segmentation and separation
We address the problem of acoustic source separation in a deep learning
framework we call "deep clustering." Rather than directly estimating signals or
masking functions, we train a deep network to produce spectrogram embeddings
that are discriminative for partition labels given in training data. Previous
deep network approaches provide great advantages in terms of learning power and
speed, but previously it has been unclear how to use them to separate signals
in a class-independent way. In contrast, spectral clustering approaches are
flexible with respect to the classes and number of items to be segmented, but
it has been unclear how to leverage the learning power and speed of deep
networks. To obtain the best of both worlds, we use an objective function that
to train embeddings that yield a low-rank approximation to an ideal pairwise
affinity matrix, in a class-independent way. This avoids the high cost of
spectral factorization and instead produces compact clusters that are amenable
to simple clustering methods. The segmentations are therefore implicitly
encoded in the embeddings, and can be "decoded" by clustering. Preliminary
experiments show that the proposed method can separate speech: when trained on
spectrogram features containing mixtures of two speakers, and tested on
mixtures of a held-out set of speakers, it can infer masking functions that
improve signal quality by around 6dB. We show that the model can generalize to
three-speaker mixtures despite training only on two-speaker mixtures. The
framework can be used without class labels, and therefore has the potential to
be trained on a diverse set of sound types, and to generalize to novel sources.
We hope that future work will lead to segmentation of arbitrary sounds, with
extensions to microphone array methods as well as image segmentation and other
domains.Comment: Originally submitted on June 5, 201
- …