120 research outputs found
Nonparallel Emotional Speech Conversion
We propose a nonparallel data-driven emotional speech conversion method. It
enables the transfer of emotion-related characteristics of a speech signal
while preserving the speaker's identity and linguistic content. Most existing
approaches require parallel data and time alignment, which is not available in
most real applications. We achieve nonparallel training based on an
unsupervised style transfer technique, which learns a translation model between
two distributions instead of a deterministic one-to-one mapping between paired
examples. The conversion model consists of an encoder and a decoder for each
emotion domain. We assume that the speech signal can be decomposed into an
emotion-invariant content code and an emotion-related style code in latent
space. Emotion conversion is performed by extracting and recombining the
content code of the source speech and the style code of the target emotion. We
tested our method on a nonparallel corpora with four emotions. Both subjective
and objective evaluations show the effectiveness of our approach.Comment: Published in INTERSPEECH 2019, 5 pages, 6 figures. Simulation
available at http://www.jian-gao.org/emoga
EmoFake: An Initial Dataset for Emotion Fake Audio Detection
Many datasets have been designed to further the development of fake audio
detection, such as datasets of the ASVspoof and ADD challenges. However, these
datasets do not consider a situation that the emotion of the audio has been
changed from one to another, while other information (e.g. speaker identity and
content) remains the same. Changing the emotion of an audio can lead to
semantic changes. Speech with tampered semantics may pose threats to people's
lives. Therefore, this paper reports our progress in developing such an emotion
fake audio detection dataset involving changing emotion state of the origin
audio named EmoFake. The fake audio in EmoFake is generated by open source
emotion voice conversion models. Furthermore, we proposed a method named Graph
Attention networks using Deep Emotion embedding (GADE) for the detection of
emotion fake audio. Some benchmark experiments are conducted on this dataset.
The results show that our designed dataset poses a challenge to the fake audio
detection model trained with the LA dataset of ASVspoof 2019. The proposed GADE
shows good performance in the face of emotion fake audio
Rhythm-Flexible Voice Conversion without Parallel Data Using Cycle-GAN over Phoneme Posteriorgram Sequences
Speaking rate refers to the average number of phonemes within some unit time,
while the rhythmic patterns refer to duration distributions for realizations of
different phonemes within different phonetic structures. Both are key
components of prosody in speech, which is different for different speakers.
Models like cycle-consistent adversarial network (Cycle-GAN) and variational
auto-encoder (VAE) have been successfully applied to voice conversion tasks
without parallel data. However, due to the neural network architectures and
feature vectors chosen for these approaches, the length of the predicted
utterance has to be fixed to that of the input utterance, which limits the
flexibility in mimicking the speaking rates and rhythmic patterns for the
target speaker. On the other hand, sequence-to-sequence learning model was used
to remove the above length constraint, but parallel training data are needed.
In this paper, we propose an approach utilizing sequence-to-sequence model
trained with unsupervised Cycle-GAN to perform the transformation between the
phoneme posteriorgram sequences for different speakers. In this way, the length
constraint mentioned above is removed to offer rhythm-flexible voice conversion
without requiring parallel data. Preliminary evaluation on two datasets showed
very encouraging results.Comment: 8 pages, 6 figures, Submitted to SLT 201
Towards General-Purpose Text-Instruction-Guided Voice Conversion
This paper introduces a novel voice conversion (VC) model, guided by text
instructions such as "articulate slowly with a deep tone" or "speak in a
cheerful boyish voice". Unlike traditional methods that rely on reference
utterances to determine the attributes of the converted speech, our model adds
versatility and specificity to voice conversion. The proposed VC model is a
neural codec language model which processes a sequence of discrete codes,
resulting in the code sequence of converted speech. It utilizes text
instructions as style prompts to modify the prosody and emotional information
of the given speech. In contrast to previous approaches, which often rely on
employing separate encoders like prosody and content encoders to handle
different aspects of the source speech, our model handles various information
of speech in an end-to-end manner. Experiments have demonstrated the impressive
capabilities of our model in comprehending instructions and delivering
reasonable results.Comment: Accepted to ASRU 202
- …