47 research outputs found
Improving Voice Trigger Detection with Metric Learning
Voice trigger detection is an important task, which enables activating a
voice assistant when a target user speaks a keyword phrase. A detector is
typically trained on speech data independent of speaker information and used
for the voice trigger detection task. However, such a speaker independent voice
trigger detector typically suffers from performance degradation on speech from
underrepresented groups, such as accented speakers. In this work, we propose a
novel voice trigger detector that can use a small number of utterances from a
target speaker to improve detection accuracy. Our proposed model employs an
encoder-decoder architecture. While the encoder performs speaker independent
voice trigger detection, similar to the conventional detector, the decoder
predicts a personalized embedding for each utterance. A personalized voice
trigger score is then obtained as a similarity score between the embeddings of
enrollment utterances and a test utterance. The personalized embedding allows
adapting to target speaker's speech when computing the voice trigger score,
hence improving voice trigger detection accuracy. Experimental results show
that the proposed approach achieves a 38% relative reduction in a false
rejection rate (FRR) compared to a baseline speaker independent voice trigger
model.Comment: Submitted to InterSpeech 202
Deep Spoken Keyword Spotting:An Overview
Spoken keyword spotting (KWS) deals with the identification of keywords in
audio streams and has become a fast-growing technology thanks to the paradigm
shift introduced by deep learning a few years ago. This has allowed the rapid
embedding of deep KWS in a myriad of small electronic devices with different
purposes like the activation of voice assistants. Prospects suggest a sustained
growth in terms of social use of this technology. Thus, it is not surprising
that deep KWS has become a hot research topic among speech scientists, who
constantly look for KWS performance improvement and computational complexity
reduction. This context motivates this paper, in which we conduct a literature
review into deep spoken KWS to assist practitioners and researchers who are
interested in this technology. Specifically, this overview has a comprehensive
nature by covering a thorough analysis of deep KWS systems (which includes
speech features, acoustic modeling and posterior handling), robustness methods,
applications, datasets, evaluation metrics, performance of deep KWS systems and
audio-visual KWS. The analysis performed in this paper allows us to identify a
number of directions for future research, including directions adopted from
automatic speech recognition research and directions that are unique to the
problem of spoken KWS
Keyword localisation in untranscribed speech using visually grounded speech models
Keyword localisation is the task of finding where in a speech utterance a
given query keyword occurs. We investigate to what extent keyword localisation
is possible using a visually grounded speech (VGS) model. VGS models are
trained on unlabelled images paired with spoken captions. These models are
therefore self-supervised -- trained without any explicit textual label or
location information. To obtain training targets, we first tag training images
with soft text labels using a pretrained visual classifier with a fixed
vocabulary. This enables a VGS model to predict the presence of a written
keyword in an utterance, but not its location. We consider four ways to equip
VGS models with localisations capabilities. Two of these -- a saliency approach
and input masking -- can be applied to an arbitrary prediction model after
training, while the other two -- attention and a score aggregation approach --
are incorporated directly into the structure of the model. Masked-based
localisation gives some of the best reported localisation scores from a VGS
model, with an accuracy of 57% when the system knows that a keyword occurs in
an utterance and need to predict its location. In a setting where localisation
is performed after detection, an of 25% is achieved, and in a setting
where a keyword spotting ranking pass is first performed, we get a localisation
P@10 of 32%. While these scores are modest compared to the idealised setting
with unordered bag-of-word-supervision (from transcriptions), these models do
not receive any textual or location supervision. Further analyses show that
these models are limited by the first detection or ranking pass. Moreover,
individual keyword localisation performance is correlated with the tagging
performance from the visual classifier. We also show qualitatively how and
where semantic mistakes occur, e.g. that the model locates surfer when queried
with ocean.Comment: 10 figures, 5 table