103 research outputs found
Linguistically Aided Speaker Diarization Using Speaker Role Information
Speaker diarization relies on the assumption that speech segments
corresponding to a particular speaker are concentrated in a specific region of
the speaker space; a region which represents that speaker's identity. These
identities are not known a priori, so a clustering algorithm is typically
employed, which is traditionally based solely on audio. Under noisy conditions,
however, such an approach poses the risk of generating unreliable speaker
clusters. In this work we aim to utilize linguistic information as a
supplemental modality to identify the various speakers in a more robust way. We
are focused on conversational scenarios where the speakers assume distinct
roles and are expected to follow different linguistic patterns. This distinct
linguistic variability can be exploited to help us construct the speaker
identities. That way, we are able to boost the diarization performance by
converting the clustering task to a classification one. The proposed method is
applied in real-world dyadic psychotherapy interactions between a provider and
a patient and demonstrated to show improved results.Comment: from v1: restructured Introduction and Background, added experimental
results with ASR text and language-only baselin
Language modelling for speaker diarization in telephonic interviews
The aim of this paper is to investigate the benefit of combining both language and acoustic modelling for speaker diarization. Although conventional systems only use acoustic features, in some scenarios linguistic data contain high discriminative speaker information, even more reliable than the acoustic ones. In this study we analyze how an appropriate fusion of both kind of features is able to obtain good results in these cases. The proposed system is based on an iterative algorithm where a LSTM network is used as a speaker classifier. The network is fed with character-level word embeddings and a GMM based acoustic score created with the output labels from previous iterations. The presented algorithm has been evaluated in a Call-Center database, which is composed of telephone interview audios. The combination of acoustic features and linguistic content shows a 84.29% improvement in terms of a word-level DER as compared to a HMM/VB baseline system. The results of this study confirms that linguistic content can be efficiently used for some speaker recognition tasks.This work was partially supported by the Spanish Project DeepVoice (TEC2015-69266-P) and by the project PID2019-107579RBI00/ AEI /10.13039/501100011033.Peer ReviewedPostprint (published version
Encoder-decoder multimodal speaker change detection
The task of speaker change detection (SCD), which detects points where
speakers change in an input, is essential for several applications. Several
studies solved the SCD task using audio inputs only and have shown limited
performance. Recently, multimodal SCD (MMSCD) models, which utilise text
modality in addition to audio, have shown improved performance. In this study,
the proposed model are built upon two main proposals, a novel mechanism for
modality fusion and the adoption of a encoder-decoder architecture. Different
to previous MMSCD works that extract speaker embeddings from extremely short
audio segments, aligned to a single word, we use a speaker embedding extracted
from 1.5s. A transformer decoder layer further improves the performance of an
encoder-only MMSCD model. The proposed model achieves state-of-the-art results
among studies that report SCD performance and is also on par with recent work
that combines SCD with automatic speech recognition via human transcription.Comment: 5 pages, accepted for presentation at INTERSPEECH 202
A Memory Augmented Architecture for Continuous Speaker Identification in Meetings
We introduce and analyze a novel approach to the problem of speaker
identification in multi-party recorded meetings. Given a speech segment and a
set of available candidate profiles, we propose a novel data-driven way to
model the distance relations between them, aiming at identifying the speaker
label corresponding to that segment. To achieve this we employ a recurrent,
memory-based architecture, since this class of neural networks has been shown
to yield advanced performance in problems requiring relational reasoning. The
proposed encoding of distance relations is shown to outperform traditional
distance metrics, such as the cosine distance. Additional improvements are
reported when the temporal continuity of the audio signals and the speaker
changes is modeled in. In this paper, we have evaluated our method in two
different tasks, i.e. scripted and real-world business meeting scenarios, where
we report a relative reduction in speaker error rate of 39.28% and 51.84%,
respectively, compared to the baseline.Comment: Submitted to ICASSP 202
- …