11 research outputs found
Spot the conversation: speaker diarisation in the wild
The goal of this paper is speaker diarisation of videos collected 'in the
wild'. We make three key contributions. First, we propose an automatic
audio-visual diarisation method for YouTube videos. Our method consists of
active speaker detection using audio-visual methods and speaker verification
using self-enrolled speaker models. Second, we integrate our method into a
semi-automatic dataset creation pipeline which significantly reduces the number
of hours required to annotate videos with diarisation labels. Finally, we use
this pipeline to create a large-scale diarisation dataset called VoxConverse,
collected from 'in the wild' videos, which we will release publicly to the
research community. Our dataset consists of overlapping speech, a large and
diverse speaker pool, and challenging background conditions.Comment: The dataset will be available for download from
http://www.robots.ox.ac.uk/~vgg/data/voxceleb/voxconverse.html . The
development set will be released in July 2020, and the test set will be
released in October 202
The Speed Submission to DIHARD II: Contributions & Lessons Learned
This paper describes the speaker diarization systems developed for the Second DIHARD Speech Diarization Challenge (DIHARD II) by the Speed team. Besides describing the system, which considerably outperformed the challenge baselines, we also focus on the lessons learned from numerous approaches that we tried for single and multi-channel systems. We present several components of our diarization system, including categorization of domains, speech enhancement, speech activity detection, speaker embeddings, clustering methods, resegmentation, and system fusion. We analyze and discuss the effect of each such component on the overall diarization performance within the realistic settings of the challenge
Speech Enhancement for Automatic Analysis of Child-Centered Audio Recordings
Analysis of child-centred daylong naturalist audio recordings has become a de-facto research protocol in the scientific study of child language development. The researchers are increasingly using these recordings to understand linguistic environment a child encounters in her routine interactions with the world. These audio recordings are captured by a microphone that a child wears throughout a day. The audio recordings, being naturalistic, contain a lot of unwanted sounds from everyday life which degrades the performance of speech analysis tasks. The purpose of this thesis is to investigate the utility of speech enhancement (SE) algorithms in the automatic analysis of such recordings. To this effect, several classical signal processing and modern machine learning-based SE methods were employed 1) as a denoiser for speech corrupted with additive noise sampled from real-life child-centred daylong recordings and 2) as front-end for downstream speech processing tasks of addressee classification (infant vs. adult-directed speech) and automatic syllable count estimation from the speech. The downstream tasks were conducted on data derived from a set of geographically, culturally, and linguistically diverse child-centred daylong audio recordings. The performance of denoising was evaluated through objective quality metrics (spectral distortion and instrumental intelligibility) and through the downstream task performance. Finally, the objective evaluation results were compared with downstream task performance results to find whether objective metrics can be used as a reasonable proxy to select SE front-end for a downstream task. The results obtained show that a recently proposed Long Short-Term Memory (LSTM)-based progressive learning architecture provides maximum performance gains in the downstream tasks in comparison with the other SE methods and baseline results. Classical signal processing-based SE methods also lead to competitive performance. From the comparison of objective assessment and downstream task performance results, no predictive relationship between task-independent objective metrics and performance of downstream tasks was found