Automatic transcription of multi-genre media archives

This paper describes some recent results of our collaborative work on developing a speech recognition system for the automatic transcription or media archives from the British Broadcasting Corporation (BBC). The material includes a wide diversity of shows with their associated metadata. The latter are highly diverse in terms of completeness, reliability and accuracy. First, we investigate how to improve lightly supervised acoustic training, when timestamp information is inaccurate and when speech deviates significantly from the transcription, and how to perform evaluations when no reference transcripts are available. An automatic timestamp correction method as well as a word and segment level combination approaches between the lightly supervised transcripts and the original programme scripts are presented which yield improved metadata. Experimental results show that systems trained using the improved metadata consistently outperform those trained with only the original lightly supervised decoding hypotheses. Secondly, we show that the recognition task may benefit from systems trained on a combination of in-domain and out-of-domain data. Working with tandem HMMs, we describe Multi-level Adaptive Networks, a novel technique for incorporating information from out-of domain posterior features using deep neural network. We show that it provides a substantial reduction in WER over other systems including a PLP-based baseline, in-domain tandem features, and the best out-of-domain tandem features.This research was supported by EPSRC Programme Grant EP/I031022/1 (Natural Speech Technology).This paper was presented at the First Workshop on Speech, Language and Audio in Multimedia, August 22-23, 2013; Marseille. It was published in CEUR Workshop Proceedings at http://ceur-ws.org/Vol-1012/

The MGB Challenge: Evaluating Multi-genre Broadcast Media Recognition

This paper describes the Multi-Genre Broadcast (MGB) Challenge at ASRU 2015, an evaluation focused on speech recognition, speaker diarization, and "lightly supervised" alignment of BBC TV recordings. The challenge training data covered the whole range of seven weeks BBC TV output across four channels, resulting in about 1,600 hours of broadcast audio. In addition several hundred million words of BBC subtitle text was provided for language modelling. A novel aspect of the evaluation was the exploration of speech recognition and speaker diarization in a longitudinal setting - i.e. recognition of several episodes of the same show, and speaker diarization across these episodes, linking speakers. The longitudinal tasks also offered the opportunity for systems to make use of supplied metadata including show title, genre tag, and date/time of transmission. This paper describes the task data and evaluation process used in the MGB challenge, and summarises the results obtained

The 2015 Sheffield System for Transcription of Multi–Genre Broadcast Media

We describe the University of Sheffield system for participation in the 2015 Multi-Genre Broadcast (MGB) challenge task of transcribing multi-genre broadcast shows. Transcription was one of four tasks proposed in the MGB challenge, with the aim of advancing the state of the art of automatic speech recognition, speaker diarisation and automatic alignment of subtitles for broadcast media. Four topics are investigated in this work: Data selection techniques for training with unreliable data, automatic speech segmentation of broadcast media shows, acoustic modelling and adaptation in highly variable environments, and language modelling of multi-genre shows. The final system operates in multiple passes, using an initial unadapted decoding stage to refine segmentation, followed by three adapted passes: a hybrid DNN pass with input features normalised by speaker-based cepstral normalisation, another hybrid stage with input features normalised by speaker feature-MLLR transformations, and finally a bottleneck-based tandem stage with noise and speaker factorisation. The combination of these three system outputs provides a final error rate of 27.5% on the official development set, consisting of 47 multi-genre shows

Deepfake detection and low-resource language speech recognition using deep learning

While deep learning algorithms have made significant progress in automatic speech recognition and natural language processing, they require a significant amount of labelled training data to perform effectively. As such, these applications have not been extended to languages that have only limited amount of data available, such as extinct or endangered languages. Another problem caused by the rise of deep learning is that individuals with malicious intents have been able to leverage these algorithms to create fake contents that can pose serious harm to security and public safety. In this work, we explore the solutions to both of these problems. First, we investigate different data augmentation methods and acoustic architecture designs to improve automatic speech recognition performance on low-resource languages. Data augmentation for audio often involves changing the characteristic of the audio without modifying the ground truth. For example, different background noise can be added to an utterance while maintaining the content of the speech. We also explored how different acoustic model paradigms and complexity affect performance on low-resource languages. These methods are evaluated on Seneca, an endangered language spoken by a Native American tribe, and Iban, a low-resource language spoken in Malaysia and Brunei. Secondly, we explore methods to determine speaker identification and audio spoofing detection. A spoofing attack involves using either a text-to-speech voice conversion application to generate audio that mimic the identity of a target speaker. These methods are evaluated on the ASVSpoof 2019 Logical Access dataset containing audio generated using various methods of voice conversion and text-to-speech synthesis

Scene Analysis for Speech and Audio Recognition

Focuses on several different approaches to handling sound mixtures: computational auditory scene analysis, multicondition training, and parallel-model-based techniques such as HMM decomposition and multisource decoding

Evaluation of Student Interpreters Using Voice Recognition and Automatic Grammar Correction

The evaluation and assessment of student interpreters have long been an issue for interpreting programs. The balance between student practice throughput, the time and human cost of assessment, and the quality of feedback is notoriously difficult to achieve. Here we demonstrate a way to rapidly assess student Chinese-to-English interpreting performance using automatic speech recognition and grammar correction software. The assessment results are compared with human graders against a set of criteria for grammar, fidelity, register, and enunciation. The results show that the semiautomatic assessment process is less time-consuming, and can give adequate feedback for enunciation, grammar, and register. Student volunteers were able to maintain engagement over a three-month period with minimal intervention from the instructor, however, interest began to drop over the long term

Vocal emotions on the brain: the role of acoustic parameters and musicality

The human voice is a powerful transmitter of emotions. This dissertation addresses three main gaps in the field of vocal emotion perception. The first is the quantification of the relative contribution of fundamental frequency (F0) and timbre cues to the perception of different emotions and their associated electrophysiological correlates. Using parameter-specific voice morphing, the results show that both F0 and timbre carry unique information that allow emotional inferences, although F0 seems to be relatively more important overall. The electrophysiological data revealed F0- and timbre-specific modulations in several ERP components, such as the P200 and the N400. Second, it was explored how musicality affects the processing of emotional voice cues, by providing a review on the literature linking musicality to emotion perception and subsequently showing that musicians have a benefit in vocal emotion perception compared to non-musicians. The present data offer original insight into the special role of pitch cues: musicians outperformed non-musicians when emotions were expressed by the pitch contour only, but not when they were expressed by vocal timbre. Although the electrophysiological patterns were less conclusive, they imply that musicality may modulate brain responses to vocal emotions. Third, this work provides a critical reflection on parameter-specific voice morphing and its suitability to study the processing of vocal emotions. Distortions in voice naturalness resulting from extreme acoustic manipulations were identified as one of the major threats to the ecological validity of the stimulus material produced with this technique. However, the results suggested that while voice morphing does affect the perceived naturalness of stimuli, behavioral measures of emotion perception were found to be remarkably robust against these distortions. Thus, the present data advocate parameter-specific voice morphing as a valid tool for vocal emotional research

Using a low-bit rate speech enhancement variable post-filter as a speech recognition system pre-filter to improve robustness to GSM speech

Includes bibliographical references.Performance of speech recognition systems degrades when they are used to recognize speech that has been transmitted through GS1 (Global System for Mobile Communications) voice communication channels (GSM speech). This degradation is mainly due to GSM speech coding and GSM channel noise on speech signals transmitted through the network. This poor recognition of GSM channel speech limits the use of speech recognition applications over GSM networks. If speech recognition technology is to be used unlimitedly over GSM networks recognition accuracy of GSM channel speech has to be improved. Different channel normalization techniques have been developed in an attempt to improve recognition accuracy of voice channel modified speech in general (not specifically for GSM channel speech). These techniques can be classified into three broad categories, namely, model modification, signal pre-processing and feature processing techniques. In this work, as a contribution toward improving the robustness of speech recognition systems to GSM speech, the use of a low-bit speech enhancement post-filter as a speech recognition system pre-filter is proposed. This filter is to be used in recognition systems in combination with channel normalization techniques