Improving Searchability of Automatically Transcribed Lectures Through Dynamic Language Modelling

Recording university lectures through lecture capture systems is increasingly common. However, a single continuous audio recording is often unhelpful for users, who may wish to navigate quickly to a particular part of a lecture, or locate a specific lecture within a set of recordings. A transcript of the recording can enable faster navigation and searching. Automatic speech recognition (ASR) technologies may be used to create automated transcripts, to avoid the significant time and cost involved in manual transcription. Low accuracy of ASR-generated transcripts may however limit their usefulness. In particular, ASR systems optimized for general speech recognition may not recognize the many technical or discipline-specific words occurring in university lectures. To improve the usefulness of ASR transcripts for the purposes of information retrieval (search) and navigating within recordings, the lexicon and language model used by the ASR engine may be dynamically adapted for the topic of each lecture. A prototype is presented which uses the English Wikipedia as a semantically dense, large language corpus to generate a custom lexicon and language model for each lecture from a small set of keywords. Two strategies for extracting a topic-specific subset of Wikipedia articles are investigated: a naïve crawler which follows all article links from a set of seed articles produced by a Wikipedia search from the initial keywords, and a refinement which follows only links to articles sufficiently similar to the parent article. Pair-wise article similarity is computed from a pre-computed vector space model of Wikipedia article term scores generated using latent semantic indexing. The CMU Sphinx4 ASR engine is used to generate transcripts from thirteen recorded lectures from Open Yale Courses, using the English HUB4 language model as a reference and the two topic-specific language models generated for each lecture from Wikipedia

Multiple Media Correlation: Theory and Applications

This thesis introduces multiple media correlation, a new technology for the automatic alignment of multiple media objects such as text, audio, and video. This research began with the question: what can be learned when multiple multimedia components are analyzed simultaneously? Most ongoing research in computational multimedia has focused on queries, indexing, and retrieval within a single media type. Video is compressed and searched independently of audio, text is indexed without regard to temporal relationships it may have to other media data. Multiple media correlation provides a framework for locating and exploiting correlations between multiple, potentially heterogeneous, media streams. The goal is computed synchronization, the determination of temporal and spatial alignments that optimize a correlation function and indicate commonality and synchronization between media objects. The model also provides a basis for comparison of media in unrelated domains. There are many real-world applications for this technology, including speaker localization, musical score alignment, and degraded media realignment. Two applications, text-to-speech alignment and parallel text alignment, are described in detail with experimental validation. Text-to-speech alignment computes the alignment between a textual transcript and speech-based audio. The presented solutions are effective for a wide variety of content and are useful not only for retrieval of content, but in support of automatic captioning of movies and video. Parallel text alignment provides a tool for the comparison of alternative translations of the same document that is particularly useful to the classics scholar interested in comparing translation techniques or styles. The results presented in this thesis include (a) new media models more useful in analysis applications, (b) a theoretical model for multiple media correlation, (c) two practical application solutions that have wide-spread applicability, and (d) Xtrieve, a multimedia database retrieval system that demonstrates this new technology and demonstrates application of multiple media correlation to information retrieval. This thesis demonstrates that computed alignment of media objects is practical and can provide immediate solutions to many information retrieval and content presentation problems. It also introduces a new area for research in media data analysis

Robust automatic transcription of lectures

Automatic transcription of lectures is becoming an important task. Possible applications can be found in the fields of automatic translation or summarization, information retrieval, digital libraries, education and communication research. Ideally those systems would operate on distant recordings, freeing the presenter from wearing body-mounted microphones. This task, however, is surpassingly difficult, given that the speech signal is severely degraded by background noise and reverberation

Robust Automatic Transcription of Lectures

Die automatische Transkription von Vorträgen, Vorlesungen und Präsentationen wird immer wichtiger und ermöglicht erst die Anwendungen der automatischen Übersetzung von Sprache, der automatischen Zusammenfassung von Sprache, der gezielten Informationssuche in Audiodaten und somit die leichtere Zugänglichkeit in digitalen Bibliotheken. Im Idealfall arbeitet ein solches System mit einem Mikrofon das den Vortragenden vom Tragen eines Mikrofons befreit was der Fokus dieser Arbeit ist

Improving searchability of automatically transcribed lectures through dynamic language modelling

Recording university lectures through lecture capture systems is increasingly common. However, a single continuous audio recording is often unhelpful for users, who may wish to navigate quickly to a particular part of a lecture, or locate a specific lecture within a set of recordings. A transcript of the recording can enable faster navigation and searching. Automatic speech recognition (ASR) technologies may be used to create automated transcripts, to avoid the significant time and cost involved in manual transcription. Low accuracy of ASR-generated transcripts may however limit their usefulness. In particular, ASR systems optimized for general speech recognition may not recognize the many technical or discipline-specific words occurring in university lectures. To improve the usefulness of ASR transcripts for the purposes of information retrieval (search) and navigating within recordings, the lexicon and language model used by the ASR engine may be dynamically adapted for the topic of each lecture. A prototype is presented which uses the English Wikipedia as a semantically dense, large language corpus to generate a custom lexicon and language model for each lecture from a small set of keywords. Two strategies for extracting a topic-specific subset of Wikipedia articles are investigated: a naïve crawler which follows all article links from a set of seed articles produced by a Wikipedia search from the initial keywords, and a refinement which follows only links to articles sufficiently similar to the parent article. Pair-wise article similarity is computed from a pre-computed vector space model of Wikipedia article term scores generated using latent semantic indexing. The CMU Sphinx4 ASR engine is used to generate transcripts from thirteen recorded lectures from Open Yale Courses, using the English HUB4 language model as a reference and the two topic-specific language models generated for each lecture from Wikipedia. Three standard metrics – Perplexity, Word Error Rate and Word Correct Rate – are used to evaluate the extent to which the adapted language models improve the searchability of the resulting transcripts, and in particular improve the recognition of specialist words. Ranked Word Correct Rate is proposed as a new metric better aligned with the goals of improving transcript searchability and specialist word recognition. Analysis of recognition performance shows that the language models derived using the similarity-based Wikipedia crawler outperform models created using the naïve crawler, and that transcripts using similarity-based language models have better perplexity and Ranked Word Correct Rate scores than those created using the HUB4 language model, but worse Word Error Rates. It is concluded that English Wikipedia may successfully be used as a language resource for unsupervised topic adaptation of language models to improve recognition performance for better searchability of lecture recording transcripts, although possibly at the expense of other attributes such as readability

Identification and correction of speech repairs in the context of an automatic speech recognition system

Recent advances in automatic speech recognition systems for read (dictated) speech have led researchers to confront the problem of recognising more spontaneous speech. A number of problems, such as disfluencies, appear when read speech is replaced with spontaneous speech. In this work we deal specifically with what we class as speech-repairs. Most disfluency processes deal with speech-repairs at the sentence level. This is too late in the process of speech understanding. Speech recognition systems have problems recognising speech containing speech-repairs. The approach taken in this work is to deal with speech-repairs during the recognition process. Through an analysis of spontaneous speech the grammatical structure of speech- repairs was identified as a possible source of information. It is this grammatical structure, along with some pattern matching to eliminate false positives, that is used in the approach taken in this work. These repair structures are identified within a word lattice and when found result in a SKIP being added to the lattice to allow the reparandum of the repair to be ignored during the hypothesis generation process. Word fragment information is included using a sub-word pattern matching process and cue phrases are also identified within the lattice and used in the repair detection process. These simple, yet effective, techniques have proved very successful in identifying and correcting speech-repairs in a number of evaluations performed on a speech recognition system incorporating the repair procedure. On an un-seen spontaneous lecture taken from the Durham corpus, using a dictionary of 2,275 words and phoneme corruption of 15%, the system achieved a correction recall rate of 72% and a correction precision rate of 75%.The achievements of the project include the automatic detection and correction of speech-repairs, including word fragments and cue phrases, in the sub-section of an automatic speech recognition system processing spontaneous speech

Spirited media : revision, race, and revelation in nineteenth-century America

text"Spirited Media" analyzes distributed structures of authorship in the reform literature of the nineteenth-century United States. The literature that emerged out of reform movements like abolitionism often was a product of complex negotiations between speech and print, involving multiple people working across media in relationships that were sometimes collaborative, sometimes cooperative, and sometimes antagonistic. The cultural authority of print and individual authorship, often unquestioned in studies that focus on major or canonical figures of the nineteenth century, has tended to obscure some of this complexity. Moving from phonography, to Josiah Henson and Uncle Tom’s Cabin, to spiritualism, to Sojourner Truth and Walt Whitman, I consider four cases in which reporters, amanuenses, spirit mediums, and poets revived and remediated the voices of abolitionists, fugitive slaves, and figures from American history. By separating publication into events—speech, inscription, revision, and print—I show that "authorship" consisted of a series of interactions over time and across media, but that in the case of reform, the stakes for proving that authorship was a clear and indisputable characteristic of print were high. For abolitionist, African American, and spiritualist speakers and writers, authority depended on authorship, which in turn depended on the transparency of the print or the medium, or the perception of a direct relationship between speaker and reader. Like authorship, this transparency was constructed by a variety of social actors for whom the author was a key site of empowerment. It was authorized by appeals to revelation and race, two constructs often sidelined in media histories, yet central to discussions of society and politics in nineteenth-century America. Thinking of authorship as a distributed phenomenon disrupts models of the unitary subject and original genius, calling attention instead to uncanny acts of reading and writing in nineteenth-century literature. This dissertation argues that we should think about the transformative power of U.S. literature as located in revelation, not just creation, and in congregating people, not just representing them.Englis

Céad mίle fáilte: a corpus-based study of the development of a community of practice within the Irish hotel management training sector

This thesis examines the discourse of a unique third-level academic institution in order to identify the variety of linguistic features, which align it, first of all, to the higher education sector in general, but more specifically to a specific professional world where students are being educated for their future careers. Specifically, a college of hotel management education in the south of Ireland is the locus of research. Students complete a four-year Business Degree in International Hotel Management during which time they gain academic and theoretical knowledge along with practical industry experience during placement internships in the industry. Data collection using oral recordings spanned a twelve-month period and two academic years. This allowed for a comprehensive matrix of recording events encapsulating the full gamut of college academic life across the three years of student presence on campus. Recordings included a variety of hotel-specific and business lectures, practical working sessions, language classes and some miscellaneous events, thus creating a one-million word spoken corpus devoted to this sector. The primary research question concerns the identification and quantification of the discourse specific to this academic and professionally-oriented environment, using corpus linguistics methodologies. Parallel to and supported by this specialised linguistic repertoire lies the development of the emergent identity among the students themselves and their place and future careers within the international hotel management sector. This aspect will be analysed within Wenger’s (1998) framework of community of practice and Lave and Wenger’s (1991) initial theory of legitimate peripheral participation. In addition, an ethnographic lens will be employed to shed light on the day-to-day operations of this college and how the totality of this unique community, expressed through its discourse, but not only so, establishes and fosters an environment where the students develop their future professional identities supported by the academic professionals who are experienced industry practitioners in the field of international hotel management.N