Automatic Speech Recognition without Transcribed Speech or Pronunciation Lexicons

Rapid deployment of automatic speech recognition (ASR) in new languages, with very limited data, is of great interest and importance for intelligence gathering, as well as for humanitarian assistance and disaster relief (HADR). Deploying ASR systems in these languages often relies on cross-lingual acoustic modeling followed by supervised adaptation and almost always assumes that either a pronunciation lexicon using the International Phonetic Alphabet (IPA), and/or some amount of transcribed speech exist in the new language of interest. For many languages, neither requirement is generally true -- only a limited amount of text and untranscribed audio is available. This work focuses specifically on scalable techniques for building ASR systems in most languages without any existing transcribed speech or pronunciation lexicons. We first demonstrate how cross-lingual acoustic model transfer, when phonemic pronunciation lexicons do exist in a new language, can significantly reduce the need for target-language transcribed speech. We then explore three methods for handling languages without a pronunciation lexicon. First we examine the effectiveness of graphemic acoustic model transfer, which allows for pronunciation lexicons to be trivially constructed. We then present two methods for rapid construction of phonemic pronunciation lexicons based on submodular selection of a small set of words for manual annotation, or words from other languages for which we have IPA pronunciations. We also explore techniques for training sequence-to-sequence models with very small amounts of data by transferring models trained on other languages, and leveraging large unpaired text corpora in training. Finally, as an alternative to acoustic model transfer, we present a novel hybrid generative/discriminative semi-supervised training framework that merges recent progress in Energy Based Models (EBMs) as well as lattice-free maximum mutual information (LF-MMI) training, capable of making use of purely untranscribed audio. Together, these techniques enabled ASR capabilities that supported triage of spoken communications in real-world HADR work-flows in many languages using fewer than 30 minutes of transcribed speech. These techniques were successfully applied in multiple NIST evaluations and were among the top-performing systems in each evaluation

Annual Report of the University, 2000-2001, Volumes 1-4

Message from the President Thank you for joining me in this look back over the past year at the University of New Mexico. It was a year filled with activity, accomplishment and challenge, and this is our opportunity to reflect back on that year. In 2000-2001 we engaged in a University-wide strategic planning process that called on the energies and talents of hundreds of individuals- faculty, staff, students and members of our broader community. The plan, which will be completed in Fall 2001, will serve as our roadmap for the future and will guide our efforts to capitalize on the opportunities and to meet the challenges of the next several years. This process has encouraged us to examine closely our mission and our values, who we are and what we aspire to become. It has given us reason to be proud of our past and cause to think seriously about how we must change in the future. While this was a year for looking ahead, it was also a year of significant accomplishment. For example, we launched a comprehensive set of programs designed to enrich the academic and social experiences of our undergraduate students. We began the implementation of Freshman Learning Communities where small cohorts of students study and learn together in a common set of courses under the guidance of a senior faculty scholar. We reorganized our advisement systems, we undertook the construction or renovation of student-centered facilities on campus, and we created new support systems to enhance student academic success. It was a year in which our support of faculty, staff and students was our highest priority. Through the support of the New Mexico Legislature, faculty and staff received significant salary increases. A new health benefits plan for graduate assistants was implemented. Our Staff as Students program enabled more than 40 staff members to obtain UNM degrees. And, a Center for Scholarship in Teaching and Learning was established to assist faculty in their efforts to develop more effective teaching skills. Finally, this was a year in which UNM dramatically expanded its role in the local community and throughout the state. Never before has the University been as active or as visible in meeting its public responsibility as it was in 2000-2001. From its active participation in economic development initiatives, to its involvement in K-12 educational improvement efforts, to its significant leadership role in health care delivery, UNM demonstrated its ability to help the state meet its most pressing social challenges. And, as UNM took on a more visible role in supporting the state\\u27s citizens, the support for UNM was returned in kind. This year, annual giving to the University rose to a record 35.3 million dollars, a 40% increase over just two years ago. All told, it has been a gratifying and successful year. However, we cannot allow our past accomplishments to mask the continued challenges facing this University. Neither will we allow these challenges to dominate our thinking and diminish out pride in what the University has achieved. So we will savor our successes and continue to move forward. As always, we thank you for sharing our dreams and for supporting the University of New Mexico. Sincerely, William C. Gordon, Presiden