Exploring Storybook Illustrations in Learning Word Meanings

This study explores storybook illustrations in learning word meanings among English learners in a university intensive language program. The impact of children’s literature on the comprehension and vocabulary development of second language children is well-documented. However, the use of the literature with adults still needs to be researched. Therefore, a mixed-method study was designed (1) to investigate whether readers who read an authentic illustrated story differed from those who read the same story without illustrations; and (2) to learn more about the readers’ process of learning words from storybook illustrations. Results suggest that illustrations play an important role in both comprehending the text and learning individual words, however issues related to the accessibility of the text and readers’ ability to use context should also be taken into consideration. The findings support prior research that the benefits of learning from context take time to become robust. The study suggests that illustrated storybooks provide a rich context for adults to infer word meanings and recommends children’s literature as an alternative source of reading in programs serving adult English learners

Exploring Storybook Illustrations in Learning Word Meanings

This study explores storybook illustrations in learning word meanings among English learners in a university intensive language program. The impact of children’s literature on the comprehension and vocabulary development of second language children is well-documented. However, the use of the literature with adults still needs to be researched. Therefore, a mixed-method study was designed (1) to investigate whether readers who read an authentic illustrated story differed from those who read the same story without illustrations; and (2) to learn more about the readers’ process of learning words from storybook illustrations. Results suggest that illustrations play an important role in both comprehending the text and learning individual words, however issues related to the accessibility of the text and readers’ ability to use context should also be taken into consideration. The findings support prior research that the benefits of learning from context take time to become robust. The study suggests that illustrated storybooks provide a rich context for adults to infer word meanings and recommends children’s literature as an alternative source of reading in programs serving adult English learners

A Baybayin word recognition system

Baybayin is a pre-Hispanic Philippine writing system used in Luzon island. With the effort in reintroducing the script, in 2018, the Committee on Basic Education and Culture of the Philippine Congress approved House Bill 1022 or the ”National Writing System Act,” which declares the Baybayin script as the Philippines’ national writing system. Since then, Baybayin OCR has become a field of research interest. Numerous works have proposed different techniques in recognizing Baybayin scripts. However, all those studies anchored on the classification and recognition at the character level. In this work, we propose an algorithm that provides the Latin transliteration of a Baybayin word in an image. The proposed system relies on a Baybayin character classifier generated using the Support Vector Machine (SVM). The method involves isolation of each Baybayin character, then classifying each character according to its equivalent syllable in Latin script, and finally concatenate each result to form the transliterated word. The system was tested using a novel dataset of Baybayin word images and achieved a competitive 97.9% recognition accuracy. Based on our review of the literature, this is the first work that recognizes Baybayin scripts at the word level. The proposed system can be used in automated transliterations of Baybayin texts transcribed in old books, tattoos, signage, graphic designs, and documents, among others

Identity and Language Socialization of Asian Transnational Adolescents across Communities of Practice: A Critical Narrative Study

A large percentage of the international secondary students in the United States come from Asian countries. Their enrollments are closely connected to the cultural, curricular, and extracurricular diversity of their American schools. Despite their contribution, stereotypical depictions of these students and deficit-informed research still abound in educational settings, leaving serious consequences for the social and academic well-being of the students. These problematic educational framings about Asian international students and the majoritarian narratives about them are mutually informative. Therefore, to counter the dominant discourses, this multimodal critical narrative study set out to recruit stories from a group of Asian transnational adolescent students to illustrate an alternative reality. Specifically, five transnational youths attending high schools in Maine shared their perspectives and experiences of identity construction and transformation as well as language learning and use in the context of navigating across their communities of practice (CoPs), i.e., the social, academic, and extracurricular communities they belonged to. With narrative inquiry guided by methodological pluralism, I collected a series of found and produced narrative artifacts as data from the five core informants and analyzed the data set through the following approaches: narrative positioning analysis, Labovian analysis, visual/multimodal analysis, portrait analysis, and thematic analysis. The outcome of these analyses are findings presented as a series of positioning profiles and thematic connections. Overall, the findings indicate a connection between these adolescent students’ social networks, CoP participation, and personal transformations. They position themselves as multifaceted, dynamic, dilemmatic, and oftentimes, in relation to the other members in their CoPs. In terms of language socialization, there is a shared understanding of communicative competence as multimodal and situated, and of CoP participation as conducive to the acquisition of the symbolic capital of English. When examined in context, these findings, though not meant to be one-size-fits-all, yield significant implications for educational research and practice targeted at this student population. Specifically, educators need to acknowledge the unequal access to participation and learning among students with different identity configurations. They will also benefit from tapping into the students’ CoP practice as well as transnational funds of knowledge as symbolic resources. This will allow them to develop a more diverse conception of competence, which in turn helps them provide affirming educational experiences to the transnational adolescents. Despite some limitations and barriers resulting from COVID-related circumstances during the data collection phase, this study is significant because the processes of the adolescent students’ storytelling in different modalities added complexity to the stories told by them and ended up being as important as the stories themselves when it came to illustrating an alternative reality of Asian transnational adolescent students’ identities and language socialization

Modularity and Neural Integration in Large-Vocabulary Continuous Speech Recognition

This Thesis tackles the problems of modularity in Large-Vocabulary Continuous Speech Recognition with use of Neural Network

On conditional random fields: applications, feature selection, parameter estimation and hierarchical modelling

There has been a growing interest in stochastic modelling and learning with complex data, whose elements are structured and interdependent. One of the most successful methods to model data dependencies is graphical models, which is a combination of graph theory and probability theory. This thesis focuses on a special type of graphical models known as Conditional Random Fields (CRFs) (Lafferty et al., 2001), in which the output state spaces, when conditioned on some observational input data, are represented by undirected graphical models. The contributions of thesis involve both (a) broadening the current applicability of CRFs in the real world and (b) deepening the understanding of theoretical aspects of CRFs. On the application side, we empirically investigate the applications of CRFs in two real world settings. The first application is on a novel domain of Vietnamese accent restoration, in which we need to restore accents of an accent-less Vietnamese sentence. Experiments on half a million sentences of news articles show that the CRF-based approach is highly accurate. In the second application, we develop a new CRF-based movie recommendation system called Preference Network (PN). The PN jointly integrates various sources of domain knowledge into a large and densely connected Markov network. We obtained competitive results against well-established methods in the recommendation field.On the theory side, the thesis addresses three important theoretical issues of CRFs: feature selection, parameter estimation and modelling recursive sequential data. These issues are all addressed under a general setting of partial supervision in that training labels are not fully available. For feature selection, we introduce a novel learning algorithm called AdaBoost.CRF that incrementally selects features out of a large feature pool as learning proceeds. AdaBoost.CRF is an extension of the standard boosting methodology to structured and partially observed data. We demonstrate that the AdaBoost.CRF is able to eliminate irrelevant features and as a result, returns a very compact feature set without significant loss of accuracy. Parameter estimation of CRFs is generally intractable in arbitrary network structures. This thesis contributes to this area by proposing a learning method called AdaBoost.MRF (which stands for AdaBoosted Markov Random Forests). As learning proceeds AdaBoost.MRF incrementally builds a tree ensemble (a forest) that cover the original network by selecting the best spanning tree at a time. As a result, we can approximately learn many rich classes of CRFs in linear time. The third theoretical work is on modelling recursive, sequential data in that each level of resolution is a Markov sequence, where each state in the sequence is also a Markov sequence at the finer grain. One of the key contributions of this thesis is Hierarchical Conditional Random Fields (HCRF), which is an extension to the currently popular sequential CRF and the recent semi-Markov CRF (Sarawagi and Cohen, 2004). Unlike previous CRF work, the HCRF does not assume any fixed graphical structures.Rather, it treats structure as an uncertain aspect and it can estimate the structure automatically from the data. The HCRF is motivated by Hierarchical Hidden Markov Model (HHMM) (Fine et al., 1998). Importantly, the thesis shows that the HHMM is a special case of HCRF with slight modification, and the semi-Markov CRF is essentially a flat version of the HCRF. Central to our contribution in HCRF is a polynomial-time algorithm based on the Asymmetric Inside Outside (AIO) family developed in (Bui et al., 2004) for learning and inference. Another important contribution is to extend the AIO family to address learning with missing data and inference under partially observed labels. We also derive methods to deal with practical concerns associated with the AIO family, including numerical overflow and cubic-time complexity. Finally, we demonstrate good performance of HCRF against rivals on two applications: indoor video surveillance and noun-phrase chunking

Off-line Thai handwriting recognition in legal amount

Thai handwriting in legal amounts is a challenging problem and a new field in the area of handwriting recognition research. The focus of this thesis is to implement Thai handwriting recognition system. A preliminary data set of Thai handwriting in legal amounts is designed. The samples in the data set are characters and words of the Thai legal amounts and a set of legal amounts phrases collected from a number of native Thai volunteers. At the preprocessing and recognition process, techniques are introduced to improve the characters recognition rates. The characters are divided into two smaller subgroups by their writing levels named body and high groups. The recognition rates of both groups are increased based on their distinguished features. The writing level separation algorithms are implemented using the size and position of characters. Empirical experiments are set to test the best combination of the feature to increase the recognition rates. Traditional recognition systems are modified to give the accumulative top-3 ranked answers to cover the possible character classes. At the postprocessing process level, the lexicon matching algorithms are implemented to match the ranked characters with the legal amount words. These matched words are joined together to form possible choices of amounts. These amounts will have their syntax checked in the last stage. Several syntax violations are caused by consequence faulty character segmentation and recognition resulting from connecting or broken characters. The anomaly in handwriting caused by these characters are mainly detected by their size and shape. During the recovery process, the possible word boundary patterns can be pre-defined and used to segment the hypothesis words. These words are identified by the word recognition and the results are joined with previously matched words to form the full amounts and checked by the syntax rules again. From 154 amounts written by 10 writers, the rejection rate is 14.9 percent with the recovery processes. The recognition rate for the accepted amount is 100 percent

Automatic Speech Recognition without Transcribed Speech or Pronunciation Lexicons

Rapid deployment of automatic speech recognition (ASR) in new languages, with very limited data, is of great interest and importance for intelligence gathering, as well as for humanitarian assistance and disaster relief (HADR). Deploying ASR systems in these languages often relies on cross-lingual acoustic modeling followed by supervised adaptation and almost always assumes that either a pronunciation lexicon using the International Phonetic Alphabet (IPA), and/or some amount of transcribed speech exist in the new language of interest. For many languages, neither requirement is generally true -- only a limited amount of text and untranscribed audio is available. This work focuses specifically on scalable techniques for building ASR systems in most languages without any existing transcribed speech or pronunciation lexicons. We first demonstrate how cross-lingual acoustic model transfer, when phonemic pronunciation lexicons do exist in a new language, can significantly reduce the need for target-language transcribed speech. We then explore three methods for handling languages without a pronunciation lexicon. First we examine the effectiveness of graphemic acoustic model transfer, which allows for pronunciation lexicons to be trivially constructed. We then present two methods for rapid construction of phonemic pronunciation lexicons based on submodular selection of a small set of words for manual annotation, or words from other languages for which we have IPA pronunciations. We also explore techniques for training sequence-to-sequence models with very small amounts of data by transferring models trained on other languages, and leveraging large unpaired text corpora in training. Finally, as an alternative to acoustic model transfer, we present a novel hybrid generative/discriminative semi-supervised training framework that merges recent progress in Energy Based Models (EBMs) as well as lattice-free maximum mutual information (LF-MMI) training, capable of making use of purely untranscribed audio. Together, these techniques enabled ASR capabilities that supported triage of spoken communications in real-world HADR work-flows in many languages using fewer than 30 minutes of transcribed speech. These techniques were successfully applied in multiple NIST evaluations and were among the top-performing systems in each evaluation