Video-based Sign Language Recognition without Temporal Segmentation

Millions of hearing impaired people around the world routinely use some variants of sign languages to communicate, thus the automatic translation of a sign language is meaningful and important. Currently, there are two sub-problems in Sign Language Recognition (SLR), i.e., isolated SLR that recognizes word by word and continuous SLR that translates entire sentences. Existing continuous SLR methods typically utilize isolated SLRs as building blocks, with an extra layer of preprocessing (temporal segmentation) and another layer of post-processing (sentence synthesis). Unfortunately, temporal segmentation itself is non-trivial and inevitably propagates errors into subsequent steps. Worse still, isolated SLR methods typically require strenuous labeling of each word separately in a sentence, severely limiting the amount of attainable training data. To address these challenges, we propose a novel continuous sign recognition framework, the Hierarchical Attention Network with Latent Space (LS-HAN), which eliminates the preprocessing of temporal segmentation. The proposed LS-HAN consists of three components: a two-stream Convolutional Neural Network (CNN) for video feature representation generation, a Latent Space (LS) for semantic gap bridging, and a Hierarchical Attention Network (HAN) for latent space based recognition. Experiments are carried out on two large scale datasets. Experimental results demonstrate the effectiveness of the proposed framework.Comment: 32nd AAAI Conference on Artificial Intelligence (AAAI-18), Feb. 2-7, 2018, New Orleans, Louisiana, US

Chinese elements : a bridge of the integration between Chinese -English translation and linguaculture transnational mobility

[Abstract] As the popularity of Chinese elements in the innovation of the translation part in Chinese CET, we realized that Chinese elements have become a bridge between linguaculture transnational mobility and Chinese-English translation.So, Chinese students translation skills should be critically improved; for example, on their understanding about Chinese culture, especially the meaning of Chinese culture. Five important secrets of skillful translation are introduced to improve students’ translation skills

Languaging in cyberspace: a case study of the effects of peer-peer collaborative dialogue on the acquisition of English idioms in task-based synchronous computer-mediated communication

Despite the growing interest in examining the link between peer-peer collaborative dialogue and second language (L2) development in recent years (Swain, Brooks, & Tocalli-Beller, 2002), much of the empirical work in this regard focused on face-to-face communication, leaving the operationalization of collaborative dialogue in text-based synchronous computer-mediated communication (SCMC) largely unexplored. In addition, while the bulk of the existing studies concerned L2 learners’ incidental learning of the linguistic structures they had difficulty with during their collaboration on communicative tasks (e.g., Watanabe & Swain, 2007), little is known about the connection between collaborative dialogue and second language acquisition (SLA) when L2 learners are faced with preselected language features that are intended for learning. Furthermore, L2 learners’ perspectives on collaborative dialogue and its contribution to L2 growth were for the most part ignored in the literature. This study seeks to address these gaps through the examination of English as a Second Language (ESL) learners’ collaboration on English idioms that are frequently used in academic discourse. Drawing on sociocultural SLA (Lantolf, 2000; Lantolf & Thorne, 2006) and the concept of “languaging” (Swain, 2006), it explores how episodes of collaborative dialogue are carried out during SCMC-based dyadic interaction, its association with SLA, and L2 learners’ opinions about its effectiveness. Sixteen intermediate learners of English as a Second language (ESL) enrolled in a college-level academic writing class participated in the current study. They filled out a questionnaire, collaborated on four English idiom learning tasks, completed a pretest and posttests, wrote reflective journals, reflected on their interaction in stimulated recalls, and responded to survey and interview questions. Within a case study design, this study drew on both quantitative and qualitative observations for data analysis. Specifically, qualitative analyses in the current study were conducted on the basis of questionnaire responses, discourse analysis of chat transcripts, and interview and stimulated recall transcripts. Quantitative measures consisted of descriptive statistics in the form of frequency counts, percentages of each type of communication strategy use and scaffolded assistance, gain scores on posttests, and Likert-scale survey results. Qualitative and quantitative results were triangulated to ensure the thoroughness and accuracy of interpretations. The results of this study revealed that in working together on the English idiom learning tasks, the participants engaged in the four patterns of dyadic interaction that have been documented in the literature on face-to-face dialogue (Storch, 2002), notably collaborative, expert/novice, dominant/dominant, and dominant/passive. Additionally, the specific patterns that the members of the dyad adopted seemed to be influenced by their perceptions of and attitudes toward the collaborative interaction and the nature of the tasks. Furthermore, during SCMC-based collaborative dialogue, the participants employed a wide array of communication strategies to manage and maintain their online exchanges, which facilitated their socialization into the academic discourse and communities. They also offered each other scaffolded assistance such as the use of contextual information while deciphering the meaning of the target idioms. Through mutual scaffolding, the participants advanced through their Zone of Proximal Development (ZPD) and achieved the transition from other-regulated to self-regulated performance. Finally, compared with the dyads with low posttest scores, the dyads with high posttest scores on the whole produced longer and more complex episodes of collaborative dialogue that clearly evidenced their cognitive processing of the target idioms, and the participants in general had a positive attitude toward the use of online chat for collaboration and target idiom learning through languaging

Phraseology in Corpus-based transaltion studies : stylistic study of two contempoarary Chinese translation of Cervantes's Don Quijote

The present work sets out to investigate the stylistic profiles of two modern Chinese versions of Cervantes???s Don Quijote (I): by Yang Jiang (1978), the first direct translation from Castilian to Chinese, and by Liu Jingsheng (1995), which is one of the most commercially successful versions of the Castilian literary classic. This thesis focuses on a detailed linguistic analysis carried out with the help of the latest textual analytical tools, natural language processing applications and statistical packages. The type of linguistic phenomenon singled out for study is four-character expressions (FCEXs), which are a very typical category of Chinese phraseology. The work opens with the creation of a descriptive framework for the annotation of linguistic data extracted from the parallel corpus of Don Quijote. Subsequently, the classified and extracted data are put through several statistical tests. The results of these tests prove to be very revealing regarding the different use of FCEXs in the two Chinese translations. The computational modelling of the linguistic data would seem to indicate that among other findings, while Liu???s use of archaic idioms has followed the general patterns of the original and also of Yang???s work in the first half of Don Quijote I, noticeable variations begin to emerge in the second half of Liu???s more recent version. Such an idiosyncratic use of archaisms by Liu, which may be defined as style shifting or style variation, is then analyzed in quantitative terms through the application of the proposed context-motivated theory (CMT). The results of applying the CMT-derived statistical models show that the detected stylistic variation may well point to the internal consistency of the translator in rendering the second half of Part I of the novel, which reflects his freer, more creative and experimental style of translation. Through the introduction and testing of quantitative research methods adapted from corpus linguistics and textual statistics, this thesis has made a major contribution to methodological innovation in the study of style within the context of corpus-based translation studies.Imperial Users onl

Phraseology in Corpus-Based Translation Studies: A Stylistic Study of Two Contemporary Chinese Translations of Cervantes's Don Quijote

The present work sets out to investigate the stylistic profiles of two modern Chinese versions of Cervantes’s Don Quijote (I): by Yang Jiang (1978), the first direct translation from Castilian to Chinese, and by Liu Jingsheng (1995), which is one of the most commercially successful versions of the Castilian literary classic. This thesis focuses on a detailed linguistic analysis carried out with the help of the latest textual analytical tools, natural language processing applications and statistical packages. The type of linguistic phenomenon singled out for study is four-character expressions (FCEXs), which are a very typical category of Chinese phraseology. The work opens with the creation of a descriptive framework for the annotation of linguistic data extracted from the parallel corpus of Don Quijote. Subsequently, the classified and extracted data are put through several statistical tests. The results of these tests prove to be very revealing regarding the different use of FCEXs in the two Chinese translations. The computational modelling of the linguistic data would seem to indicate that among other findings, while Liu’s use of archaic idioms has followed the general patterns of the original and also of Yang’s work in the first half of Don Quijote I, noticeable variations begin to emerge in the second half of Liu’s more recent version. Such an idiosyncratic use of archaisms by Liu, which may be defined as style shifting or style variation, is then analyzed in quantitative terms through the application of the proposed context-motivated theory (CMT). The results of applying the CMT-derived statistical models show that the detected stylistic variation may well point to the internal consistency of the translator in rendering the second half of Part I of the novel, which reflects his freer, more creative and experimental style of translation. Through the introduction and testing of quantitative research methods adapted from corpus linguistics and textual statistics, this thesis has made a major contribution to methodological innovation in the study of style within the context of corpus-based translation studies

Effects of short-term memory and content representation type on mobile language learning

Due to the rapid advancements in mobile communication and wireless technologies, many researchers and educators have started to believe that these emerging technologies can be leveraged to support formal and informal learning opportunities. Mobile language learning can be effectively implemented by delivering learning content through mobile phones. Because the screen size of mobile phones is limited, the presentation of materials using different Learning Content Representation (LCR) types is an issue that needs to be explored. This study addresses the issue of content adaptation in mobile language learning environments. Two dimensions have been taken into consideration to identify a promising solution: instructional strategies (LCR types: written annotation and pictorial annotation), and learners’ cognitive models (verbal and visual short-term memory). Our findings show that providing learning content with pictorial annotation in a mobile language learning environment can help learners with lower verbal and higher visual ability because such learners find it easier to learn content presented in a visual rather than in a verbal form. Providing learning content with both written and pictorial annotation can also help learners with both high verbal and high visual abilities. According to the Cognitive Load Theory, providing too much information may produce a higher cognitive load and lead to irritation and a lack of concentration. Our findings also suggest that providing just the basic learning materials is more helpful to learners with low verbal and visual abilities

Language Learning in Computer-Mediated Collaborative Writing

This dissertation investigated the impact of computer-mediated collaborative writing on Chinese EFL learners' language learning and writing skills through examining both writing products and processes. The potential influence of dyadic type and language proficiency was also explored through quantitative and qualitative analyses. The study adapted a quasi-experimental pre- and post-test design and involved 135 non-native English speakers (NNES) and 45 native English speakers (NES). Participants completed two writing tasks online either with a partner or individually over six weeks. The pre- and post-tests were designed to examine whether the language and writing skill developments from computer-mediated collaboration transferred into individual writing. Through analyses of the gain scores on language complexity, accuracy, fluency and overall performance, the findings revealed that intermediate proficiency learners had higher improvements than advanced proficiency learners over time. Although statistically significant differences were observed on fluency and overall performance across groups, the results did not show influence of writing modes, language proficiency and dyadic types on accuracy and complexity. Language-related episodes (LRE), non-language-related episodes (NLRE), and uptakes in the text-chat logs and the collaborative writing texts were used to measure potential noticing and language learning. The analysis showed that the advanced NNES-NES dyads had a higher frequency of LREs, NLREs, and correct immediate and delayed uptakes. The in-depth qualitative analysis of nine selected dyads revealed that advanced learners were more engaged in interactions in both NNES-NES and NNES-NNES dyads. However, intermediate learners showed more learning incidents, higher motivation and more confidence in NNES-NNES dyads than in NES-NNES dyads. The perception survey indicated that the NNES participants had a positive attitude to computer-mediated collaborative writing. The advanced learners in the NNES-NES dyads had the highest percentage of reporting self-perceived improvements and confidence in English writing. The findings of this dissertation research indicate that computer-mediated collaborative writing is beneficial to Chinese EFL learners when they are actively engaged in interactions during the writing process. The study also confirmed that dyadic type and language proficiency affect learner's performance, with intermediate learners more likely to benefit from collaboration with NNESs and advanced learners more likely to benefit from collaboration with NESs

Pushed-output instruction for vocabulary learning: Exploring differences in learning gains and lexical profiling

Previous research has shown that vocabulary can be learned through pushed-output activities. However, the few previous studies on the topic have mainly focused on the acquisition of nouns. Little is known about the acquisition of other parts of speech or about other components of lexical mastery achieved through pushed-output activities. This thesis examines the effectiveness of spoken pushed-output instruction on learning the multiple meaning senses of single-word verbs and phrasal verbs by presenting two classroom intervention studies. Study 1 explored differences between the effectiveness of spoken pushed-output and traditional vocabulary-focused instructions for learning polysemous single-word verbs and phrasal verbs. A between-subjects design was used, which included three conditions: no instruction, traditional vocabulary instruction and spoken pushed-output instruction. Both receptive and productive knowledge were investigated. The data were analysed using two approaches: (1) examining the receptive and productive vocabulary gains after instruction and (2) looking beyond the vocabulary gains by examining the lexical profile of the spoken production after instruction (i.e., overall text length, mean length of utterances, lexical diversity, lexical density and lexical sophistication). The findings indicated that with spoken pushed-output instruction, learners significantly improved not only in learning the multiple meaning senses of the target items but also in producing these meaning senses more fluently in longer, more lexically diverse, lexically denser and lexically sophisticated stretches of language. The results also indicated that single-word verbs could be learned at a similar rate to that of phrasal verbs. The results also showed that, except for the receptive gains of the first meaning sense, which had an advantage over the other meaning senses, no other differences among the three meaning senses emerged. This study demonstrated the advantage of spoken pushed-output instruction, justifying its use in the classroom. However, there are many different types of spoken pushed-output activities that may be implemented, making it logical to ask which are the most effective. Study 2 explored the effects of three different spoken pushed-output activities on learning polysemous single-word verbs and phrasal verbs: sentence reconstruction, listen-and-retell meaning, and picture description. The results indicated that all three activities resulted in similar recall scores but differed in their effectiveness for meaning recognition. The sentence reconstruction activity was found to be the most effective activity at the recognition level (as shown by the scores of the receptive test). The results also indicated that under similar instruction conditions, phrasal verbs are likely to be learned receptively and productively at a similar rate to single-word verbs. The results also showed that the first meaning sense was more easily recognised; however, no differences emerged neither in the recall scores nor in the mean length of utterances scores. Overall, the findings presented in the thesis support the use of spoken pushed-output instruction in the classroom for teaching single words and formulaic sequences. Further, the findings support the idea that, if the type and amount of instruction are controlled to be the same for single-word verbs and phrasal verbs, the learnability of these two types of items may be the same. While the findings cannot be easily generalised to other types of formulaic sequences, they do encourage further research on the teaching of formulaic sequences