Learner Corpus Research Meets Chinese as a Second Language Acquisition: Achievements and Challenges

The article sheds light on Chinese Learner Corpus Research (CLCR), emphasizing advances and lacks in this field. First, the paper describes the potentials of learner corpora in the investigation of learner language. The specificity of learner corpus data compared to learner data in Second Language Acquisition (SLA) studies will be also analyzed. Second, it provides an overview of Chinese learner corpus-based research and reviews existing L2 Chinese learner corpora. The paper highlights the lack of L2 Chinese learner corpora collecting data from Italian learners and discuss the challenges and the needs of compiling L2 Chinese corpora to conduct studies on the acquisition of L2 Chinese by learners whose L1 is other than English or an Asian language. This issue is addressed by taking into account recent projects integrating the LCR methodology with L2 Chinese studies for Italian-speaking learners. Finally, the paper encourages a concrete integration between the application of the methodological framework of LCR and the implementation of the theoretical interpretation of data of SLA research in the design of Chinese acquisitional studies

An Information-Theoretic Approach and Dataset for Probing Gender Stereotypes in Multilingual Masked Language Models

Bias research in NLP is a rapidly growing and developing field. Similar to CrowS-Pairs (Nangia et al., 2020), we assess gender bias in masked-language models (MLMs) by studying pairs of sentences with gender swapped person references. Most bias research focuses on and often is specific to English.Using a novel methodology for creating sentence pairs that is applicable across languages, we create, based on CrowS-Pairs, a multilingual dataset for English, Finnish, German, Indonesian and Thai.Additionally, we propose SJSD, a new bias measure based on Jensen–Shannon divergence, which we argue retains more information from the model output probabilities than other previously proposed bias measures for MLMs.Using multilingual MLMs, we find that SJSD diagnoses the same systematic biased behavior for non-English that previous studies have found for monolingual English pre-trained MLMs. SJSD outperforms the CrowS-Pairs measure, which struggles to find such biases for smaller non-English datasets

Undesirable biases in NLP: Averting a crisis of measurement

As Natural Language Processing (NLP) technology rapidly develops and spreads into daily life, it becomes crucial to anticipate how its use could harm people. However, our ways of assessing the biases of NLP models have not kept up. While especially the detection of English gender bias in such models has enjoyed increasing research attention, many of the measures face serious problems, as it is often unclear what they actually measure and how much they are subject to measurement error. In this paper, we provide an interdisciplinary approach to discussing the issue of NLP model bias by adopting the lens of psychometrics -- a field specialized in the measurement of concepts like bias that are not directly observable. We pair an introduction of relevant psychometric concepts with a discussion of how they could be used to evaluate and improve bias measures. We also argue that adopting psychometric vocabulary and methodology can make NLP bias research more efficient and transparent

Directional adposition use in English, Swedish and Finnish

Directional adpositions such as to the left of describe where a Figure is in relation to a Ground. English and Swedish directional adpositions refer to the location of a Figure in relation to a Ground, whether both are static or in motion. In contrast, the Finnish directional adpositions edellä (in front of) and jäljessä (behind) solely describe the location of a moving Figure in relation to a moving Ground (Nikanne, 2003). When using directional adpositions, a frame of reference must be assumed for interpreting the meaning of directional adpositions. For example, the meaning of to the left of in English can be based on a relative (speaker or listener based) reference frame or an intrinsic (object based) reference frame (Levinson, 1996). When a Figure and a Ground are both in motion, it is possible for a Figure to be described as being behind or in front of the Ground, even if neither have intrinsic features. As shown by Walker (in preparation), there are good reasons to assume that in the latter case a motion based reference frame is involved. This means that if Finnish speakers would use edellä (in front of) and jäljessä (behind) more frequently in situations where both the Figure and Ground are in motion, a difference in reference frame use between Finnish on one hand and English and Swedish on the other could be expected. We asked native English, Swedish and Finnish speakers’ to select adpositions from a language specific list to describe the location of a Figure relative to a Ground when both were shown to be moving on a computer screen. We were interested in any differences between Finnish, English and Swedish speakers. All languages showed a predominant use of directional spatial adpositions referring to the lexical concepts TO THE LEFT OF, TO THE RIGHT OF, ABOVE and BELOW. There were no differences between the languages in directional adpositions use or reference frame use, including reference frame use based on motion. We conclude that despite differences in the grammars of the languages involved, and potential differences in reference frame system use, the three languages investigated encode Figure location in relation to Ground location in a similar way when both are in motion. Levinson, S. C. (1996). Frames of reference and Molyneux’s question: Crosslingiuistic evidence. In P. Bloom, M.A. Peterson, L. Nadel & M.F. Garrett (Eds.) Language and Space (pp.109-170). Massachusetts: MIT Press. Nikanne, U. (2003). How Finnish postpositions see the axis system. In E. van der Zee & J. Slack (Eds.), Representing direction in language and space. Oxford, UK: Oxford University Press. Walker, C. (in preparation). Motion encoding in language, the use of spatial locatives in a motion context. Unpublished doctoral dissertation, University of Lincoln, Lincoln. United Kingdo

DEVELOPMENT AND VALIDATION OF A READING PROFICIENCY SCALE FOR CHINESE AS A SECOND LANGUAGE

This study develops and validates a Chinese as a Second Language (CSL) Reading Proficiency Scale. Reading ability is essential to the development of both basic communicative and cognitive academic language proficiency. The assessment of CSL reading needs further research. Test development starts with identifying the construct to measure. There is a long history of operationalizing theoretical constructs of language proficiency by developing proficiency scales. Existing reading proficiency scales are not directly applicable to CSL test development mainly due to the lack of theoretical underpinnings and/or empirical verification. Since Chinese uses a logographic and deep orthography, its unique linguistic features necessitate cognitive processes that do not exist in reading alphabetic languages. Very little is known about how to operationalize the construct of CSL reading for assessment purposes. Test developers need more guidance regarding: what component reading skills/knowledge should be measured and their relative levels of difficulty. This research fills in these gaps by developing the CSL Reading Proficiency Scale through a data-based approach. The scale development began with modeling the construct of CSL reading ability on the basis of dimension analysis of reading ability as conceptualized in existing theoretical works. Second, an operational descriptive scheme with four descriptive parameters was established. Third, a descriptor pool was developed through comprehensively documenting existing descriptors and creating new descriptors. Descriptors were rewritten to include these descriptive parameters and then validated qualitatively through expert review. Finally, fifty-seven surviving descriptors were compiled into a Likert-scale questionnaire. Ninety-five college-level CSL teachers evaluated 179 students’ reading ability against this questionnaire. To examine data collected by this questionnaire, confirmatory factor analysis was used to uncover the factor structure of the questionnaire and to classify descriptors into multiple subscales; Rasch analysis was then used to scale descriptors within each subscale separately and to screen descriptors that did not fit the model. Based on the findings, the CSL Reading Proficiency Scale was represented as a multi-level progression scale with three subscales. The Scale specifies characteristics of each level on the reading proficiency continuum. Besides its significance in test development, the Scale will benefit curriculum design, classroom instruction, and textbook compiling.Doctor of Educatio

Natural Language Processing for Teaching Ancient Languages

Konstantin Schulz shows various applications of natural language processing (NLP) to the field of Classics, especially to Latin texts. He addresses different levels of linguistic analysis while also highlighting educational benefits and important theoretical pitfalls, especially in vocabulary learning. NLP can solve some problems reasonably well, like tailoring exercises to the learners' current state of knowledge. However, some tasks still prove to be too difficult for machines at the moment, e.g. reliable and highly accurate parsing of syntax for historical languages

Task-Test: what lies beyond implementing a task-based assessment? Comparing learners' performance and unveiling learners' perception in a testing situation

Dissertação (mestrado) - Universidade Federal de Santa Catarina, Centro de Comunicação e Expressão, Programa de Pós-Graduação em Inglês: Estudos linguísticos e Literários, Florianópolis, 2014.Na área de Aquisição de Segunda Língua, a abordagem baseada em tarefas propõe um ensino comunicativo através do uso de tarefas (Skehan, 2003). Da mesma forma, pesquisadores e professores desta área explicam que a abordagem usada em sala de aula deve ser similar à maneira que os alunos são testados. Levando isso em consideração, é possível afirmar que, em uma sala de aula na qual a abordagem baseada em tarefas é aplicada, a testagem dos alunos deve ser conduzida da mesma forma. Assim, esse trabalho tem como objetivo investigar o impacto de um Teste-Tarefa, ou seja, um teste que contém elementos de uma tarefa levando em consideração os critérios estabelecidos por Ellis (2003), no desempenho de alunos de dois grupos distintos - grupo 1 que seguiu a abordagem estabelecida pelo livro-texto e grupo 2 que recebeu um tratamento seguindo a Abordagem Baseada em Tarefas. Além disso, a presente pesquisa tentou entender a relação entre o teste e a abordagem metodológica usada para ensino de segunda língua. Ainda, buscando um enfoque mais qualitativo, esse estudo objetivou pesquisar a percepção dos alunos sobre o Teste-Tarefa no sentido de saber se o mesmo é ou não uma ferramenta apropriada para medir seu aprendizado em L2. Trinta e dois participantes, divididos em dois grupos, responderam o Teste-Tarefa, um questionário sobre dados pessoais bem como outro sobre o teste em si. Em relação à comparação do desempenho dos alunos levando em consideração a acurácia, complexidade e outcome (resposta comunicativa de uma tarefa), Testes-T independentes foram utilizados. Quanto ao objetivo de entender a percepção dos participantes sobre o teste, uma avaliação qualitativa dos questionários foi feita. Resultados indicaram falta de significância estatística para ambas acuracia e complexidade enquanto que para a medida de outcome, a significância foi aproximada. Em relação aos questionários, os mesmos demonstraram que os alunos identificaram os elementos de uma tarefa no Teste-Tarefa e, em virtude desses resultados, o Teste-Tarefa foi considerado pelos alunos como uma ferramenta apropriada para medir seu aprendizado em L2. Levando em consideração tais implicações trazidas pela análise dos dados quantitativos e qualitativos, ênfase na importância de uma abordagem coerente para ensino e testagem é sugerida. Ademais, o Teste-Tarefa foi considerado uma ferramenta construtiva e positiva para testagem no contexto do ensino comunicativo.Abstract : Within the area of Second Language Acquisition (SLA), the Task Based Approach (TBA) proposes communicative teaching through the use of tasks (Skehan, 2003). At the same time, researchers in the area of SLA agree that the approach used by the teacher in class should be similar to the way the students are going to be tested. Bearing this in mind, it can be assumed that in TBA classroom, students' assessment should also follow TBA tenets. Therefore, this study aimed at investigating the impact of a TaskTest, that is, a written test which contained elements of a task according to Ellis' (2003) criteria, on students' language performance in two different groups - group 1 whose classes followed the textbook and group 2 which was exposed to Task Based lessons. Additionally, this research tried to understand the relationship of this test to the methodological approach used for second language teaching. Moreover, taking a more qualitative stance, this study aimed at unfolding students' perceptions of the Task-Test in terms of it being an appropriate tool for measuring their L2 acquisition. Thirty two participants divided in two groups completed the Task-Test and responded to a profile and a post task questionnaire. Research data were scrutinized qualitatively and quantitatively. As for comparing the participants' writing performance in terms of accuracy, complexity and outcome achievement, independent T-tests were run. As regards unfolding students? perception of the Task-Test, a qualitative appraisal of students' answers was made. The results of this study indicated lack of statistical significance for both accuracy and complexity measures and approached significance for the outcome measure. In relation to the questionnaires, learners' answers demonstrated that the students were able to identify task elements in the Task-Test and used these characteristics to explain why they considered the test an effective tool for measuring their L2 performance. Based on the results from both quantitative and qualitative analyses, emphasis on the importance of a coherent approach for teaching and testing may be suggested. Furthermore, the Task-Test was considered a constructive and positive tool for communicative assessment