Automated Error Detection for Developing Grammar Proficiency of ESL Learners

Thanks to natural language processing technologies, computer programs are actively being used not only for holistic scoring, but also for formative evaluation of writing. CyWrite is one such program that is under development. The program is built upon Second Language Acquisition theories and aims to assist ESL learners in higher education by providing them with effective formative feedback to facilitate autonomous learning and improvement of their writing skills. In this study, we focus on CyWrite’s capacity to detect grammatical errors in student writing. We specifically report on (1) computational and pedagogical approaches to the development of the tool in terms of students’ grammatical accuracy, and (2) the performance of our grammatical analyzer. We evaluated the performance of CyWrite on a corpus of essays written by ESL undergraduate students with regards to four types of grammatical errors: quantifiers, subject-verb agreement, articles, and run-on sentences. We compared CyWrite’s performance at detecting these errors to the performance of a well-known commercially available AWE tool, Criterion. Our findings demonstrated better performance metrics of our tool as compared to Criterion, and a deeper analysis of false positives and false negatives shed light on how CyWrite’s performance can be improved

Optativ i imperativ v funkcii zapreta na dejstvie v gotskom jazyke (= Optative and imperative prohibitions in Gothic)

In the Greek original text of the New Testament, prohibitions are expressed with two negative verb forms: the present imperative and the aorist conjunctive. The grammatical equivalents of these two forms in Gothic are the imperative and the present optative, respectfully. However, we do not find a one-to-one verb form mapping from Greek onto Gothic in Wulfila\u27s translation of the Bible. Statistically, in prohibition contexts in Gothic, the optative is more frequently used than the imperative, while in Greek, on the contrary, the imperative is more frequent than the conjunctive. In this article, we argue that Greek and Gothic differ in the grammatical functions of verb moods used to express prohibitions. Through an analysis of a representative sample of contexts, we demonstrate that, while in Greek the distinction between the present imperative and the aorist conjunctive is aspectual, in Gothic the distinction between the imperative and the present optative is modal. Specifically, the Gothic optative is used when the prohibited actions and states are potential (i.e., upcoming, possible, undesirable, non-recommended, etc.); the Gothic imperative, on the other hand, is used to express prohibitions of real (vs. potential) actions, from the speaker\u27s viewpoint, and encouragement that such actions be terminated. In terms of the aspect, the former actions and states in many cases are general and universal (vs. occasion-specific), while the latter can be one-time, repeated, or ongoing actions. In the Gothic language of the Epistles, the optative mood is further specialized in expressing the instructional speech acts (teachings, recommendations, commandments, which are directed towards an unlimited and indefinite circle of addressees) and is mainly used in paraenetic contexts, wherein in the Greek original both the imperative and the conjunctive forms are used

The affordances of process-tracing technologies for supporting L2 writing instruction

The research literature on L2 writing processes contains a multitude of insights that could inform writing instruction, but writing teachers are constrained in their capacity to make use of these insights insofar as they lack detailed information about how their students actually engage in the processes of writing. At the same time, writing-process researchers have been using powerful technologies that are potentially applicable in educational settings to trace writers’ process engagement—namely, keystroke-logging and eye-tracking. In this article, we describe a pilot effort to integrate these technologies into L2 writing instruction with college-level ESL students. In addition to illustrating three key affordances of these technologies that emerged from the piloting, we discuss the conceptual framework that informed our efforts as well as challenges that will need to be addressed to facilitate further integration of process tracing into L2 writing pedagogy

Exploring the potential of process-tracing technologies to support assessment for learning of L2 writing

Assessment for learning (AfL) seeks to support instruction by providing information about students’ current state of learning, the desired end state of learning, and ways to close the gap. AfL of second-language (L2) writing faces challenges insofar as feedback from instructors tends to focus on written products while neglecting most of the processes that gave rise to them, such as planning, formulation, and evaluation. Meanwhile, researchers studying writing processes have been using keystroke logging (KL) and eye-tracking (ET) to analyze and visualize process engagement. This study explores whether such technologies can support more meaningful AfL of L2 writing. Two Chinese L1 students studying at a U.S. university who served as case studies completed a series of argumentative writing tasks while a KL-ET system traced their processes and then produced visualizations that were used for individualized tutoring. Data sources included the visualizations, tutoring-session transcripts, the participants’ assessed final essays, and written reflections. Findings showed the technologies, in combination with the assessment dialogues they facilitated, made it possible to (1) position the participants in relation to developmental models of writing; (2) identify and address problems with planning, formulation, and revision; and (3) reveal deep-seated motivational issues that constrained the participants’ learning

A system for adaptive high-variability segmental perceptual training: Implementation, effectiveness, transfer

Many types of L2 phonological perception are often difficult to acquire without instruction. These difficulties with perception may also be related to intelligibility in production. Instruction on perception contrasts is more likely to be successful with the use of phonetically variable input made available through computer-assisted pronunciation training. However, few computer-assisted programs have demonstrated flexibility in diagnosing and treating individual learner problems or have made effective use of linguistic resources such as corpora for creating training materials. This study introduces a system for segmental perceptual training that uses a computational approach to perception utilizing corpus-based word frequency lists, high variability phonetic input, and text-to-speech technology to automatically create discrimination and identification perception exercises customized for individual learners. The effectiveness of the system is evaluated in an experiment with pre- and post-test design, involving 32 adult Russian-speaking learners of English as a foreign language. The participants’ perceptual gains were found to transfer to novel voices, but not to untrained words. Potential factors underlying the absence of word-level transfer are discussed. The results of the training model provide an example for replication in language teaching and research settings

Automated extraction of revision events from keystroke data

Revision plays an important role in writing, and as revisions break down the linearity of the writing process, they are crucial in describing writing process dynamics. Keystroke logging and analysis have been used to identify revisions made during writing. Previous approaches include the manual annotation of revisions, building nonlinear S-notations, and the automated extraction of backspace keypresses. However, these approaches are time-intensive, vulnerable to construct, or restricted. Therefore, this article presents a computational approach to the automatic extraction of full revision events from keystroke logs, including both insertions and deletions, as well as the characters typed to replace the deleted text. Within this approach, revision candidates are first automatically extracted, which allows for a simplified manual annotation of revision events. Second, machine learning is used to automatically detect revision events. For this, 7120 revision events were manually annotated in a dataset of keystrokes obtained from 65 students conducting a writing task. The results showed that revision events could be automatically predicted with a relatively high accuracy. In addition, a case study proved that this approach could be easily applied to a new dataset. To conclude, computational approaches can be beneficial in providing automated insights into revisions in writing.</p

L2-ARCTIC: A Non-Native English Speech Corpus

In this paper, we introduce L2-ARCTIC, a speech corpus of non-native English that is intended for research in voice conversion, accent conversion, and mispronunciation detection. This initial release includes recordings from ten non-native speakers of English whose first languages (L1s) are Hindi, Korean, Mandarin, Spanish, and Arabic, each L1 containing recordings from one male and one female speaker. Each speaker recorded approximately one hour of read speech from the Carnegie Mellon University ARCTIC prompts, from which we generated orthographic and forced-aligned phonetic transcriptions. In addition, we manually annotated 150 utterances per speaker to identify three types of mispronunciation errors: substitutions, deletions, and additions, making it a valuable resource not only for research in voice conversion and accent conversion but also in computer-assisted pronunciation training. The corpus is publicly accessible at https://psi.engr.tamu.edu/l2-arctic-corpus/

Timed written picture naming in 14 European languages

We describe the Multilanguage Written Picture Naming Dataset. This gives trial-level data and time and agreement norms for written naming of the 260 pictures of everyday objects that compose the colorized Snodgrass and Vanderwart picture set (Rossion & Pourtois in Perception, 33, 217–236, 2004). Adult participants gave keyboarded responses in their first language under controlled experimental conditions (N = 1,274, with subsamples responding in Bulgarian, Dutch, English, Finnish, French, German, Greek, Icelandic, Italian, Norwegian, Portuguese, Russian, Spanish, and Swedish). We measured the time to initiate a response (RT) and interkeypress intervals, and calculated measures of name and spelling agreement. There was a tendency across all languages for quicker RTs to pictures with higher familiarity, image agreement, and name frequency, and with higher name agreement. Effects of spelling agreement and effects on output rates after writing onset were present in some, but not all, languages. Written naming therefore shows name retrieval effects that are similar to those found in speech, but our findings suggest the need for cross-language comparisons as we seek to understand the orthographic retrieval and/or assembly processes that are specific to written output

Combined deployable keystroke logging and eyetracking for investigating L2 writing fluency

Although fluency is an important sub-construct of language proficiency, it has not received as much attention in L2 writing research as complexity and accuracy have, in part due to the lack of methodological approaches for the analysis of large datasets of writing-process data. This article presents a method of time-aligned keystroke logging and eye tracking and reports an empirical study investigating L2 writing fluency through this method. Twenty-four undergraduate students at a private university in Turkey performed two writing tasks delivered through a web text editor with embedded keystroke logging and eye-tracking capabilities. Linear mixed-effects models were fit to predict indices of pausing and reading behaviors based on language status (L1 vs. L2) and linguistic context factors. Findings revealed differences between pausing and eye-fixation behavior in L1 and L2 writing processes. The paper concludes by discussing the affordances of the proposed method from the theoretical and practical standpoints