Towards Effective Integration and Positive Impact of Automated Writing Evaluation in L2 Writing

The increasing dominance of English has elevated the need to develop an ability to effectively communicate in writing, and this has put a strain on second language education programs worldwide. Faced with time-consuming and copious commenting on student drafts and inspired by the promise of computerized writing assessment, many educational technology enthusiasts are looking to A WE [automated writing evaluation] as a silver bullet for language and literacy development (Warschauer & Ware, 2006, p. 175). This chapter reviews what AWE offers for learners and teachers and raises a number of controversies regarding A WE effectiveness with the underlying message that clear milestone targets need to be set with respect to A WE development, implementation, and evaluation in order to ensure positive impact of this technology on L2 writing. In support of this message, the chapter introduces an example-lADE, a prototype of contextbased A WE conceptualized and operationalized to address latent issues through a synthesis of theoretical premises and learning needs. Multifaceted empirical evaluation of lADE further provides insights into processes triggered by interaction with A WE technology and foregrounds a call for future research needed to inform effective application of AWE in L2 writing classrooms

Online tutor for research writing

English is the most prominent second language used in educational programs throughout the world. Unfortunately, there is a limitation of time and skill to guide students with learning the language and for evaluating their writings. Automated Writing Evaluation (AWE) tools would help in addressing this gap. In this thesis, I document a contribution to the field of Automated Writing Evaluation in the form of a new AWE tool called the Research Writing Tutor (RWT). The system design, user interface design, and features of this tool are introduced first, and then the findings obtained from an user evaluation study are reported. The website has been designed and developed to be user friendly. This tool could be of great use to graduate students and undergraduates in writing research reports, articles, and thesis or dissertations. Unlike most studies that concentrate on the accuracy of the AWE systems, this study aims at the usability and utility of the RWT in addition to the trust on automated systems

The role of feedback in the processes and outcomes of academic writing in english as a foreign language at intermediate and advanced levels

Providing feedback on students’ texts is one of the essential components of teaching second language writing. However, whether and to what extent students benefit from feedback has been an issue of considerable debate in the literature. While many researchers have stressed its importance, others expressed doubts about its effectiveness. Regardless of these continuing and well-established debates, instructors consider feedback as a worthwhile pedagogical practice for second language learning. Based on this premise, I conducted three experimental studies to investigate the role of written feedback in Myanmar and Hungarian tertiary EFL classrooms. Additionally, I studied syntactic features and language-related error patterns in Hungarian and Myanmar students’ writing. This attempt was made to understand how students with different writing proficiency acted upon teacher and automated feedback. The first study examined the efficacy of feedback on Myanmar students’ writing over a 13-week semester and how automated feedback provided by Grammarly could be integrated into writing instruction as an assistance tool for writing teachers. Results from pre-and post-tests demonstrated that students’ writing performance improved along the lines of four assessment criteria: task achievement, coherence and cohesion, grammatical range and accuracy, and lexical range and accuracy. Further results from a written feedback analysis revealed that the free version of Grammarly provided feedback on lower-level writing issues such as articles and prepositions, whereas teacher feedback covered both lower-and higher-level writing concerns. These findings suggested a potential for integrating automated feedback into writing instruction. As limited attention was given to how feedback influences other aspects of writing development beyond accuracy, the second study examined how feedback influences the syntactic complexity of Myanmar students’ essays. Results from paired sample t-tests revealed no significant differences in the syntactic complexity of students’ writing when the comparison was made between initial and revised texts and between pre-and post-tests. These findings suggested that feedback on students’ writing does not lead them to write less structurally complex texts despite not resulting in syntactic complexity gains. The syntactic complexity of students’ revised texts varied among high-, mid-, and low-achieving students. These variations could be attributed to proficiency levels, writing prompts, genre differences, and feedback sources. The rationale for conducting the third study was based on the theoretical orientation that differential success in learners’ gaining from feedback largely depended on their engagement with the feedback rather than the feedback itself. Along these lines of research, I examined Hungarian students’ behavioural engagement (i.e., students’ uptake or revisions prompted by written feedback) with teacher and automated feedback in an EFL writing course. In addition to the engagement with form-focused feedback examined in the first study, I considered meaning-focused feedback, as feedback in a writing course typically covers both linguistic and rhetorical aspects of writing. The results showed differences in feedback focus (the teacher provided form-and meaning-focused feedback) with unexpected outcomes: students’ uptake of feedback resulted in moderate to low levels of engagement with feedback. Participants incorporated more form-focused feedback than meaning-focused feedback into their revisions. These findings contribute to our understanding of students’ engagement with writing tasks, levels of trust, and the possible impact of students’ language proficiency on their engagement with feedback. Following the results that Myanmar and Hungarian students responded to feedback on their writing differently, I designed a follow-up study to compare syntactic features of their writing as indices of their English writing proficiency. In addition, I examined language-related errors in their texts to capture the differences in the error patterns in the two groups. Results from paired sample t-tests showed that most syntactic complexity indices distinguished the essays produced by the two groups: length of production units, sentence complexity, and subordination indices. Similarly, statistically significant differences were found in language-related error patterns in their texts: errors were more prevalent in Myanmar students’ essays. The implications for research and pedagogical practices in EFL writing classes are discussed with reference to the rationale for each study

A review and evaluation of the Langley Research Center's scientific and technical information program. Results of phase 1: Knowledge and attitudes survey, LaRC research personnel

The effectiveness of the Langley STI program was assessed using feedback obtained from Langley engineers and scientists. A survey research procedure was conducted in two stages. Personal interviews with 64 randomly selected Langley engineers and scientists were used to obtain information for questionnaire development. Data were then collected by means of the questionnaire which covered various aspects of the Langley STI program, utilized both open and closed ended questions and was pretested for finalization. The questions were organized around the six objectives for Phase 1. The completed questionnaires were analyzed. From the analysis of the data, recommendations were made for improving the Langley STI program