A meta-analysis of effectiveness studies on computer technology-supported language learning

With the aim of summarizing years of research comparing pedagogies for second/foreign language teaching supported with computer technology and pedagogy not-supported by computer technology, a meta-analysis was conducted of empirical research investigating language outcomes. Thirty-seven studies yielding 52 effect sizes were included, following a search of literature from 1970 to 2006 and screening of studies based on stated criteria. The differences in research designs required subdivision of studies, but overall results favored the technology-supported pedagogy, with a small, but positive and statistically significant effect size. Second/foreign language instruction supported by computer technology was found to be at least as effective as instruction without technology, and in studies using rigorous research designs the CALL groups outperformed the non-CALL groups. The analyses of instructional conditions, characteristics of participants, and conditions of the research design did not provide reliable results because of the small number of effect sizes representing each group. The meta-analysis results provide an empirically-based response to the questions of whether or not technology-supported pedagogies enhance language learning, and the process of conducting the meta-analysis pointed to areas in research methodology that would benefit from attention in future research

Validity Arguments for Diagnostic Assessment Using Automated Writing Evaluation

Two examples demonstrate an argument-based approach to validation of diagnostic assessment using automated writing evaluation (AWE). Criterion ®, was developed by Educational Testing Service to analyze students’ papers grammatically, providing sentence-level error feedback. An interpretive argument was developed for its use as part of the diagnostic assessment process in undergraduate university English for academic purposes (EAP) classes. The Intelligent Academic Discourse Evaluator (IADE) was developed for use in graduate EAP university classes, where the goal was to help students improve their discipline-specific writing. The validation for each was designed to support claims about the intended purposes of the assessments. We present the interpretive argument for each and show some of the data that have been gathered as backing for the respective validity arguments, which include the range of inferences that one would make in claiming validity of the interpretations, uses, and consequences of diagnostic AWE-based assessments

Call–English as a Second Language

CALL for English as a second language1 is an interdisciplinary area of inquiry which has been influenced primarily by educational technology (Reiser, 1987) but also by fields such as computational linguistics2 and recently by applied linguistics as well. These related fields contribute diverse epistemologies which shape CALL research questions and methods. The diversity in CALL research can also be explained in part by the current variety of approaches to CALL development and use. Through the 1970s and early 1980s, pedagogical objectives in CALL were focused primarily, although not exclusively, on improving specified areas of learner\u27s grammatical knowledge through approaches borrowed from educational technology (Hart 1981, Hope, Taylor and Pusack 1984, Wyatt 1984). Today, in contrast, CALL is used for a variety of pedagogical objectives through many different types of software such as microworlds (Coleman 1985, Papert 1980), grammar checkers (Hull, Ball, Fox, Levin and McCutchen 1987), pronunciation feedback systems (Anderson-Hseih 1994, Pennington 1991), intelligent tutoring systems (Chanier, Pengelly, Twidale and Self 1992), concordancer programs (Johns 1986, Tribble and Jones 1990), word processing (Pennington 1993), and software for computer-mediated communication (Kaye 1992). These diverse approaches to CALL are predicated on different beliefs about teaching and learning (Higgins 1995, Kenning and Kenning 1990, Sanders and Kenner 1983, Stevens 1992). Rather than reviewing these CALL philosophies, this article will focus on the evolution of research traditions dedicated to the empirical study of CALL use for ESL. Accordingly, the term CALL research is employed to refer to empirical research on the use of CALL