Gauging the Effectiveness of Educational Technology Integration in Education: What the Best-Quality Meta-Analyses Tell Us

This chapter examines quantitative research in the literature of technology integration in education from the perspective of the meta-analyses of primary studies conducted from 1982 to 2015. The intent is to identify and review the best of these meta-analyses. Fifty-two meta-analyses were originally identified and evaluated for methodological quality using the Meta-Analysis Methodological Quality Review Guide (MMQRG), and the best 20 were selected and are included for review here. Some describe the effects of technology integration within specific content areas and some are more general. Technology integration in education is one of the most fluid areas of research, reflecting the incredible pace of the evolution of computer-based tools and applications. Just navigating through the vast primary empirical literature presents a real challenge to those interested in evaluating the educational effectiveness of technology. Systematic reviews in the field are numerous and quite diverse in their methodological quality, introducing potential bias in the interpretation of findings (Bernard RM, Borokhovski E, Schmid RF, Tamim RM. J Comput High Educ 26(3):183–209, 2014), thus bringing into question their applied value. This chapter identifies and reviews the best of these meta-analyses. In addition to overall statistical analyses of this collection, the findings of six of the most recent and best meta-analyses (after 2010) are summarized in more detail. The discussion focuses on the interpretation of the current findings, considers future alternatives to primary research in this area, and examines how meta-analysts might address them

The impact of word recognition from speech (WRS) proficiency level on interaction, task success and word learning: design implications for CALL to develop L2 WRS

This paper reports on task interaction, task success and word learning among second language (L2) learners of different levels of word recognition from speech (WRS) proficiency who used a CALL application previously shown to be effective in the development of L2 WRS. Participants (N = 65) were categorised into three levels of L2 WRS proficiency according to their baseline ability to accurately recognise the phonological form of words in connected speech. All participants undertook the same web-based intervention task which involved reconstructing spoken text into the written form in a number of stages. In terms of task interaction, lower WRS proficiency participants chose to listen to stimulus material more often and changed their efforts at reconstruction significantly more frequently than participants of higher WRS proficiency levels. In relation to task success, lower WRS proficiency learners were significantly less accurate overall and less immediate in their arrival at peak reconstruction accuracy than higher WRS proficiency participants. A comparison of gain scores also showed that participants of different L2 WRS proficiency experienced significantly different word learning outcomes after using the application, with moderate WRS proficiency participants significantly outperforming lower WRS proficiency level learners in relation to academic word learning. A synthesis of the results is used to recommend design features of CALL for WRS development which will more adequately cater to learners of different L2 WRS proficiency levels. The paper concludes with some avenues for future research and some implications for the integration of computer-mediated L2 WRS development as a component of L2 listening instruction