Manipulating basic characteristics of the Rapid Automatized Naming task in search for its most reliable connections to reading performance

Introduction. Connections between Rapid Automatized Naming (RAN) task performance and reading are well documented. Primary empirical studies and meta-analyses established and described associations between specific RAN subtasks and reading outcomes. The cognitive nature of these associations, however, remains largely underexplored. This study attempts to address the issue by explicitly manipulating some critical characteristics of the RAN task (stimuli types, combinations, and familiarity) and conditions of its administration (attention demand) in search for factors that affect RAN performance and underlie its connections to reading competencies. Method. Ten modified RAN subtasks were created by manipulating type and familiarity of the stimuli, size of the stimuli source set, and demand to attention (cognitive controlled processing), involved in RAN performance. Measures of ballistic and efficiency-based automaticity, attention control, and reading rate were collected and analyzed using, ANOVA – with respect to performance on modified RAN subtasks, and correlational and multiple regression analyses – to address interrelations among major independent variables and their connections to reading rate. Results. The study found differential sensitivity of the RAN performance to the explored experimental manipulations. Specifically, significant main effects on naming speed were observed for stimuli type, stimuli familiarity and attention demand. RAN performance on most of the modified subtasks (seven out of ten) was significantly correlated with the measure of attention control, whereas only one correlation between RAN and measures of automaticity was statistically significant. Findings of multiple regression analyses confirmed this pattern of results. Attention factor explained substantially larger portion of variance in performance on modified RAN than both indices of automaticity combined. Reading rate was significantly correlated with bigram-based RAN (supposedly reflecting practice), and its correlations with other modified subtasks were higher for the elevated attention demand conditions, in one case exceeding significance level. Discussion. Understanding the cognitive nature of RAN is important for informing instructional practice of what reading skills might require special attention. This study explored specific conditions to which RAN performance may be especially sensitive. Modified RAN subtasks were markedly influenced by experimental manipulations, especially with regard to attention demand, indicating that attention, more than automaticity, could be a factor underlying naming speed as a predictor of reading

Detecting bias in meta-analyses of distance education research: big pictures we can rely on

© 2014, © 2014 Open and Distance Learning Association of Australia, Inc. This article has two interrelated purposes. The first is to explain how various forms of bias, if introduced during any stage of a meta-analysis, can provide the consumer with a misimpression of the state of a research literature. Five of the most important bias-producing aspects of a meta-analysis are presented and discussed. Second, armed with this information, we examine 15 meta-analyses of the literatures of distance education (DE), online learning (OL), and blended learning (BL), conducted from 2000 to 2014, with the intention of assessing potential sources of bias in each. All of these meta-analyses address the question: “How do students taking courses through DE, OL, and BL compare to students engaged in pure classroom instruction in terms of learning achievement outcomes?” We argue that questions asked by primary researchers must change to reflect issues that will drive improvements in designing and implementing DE, OL, and BL courses

An exploration of bias in meta-analysis: the case of technology integration research in higher education

© 2014, Springer Science+Business Media New York. This article contains a second-order meta-analysis and an exploration of bias in the technology integration literature in higher education. Thirteen meta-analyses, dated from 2000 to 2014 were selected to be included based on the questions asked and the presence of adequate statistical information to conduct a quantitative synthesis. The weighted random effects average was g++ = 0.393, p \u3c .000. The article goes on to report an assessment of the methodological quality of the thirteen studies based on Cooper’s (Research synthesis and meta-analysis: a step-by-step approach. Sage, Thousand Oaks, 2010) seven stages in the development of a meta-analysis. Two meta-analyses were found to have five out of seven stages where methodological flaws could potentially create biased results. Five meta-analyses contained two flawed stages and one contained one flawed stage. Four of the stages where methodological flaws can create bias are described in detail. The final section attempts to determine how much influence the methodological flaws exerted on the results of the second-order meta-analysis

A Meta-Analysis of Teacher and Student-Centered Practices and Processes in Undergraduate Science Education

This meta-analysis investigates the effects of four instructional dimensions rated on a scale from more Teacher-centered (T-C) to more Student-centered (S-C) plus several coded moderator variables on the achievement of undergraduate students in science education courses. More student-centered conditions served as the ‘treatment’ while more teacher-centered conditions were considered the ‘control.’ Hedges’ g, operationalized as the adjusted standardized differences between treatment and control means, served as the outcome measure. The weighted average difference between groups was g̅ = 0.34, k = 140 (random effects analysis), indicating an overall difference in favor of student-centered instruction. Out of four rated dimensions (Pacing, Teacher’s Role, Flexibility, and Adaptation) only Flexibility was significant in metaregression as a negative predictor of effect size. Two demographic variables (i.e., class size & subject matter), and one instructional moderator variables (i.e., technology use) were also significant when added to Flexibility, producing a model that accounted for 36% of total variation in effect size

Are contextual and designed student-student interaction treatments equally effective in distance education?

This systematic review draws from and builds upon the results of a meta-analysis of the achievement effects of three types of interaction treatments in distance education: student-student, student-teacher, and student-content (Bernard et al., Review of Educational Research, 79(3), 1243-1289, 2009). This follow-up study considers two forms of student-student interaction treatments, contextual interaction and designed interaction. Typical contextual interaction treatments contain the necessary conditions for student-student interaction to occur, but are not intentionally designed to create collaborative learning environments. By contrast, designed interaction treatments are intentionally implemented collaborative instructional conditions for increasing student learning. Our meta-analysis compared the effect of these two types of interaction treatments on student achievement outcomes. The results favored designed interaction treatments over contextual interaction treatments. Examples of designed interaction treatments and a discussion of study results and their potential implications for research and instruction in distance education and online learning are presented. © 2012 Copyright Open and Distance Learning Association of Australia, Inc

A Quantitative Synthesis of Outcomes of Educational Technology Approaches in K-12 Mathematics

Researchers have consistently examined the effects of educational technology on K-12 students’ mathematics learning, but previous meta-analyses suffered from lax inclusion criteria, resulting in inflated effect sizes, and also treated educational technology as a method rather than a delivery mechanism. The aim of the present meta-analysis was to examine the effects of ed-tech on mathematics learning employing rigorous inclusion criteria and examining the effects of both study-related and program-related moderators. Preliminary results from a random-effects model revealed a significant, small effect for mathematics learning (g = +.12). Statistically significant moderators included study design, sample size, blended as compared to fully tech, fidelity of implementation, and personalization. The findings support the positive effects of educational technology, and also provide considerations for program evaluators in terms of study design, and for program developers in terms of intervention design.The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, though Grant R305A210186 to Johns Hopkins University. The opinions expressed are those of the authors and do not represent views of the Institute of the U.S. Department of Education

Tablets for Teaching and Learning : A Systematic Review and Meta-Analysis

Tablets and smart mobile devices are the most recent addition to the long list of technological innovations believed to support and enhance the teaching process and learning process. This review aimed at going beyond the general hype around tablets and smart mobile devices to investigate the evidence supporting their use in educational contexts. To achieve this purpose, a systematic review of quantitative and qualitative research studies published since 2010 was completed. A rigorous review process resulted in the inclusion of 27 quantitative studies that were subjected to a full-scale meta-analytic procedure, and 41 qualitative research studies that were reviewed for substantive study characteristics. A significant average effect size was found for studies comparing tablet use contexts with no tablet use contexts (g+ = 0.23, k = 28). For studies comparing two different uses of tablets by students, the average effect size (g+ = 0.68, k = 12) showed a significant favouring of more student-centred pedagogical use of technology. Although not statistically tested, the findings also indicate that higher effect sizes are achieved when the devices are used with a student-centred approach rather than within teacher-led environments. Similarly, the qualitative literature review revealed that tablets and smart mobile devices are garnering positive perceptions within educational contexts, with the strongest support showing for the technologies’ effectiveness in particular tasks and when used within more student-active contexts. Finally, the review provides an overview of the Turkish Fatih Project as a case study and highlights the lessons learned

Systematically Searching Empirical Literature in the Social Sciences: Results from Two Meta-Analyses Within the Domain of Education

Introduction. This paper provides an overview of the information retrieval strategy employed for two meta-analyses, conducted by a systematic review team at Concordia University (Montreal, QC, Canada). Both papers draw on standards first articulated by H.M. Cooper and further developed by the Campbell Collaboration, which promote a comprehensive approach to systematically searching an extensive array of resources (bibliographic databases, print resources, citation indices, etc.) in order to locate both published and unpublished research. The goal is to verify if searching comprehensively through multiple resources retrieves studies that are unique, and hence, improve the overall representativeness of a diverse body of literature. We also analyze the sensitivity and specificity of the results by data source. Methods. In order to determine the source sensitivity, we consider percentage of results from each source retrieved for full-text review. In order to determine the source specificity, we derive a percentage from the total number of studies included in the final meta-analysis compared against the overall number of initial results found. Results. Results demonstrate the need to search beyond the subject-specific databases of a particular discipline as unique results can be found in many places. Databases for related disciplines provided 129 unique includes to each meta-analysis, and multidisciplinary databases provided 44 and 99 unique includes for the two meta-analyses in question respectively. Manual search techniques were much more sensitive and specific than electronic searches of databases and yield a higher percentage of final includes. Discussion. The results demonstrate the utility of a comprehensive information retrieval methodology like that proposed by the Campbell Collaboration, which goes beyond the main subject databases to locate the full range of information sources, including grey literature