The Role of Theory in Quantitative Data Analysis

If you take theory and models seriously, then (a) you need to elaborate clearly for yourself ‘what counts’ and how things supposedly fit together, and (b) you must hold yourself accountable to data.… From my perspective, theory is – or should be – the lifeblood of the empirical scientist. (Schoenfeld, 2010, p. 105) The process of theorizing, collecting evidence, testing theory, revising theory, and then working through the cycle again is the basis of science. Theory is no less important when conducting a data analysis using quantitative methods. All statistical textbooks spend significant space on discussing the assumptions of statistical tests, the data requirements for a given estimation procedure, and the boundaries around the conclusions that can be drawn from results. Theory about the phenomenon that one is examining through quantitative data analysis is not only the driver for the methods used to collect evidence, but most importantly, the decisions made about how to model and test that theory

The Use of Meta-Analytic Statistical Significance Testing

Meta-analysis multiplicity, the concept of conducting multiple tests of statistical significance within one study, is an underdeveloped literature (Tendal, Nüesch, Higgins, Jüni, & Gøtzsche, 2011). We address this issue by considering how Type I errors can impact meta-analytic results, suggest how statistical power may be affected through the use of multiplicity corrections, and propose how meta-analysts should analyze multiple tests of statistical significance. The context for this study is a meta-review of meta-analyses published in two leading review journals in education and psychology. Our review of 130 meta-analyses revealed a strong reliance on statistical significance testing without considering of Type I errors or the use of multiplicity corrections. In order to provide valid conclusions, meta-analysts must consider these issues prior to conducting the study

“Do this in memory of me”: Examining Catholic subjectivity and teacher education

This paper is about Catholic subjectivity and teacher education. We explore multiple notions of Catholic subjectivity drawing from their Korean, Irish American, and Filipino-Polish heritages. Lived religion and memory writing are conceptual and methodological foundations of this paper. We examine multiple meanings of Catholic subjectivity via self-reflexive investigations on self, others, and the community in diverse sociocultural contexts. We argue that attention to teacher subjectivity with spirituality is deeply aligned with promoting the public good, such as advancing diversity and social justice issues in teacher education

Validation of the Employment Hope Scale: Measuring Psychological Self-Sufficiency Among Low-Income Jobseekers

The Employment Hope scale (EHS) was designed to measure the empowerment-based self-sufficiency (SS) outcome among low-income job-seeking clients. This measure captures the psychological SS dimension as opposed to the more commonly used economic SS in workforce development and employment support practice. The study validates the EHS and reports its psychometric properties. Method: An exploratory factor analysis (EFA) was conducted using an agency data from the Cara Program in Chicago, United States. The principal axis factor extraction process was employed to identify the factor structure. Results: EFA resulted in a 13-item two-factor structure with Factor 1 representing “Psychological Empowerment” and Factor 2 representing “Goal-Oriented Pathways.” Both factors had high internal consistency reliability and construct validity. Conclusions: While findings may be preliminary, this study found the EHS to be a reliable and valid measure, demonstrating its utility in assessing psychological SS as an empowerment outcome among low-income jobseekers

Quality of Research Evidence in Education: How Do We Know?

The persistence of inequitable education is the fundamental fact facing education researchers as we reflect on the quality and value of the evidence we produce (American Educational Research Association & National Academy of Education, 2020; Educational Opportunity Monitoring Project, 2020). As a field, we must critically examine what it means for us to develop increasingly sophisticated research tools and research design models while disparate outcomes along familiar lines of race and class continue apace. This issue’s importance has been laid bare by the COVID-19 pandemic and the global protests for racial justice in the wake of George Floyd’s murder. If our research endeavors are not effectively combating racism in education, providing help as our schools refashion themselves for remote and hybrid teaching, or supporting schools in other ways to address the myriad of equity gaps they face, then what are we doing? What are we generating evidence of and for

Risk Factors of Sexual Assault Victimization within the U.S. Military

INTRODUCTION: Sexual assault (SA) victimization affects thousands of service members in the military each year and identifying risk factors of SA is essential to inform prevention efforts. AIM: To synthesize literature on SA within the military to determine risk factors of SA incidence. METHODS: Risk factors from 6 epidemiological studies were compared via meta-analysis using R. Odds ratios and tests of heterogeneity were calculated to illustrate the collective odds of SA given each risk factor across the studies and to show variability. Odds ratios were calculated separately for risk factors only mentioned in one study. RESULTS: Women (OR =16.37), persons reporting sexual harassment during service (OR = 14.54), persons with a SA history (OR = 3.99), enlisted rank (OR = 2.47), non-married persons (OR = 2) and persons with no college experience were at greater risk of SA (OR = 1.32). Being White was found to be a protective factor (OR = 0.76). Our descriptive analysis found that experiencing stalking (OR = 11.84), being a sexual minority (OR = 2.15) or transgender increased the risk of SA (OR = 1.91). However, transgender womxn were at lower risk of SA than transgender mxn (OR = 0.42). DISCUSSION: It may be useful to develop tailored prevention programs for those identified as at risk according to our findings. Further, more needs to be done to address the environmental and cultural factors specific to the military that perpetuate SA incidence

The Question of School Resources and Student Achievement: A History and Reconsideration

One question posed continually over the past century of education research is to what extent school resources affect student outcomes. From the turn of the century to the present, a diverse set of actors, including politicians, physicians, and researchers from a number of disciplines, have studied whether and how money that is provided for schools translates into increased student achievement. The authors discuss the historical origins of the question of whether school resources relate to student achievement, and report the results of a meta- analysis of studies examining that relationship. They find that policymakers, researchers, and other stakeholders have addressed this question using diverse strategies. The way the question is asked, and the methods used to answer it, is shaped by history, as well by the scholarly, social, and political concerns of any given time. The diversity of methods has resulted in a body of literature too diverse and too inconsistent to yield reliable inferences through meta-analysis. The authors suggest that a collaborative approach addressing the question from a variety of disciplinary and practice perspectives may lead to more effective interventions to meet the needs of all students

AHRQ series on complex intervention systematic reviews-paper 5: advanced analytic methods.

BACKGROUND AND OBJECTIVE: Advanced analytic methods for synthesizing evidence about complex interventions continue to be developed. In this paper, we emphasize that the specific research question posed in the review should be used as a guide for choosing the appropriate analytic method. METHODS: We present advanced analytic approaches that address four common questions that guide reviews of complex interventions: (1) How effective is the intervention? (2) For whom does the intervention work and in what contexts? (3) What happens when the intervention is implemented? and (4) What decisions are possible given the results of the synthesis? CONCLUSION: The analytic approaches presented in this paper are particularly useful when each primary study differs in components, mechanisms of action, context, implementation, timing, and many other domains