How Do We Assess Civic Attitudes Toward Equal Rights? Data and Methodology

This open access thematic report identifies factors and conditions that can help schools and education systems promote tolerance in a globalized world. The IEA’s International Civic and Citizenship Study (ICCS) is a comparative research program designed to investigate the ways in which young people are prepared to undertake their roles as citizens, and provides a wealth of data permitting not only comparison between countries but also comparisons between schools within countries, and students within countries. Advanced analytical methods provide insights into relationships between students’ attitudes towards cultural diversity and the characteristics of the students themselves, their families, their teachers and school principals. The rich diversity of educational and cultural contexts in the 38 countries who participated in ICCS 2009 are also acknowledged and addressed. Readers interested in civic education and adolescents’ attitudes towards cultural diversity will find the theoretical perspectives explored engaging. For readers interested in methodology, the advanced analytical methods employed present textbook examples of how to address cross-cultural comparability of measurement instruments and multilevel data structures in international large-scale assessments (ILSA). Meanwhile, those interested in educational policy should find the identification and comparison of malleable factors across education systems that contribute to positive student attitudes towards cultural diversity a useful and thought-provoking resource

Differential Item Functioning in PISA Due to Mode Effects

One of the most important goals of the Programme for International Student Assessment (PISA) is assessing national changes in educational performance over time. These so-called trend results inform policy makers about the development of ability of 15-year-old students within a specific country. The validity of those trend results prescribes invariant test conditions. In the 2015 PISA survey, several alterations to the test administration were implemented, including a switch from paper-based assessments to computer-based assessments for most countries (OECD 2016a). This alteration of the assessment mode is examined by evaluating if the items used to assess trends are subject to differential item functioning across PISA surveys (2012 vs. 2015). Furthermore, the impact on the trend results due to the change in assessment mode of the Netherlands is assessed. The results show that the decrease reported for mathematics in the Netherlands is smaller when results are based upon a separate national calibration.</p

Assessing the adequacy of self-reported alcohol abuse measurement across time and ethnicity: cross-cultural equivalence across Hispanics and Caucasians in 1992, non-equivalence in 2001–2002

<p>Abstract</p> <p>Background</p> <p>Do estimates of alcohol abuse reflect true levels across United States Hispanics and non-Hispanic Caucasians, or does culturally-based, systematic measurement error (i.e., measurement bias) affect estimates? Likewise, given that recent estimates suggest alcohol abuse has increased among US Hispanics, the field should also ask, "Does cross-ethnic change in alcohol abuse across time reflect true change or does measurement bias influence change estimates?"</p> <p>Methods</p> <p>To address these questions, I used confirmatory factor analyses for ordered-categorical measures to probe for measurement bias on two large, standardized, nationally representative, US surveys of alcohol abuse conducted in 1992 and 2001–2002. In 2001–2002, analyses investigated whether 10 items operationalizing DSM-IV alcohol abuse provided equivalent measurement across Hispanic (<it>n </it>= 4,893) and non-Hispanic Caucasians (<it>n </it>= 16,480). In 1992, analyses examined whether a reduced 6 item item-set provided equivalent measurement among 834 Hispanic and 14,8335 non-Hispanic Caucasians.</p> <p>Results</p> <p>In 1992, findings demonstrated statistically significant measurement bias for two items. However, sensitivity analyses showed that item-level bias did not appreciably bias item-set based alcohol abuse estimates among this cohort. For 2001–2002, results demonstrated statistically significant bias for seven items, suggesting caution regarding the cross-ethnic equivalence of alcohol abuse estimates among the current US Hispanic population. Sensitivity analyses indicated that item-level differences <it>did </it>erroneously impact alcohol abuse rates in 2001–2002, underestimating rates among Hispanics relative to Caucasians.</p> <p>Conclusion</p> <p>1992's item-level findings suggest that estimates of drinking related social or legal problems may underestimate these specific problems among Hispanics. However, impact analyses indicated no appreciable effect on alcohol abuse estimates resulting from the item-set. Efforts to monitor change in alcohol abuse diagnoses among the Hispanic community can use 1992 estimates as a valid baseline. In 2001–2002, item-level measurement bias on seven items did affect item-set based estimates. Bias underestimated Hispanics' self-reported alcohol abuse levels relative to non-Hispanic Caucasians. Given the cross-ethnic equivalence of 1992 estimates, bias in 2001–2002 speciously minimizes current increases in drinking behavior evidenced among Hispanics. Findings call for increased public health efforts among the Hispanic community and underscore the necessity for cultural sensitivity when generalizing measures developed in the majority to minorities.</p

Estimation of the correlation coefficient using the Bayesian Approach and its applications for epidemiologic research

BACKGROUND: The Bayesian approach is one alternative for estimating correlation coefficients in which knowledge from previous studies is incorporated to improve estimation. The purpose of this paper is to illustrate the utility of the Bayesian approach for estimating correlations using prior knowledge. METHODS: The use of the hyperbolic tangent transformation (ρ = tanh ξ and r = tanh z) enables the investigator to take advantage of the conjugate properties of the normal distribution, which are expressed by combining correlation coefficients from different studies. CONCLUSIONS: One of the strengths of the proposed method is that the calculations are simple but the accuracy is maintained. Like meta-analysis, it can be seen as a method to combine different correlations from different studies

Measurement invariance of the Internet Gaming Disorder Scale–Short-Form (IGDS9-SF) between Australia, the USA, and the UK

The Internet Gaming Disorder Scale-Short-Form (IGDS9-SF) is widely used to assess Internet Gaming Disorder behaviors. Investigating cultural limitations and implications in its applicability is imperative. One way to evaluate the cross-cultural feasibility of the measure is through measurement invariance analysis. The present study used Multigroup Confirmatory Factor Analysis (MGCFA) to examine the IGDS9-SF measurement invariance across gamers from Australia, the United States of America (USA), and the United Kingdom (UK). To accomplish this, 171 Australian, 463 USA, and 281 UK gamers completed the IGDS9-SF. Although results supported the one-factor structure of the IGD construct, they indicated cross-country variations in the strength of the relationships between the indicators and their respective factor (i.e., non-invariant loadings of items 1, 2, 5), and that the same scores may not always indicate the same level of IGD severity across the three groups (i.e., non-invariant intercepts for items 1, 5, 7, 9)

Measurement invariance of the nine-item Internet Gaming Disorder Scale (IGDS9-SF) across Albania, USA, UK, and Italy

The IGDS9-SF, which assesses Internet Gaming Disorder behaviors, has been validated in a number of countries (Portugal, Italy, Iran, Slovenia), although the psychometric equivalence of the instrument has been assessed only across Australia, the USA, the UK, and India. This research aimed at providing further cross-cultural insights into IGD by assessing the factorial structure of the IGDS9-SF in Albania and investigating its measurement invariance across Albanian, Italian, American, and British gamers. Multi-Group Confirmatory Factor Analyses were performed on a sample of 1411 participants from Albania (n=228), USA (n=237), the UK (n=275), and Italy (n=671). The CFAs confirmed the single-factor structure in the four countries. Measurement invariance supported the configural invariance and partially supported the metric and scalar invariance. Overall, the findings provided evidence for the underlying factor assessing IGD across the countries, although the specific meaning of the construct was non-identical

Differential Item Functioning on Antisocial Behavior Scale Items for Adolescents and Young Adults from Single-Parent and Two-Parent Families

We investigated measurement equivalence in two antisocial behavior scales (i.e., one scale for adolescents and a second scale for young adults) by examining differential item functioning (DIF) for respondents from single-parent (n = 109) and two-parent families (n = 447). Even though one item in the scale for adolescents and two items in the scale for young adults showed significant DIF, the two scales exhibited non-significant differential test functioning (DTF). Both uniform and nonuniform DIF were investigated and examples of each type were identified. Specifically, uniform DIF was exhibited in the adolescent scale whereas nonuniform DIF was shown in the young adult scale. Implications of DIF results for assessment of antisocial behavior, along with strengths and limitations of the study, are discussed

Methodological issues in cross-cultural research

Regardless of whether the research goal is to establish cultural universals or to identify and explain cross-cultural differences, researchers need measures that are comparable across different cultures when conducting cross-cultural studies. In this chapter, we describe two major strategies for enhancing cross-cultural comparability. First, we discuss a priori methods to ensure the comparability of data in cross-cultural surveys. In particular, we review findings on cross-cultural differences based on the psychology of survey response and provide suggestions on how to deal with these cultural differences in the survey design stage. Second, we discuss post hoc methods to ascertain data comparability and enable comparisons in the presence of threats to equivalence