The Longitudinal Measurement of Reasoning Abilities in Students With Special Educational Needs

Students with special educational needs in the area of learning (SEN-L) have learning disabilities that can lead to academic difficulties in regular schools. In Germany, these students are frequently enrolled in special schools providing specific training and support for these students. Because of their cognitive difficulties, it is unclear whether standard achievement tests that are typically administered in educational large-scale assessments (LSA) are suitable of students with SEN-L. The present study evaluated the psychometric properties of a short instrument for the assessment of reasoning abilities that was administered as part of a longitudinal LSA to German students from special schools (N = 324) and basic secondary schools (N = 338) twice within 6 years. Item response modeling demonstrated an essentially unidimensional scale for both school types. Few items exhibited systematic differential item functioning (DIF) between students with and without SEN-L, allowing for valid cross-group comparisons. However, change analyses across the two time points needed to account for longitudinal DIF among students with SEN-L. Overall, the cognitive test allowed for a valid measurement of reasoning abilities in students with SEN-L and comparative analyses regarding students without SEN-L. These results demonstrate the feasibility of incorporating students with SEN-L into educational LSAs

The longitudinal measurement of reasoning abilities in students with special educational needs

Students with special educational needs in the area of learning (SEN-L) have learning disabilities that can lead to academic difficulties in regular schools. In Germany, these students are frequently enrolled in special schools providing specific training and support for these students. Because of their cognitive difficulties, it is unclear whether standard achievement tests that are typically administered in educational large-scale assessments (LSA) are suitable of students with SEN-L. The present study evaluated the psychometric properties of a short instrument for the assessment of reasoning abilities that was administered as part of a longitudinal LSA to German students from special schools (N = 324) and basic secondary schools (N = 338) twice within 6 years. Item response modeling demonstrated an essentially unidimensional scale for both school types. Few items exhibited systematic differential item functioning (DIF) between students with and without SEN-L, allowing for valid cross-group comparisons. However, change analyses across the two time points needed to account for longitudinal DIF among students with SEN-L. Overall, the cognitive test allowed for a valid measurement of reasoning abilities in students with SEN-L and comparative analyses regarding students without SEN-L. These results demonstrate the feasibility of incorporating students with SEN-L into educational LSAs. (DIPF/Orig.

Setting a standard for low reading proficiency. A comparison of the bookmark procedure and constrained mixture Rasch model

In order to draw pertinent conclusions about persons with low reading skills, it is essential to use validated standard-setting procedures by which they can be assigned to their appropriate level of proficiency. Since there is no standard-setting procedure without weaknesses, external validity studies are essential. Traditionally, studies have assessed validity by comparing different judgement-based standard-setting procedures. Only a few studies have used model-based approaches for validating judgement-based procedures. The present study addressed this shortcoming and compared agreement of the cut score placement between a judgement-based approach (i.e., Bookmark procedure) and a model-based one (i.e., constrained mixture Rasch model). This was performed by differentiating between individuals with low reading proficiency and those with a functional level of reading proficiency in three independent samples of the German National Educational Panel Study that included students from the ninth grade (N = 13,897) as well as adults (Ns = 5,335 and 3,145). The analyses showed quite similar mean cut scores for the two standard-setting procedures in two of the samples, whereas the third sample showed more pronounced differences. Importantly, these findings demonstrate that model-based approaches provide a valid and resource-efficient alternative for external validation, although they can be sensitive to the ability distribution within a sample. (DIPF/Orig.

Data for Psychological Research in the Educational Field: Spotlights, Data Infrastructures, and Findings from Research

In recent years, there has been a growing emphasis on the importance of open data and data sharing in scientific research (Nosek et al., 2015; van der Zee & Reich, 2018). However, in the educational field, access to FAIR (findable, accessible, interoperable, and reusable) data remains a significant challenge (Wilkinson et al., 2016). This special collection addresses this challenge by highlighting psychological data in educational research and showcasing examples of data that have been shared and made available to the scientific community in accordance with FAIR principles. With this special collection, we aim to explicitly encourage the use of shared research data for individual research projects

A systematic review of attitudes, anxiety, acceptance, and trust towards social robots

As social robots become more common, there is a need to understand how people perceive and interact with such technology. This systematic review seeks to estimate people’s attitudes toward, trust in, anxiety associated with, and acceptance of social robots; as well as factors that are associated with these beliefs. Ninety-seven studies were identified with a combined sample of over 13,000 participants and a standardized score was computed for each in order to represent the valence (positive, negative, or neutral) and magnitude (on a scale from 1 to − 1) of people’s beliefs about robots. Potential moderating factors such as the robots’ domain of application and design, the type of exposure to the robot, and the characteristics of potential users were also investigated. The findings suggest that people generally have positive attitudes towards social robots and are willing to interact with them. This finding may challenge some of the existing doubt surrounding the adoption of robotics in social domains of application but more research is needed to fully understand the factors that influence attitudes

Crowdsourcing hypothesis tests: Making transparent how design choices shape research results

To what extent are research results influenced by subjective decisions that scientists make as they design studies? Fifteen research teams independently designed studies to answer fiveoriginal research questions related to moral judgments, negotiations, and implicit cognition. Participants from two separate large samples (total N > 15,000) were then randomly assigned to complete one version of each study. Effect sizes varied dramatically across different sets of materials designed to test the same hypothesis: materials from different teams renderedstatistically significant effects in opposite directions for four out of five hypotheses, with the narrowest range in estimates being d = -0.37 to +0.26. Meta-analysis and a Bayesian perspective on the results revealed overall support for two hypotheses, and a lack of support for three hypotheses. Overall, practically none of the variability in effect sizes was attributable to the skill of the research team in designing materials, while considerable variability was attributable to the hypothesis being tested. In a forecasting survey, predictions of other scientists were significantly correlated with study results, both across and within hypotheses. Crowdsourced testing of research hypotheses helps reveal the true consistency of empirical support for a scientific claim.</div

Balanced and positively worded personality short-forms: Mini-IPIP validity and cross-cultural invariance

Background The Mini-IPIP scales (Donellan et al., 2006) are possibly one of the most commonly used short inventories for measuring the Big Five Factors of personality. In this study, we aimed to investigate the psychometric properties of two Mini-IPIP Spanish short forms, one balanced and one positively wording (PW). Method Two samples, one from native Spanish speakers and another from native English speakers, made up a total of 940 participants in this study. The short forms were translated and adapted based on international guidelines. Reliability (internal and composite) and validity analyses (construct ESEM, concurrent, predictive and cross-cultural invariance through multi-group factorial models) were performed. Results For both the balanced scale and the PW one, modeling a method factor was not relevant. The reliability and validity indices of both forms were according to theory and prior studies’ findings: (a) personality factors were medium-high related to affective factors; (b) personality factors were less related to life satisfaction than affective factors; (c) life satisfaction was medium-high related to affective factors; (d) neuroticism appeared mainly related to all criteria variables; and (e) an acceptable level of invariance was achieved with regard to the English version. Discussion This study contributes to research on personality assessment by providing the first evidence regarding the psychometric properties of a PW short measure. These results suggest that PW short scales of personality used after data screening techniques may be appropriate for future studies (e.g., cross-cultural, content validity)

Adaptation and psychometric properties of the ISPCAN Child Abuse Screening Tool for use in trials (ICAST-Trial) among South African adolescents and their primary caregivers

© 2018 The Authors. Child abuse prevention research has been hampered by a lack of validated multi-dimensional non-proprietary instruments, sensitive enough to measure change in abuse victimization or behavior. This study aimed to adapt the ICAST child abuse self-report measure (parent and child) for use in intervention studies and to investigate the psychometric properties of this substantially modified tool in a South African sample. First, cross-cultural and sensitivity adaptation of the original ICAST tools resulted in two preliminary measures (ICAST-Trial adolescents: 27 items, ICAST-Trial caregivers: 19 items). Second, ICAST-Trial data from a cluster randomized trial of a parenting intervention for families with adolescents (N = 1104, 552 caregiver-adolescent dyads) was analyzed. Confirmatory factor analysis established the hypothesized 6-factor (adolescents) and 4-factor (caregivers) structure. Removal of two items for adolescents and five for caregivers resulted in adequate model fit. Concurrent criterion validity analysis confirmed hypothesized relationships between child abuse and adolescent and caregiver mental health, adolescent behavior, discipline techniques and caregiver childhood abuse history. The resulting ICAST-Trial measures have 25 (adolescent) and 14 (caregiver) items respectively and measure physical, emotional and contact sexual abuse, neglect (both versions), and witnessing intimate partner violence and sexual harassment (adolescent version). The study established that both tools are sensitive to measuring change over time in response to a parenting intervention. The ICAST-Trial should have utility for evaluating the effectiveness of child abuse prevention efforts in similar socioeconomic contexts. Further research is needed to replicate these findings and examine cultural appropriateness, barriers for disclosure, and willingness to engage in child abuse research

Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty

This study explores how researchers’ analytical choices affect the reliability of scientific findings. Most discussions of reliability problems in science focus on systematic biases. We broaden the lens to emphasize the idiosyncrasy of conscious and unconscious decisions that researchers make during data analysis. We coordinated 161 researchers in 73 research teams and observed their research decisions as they used the same data to independently test the same prominent social science hypothesis: that greater immigration reduces support for social policies among the public. In this typical case of social science research, research teams reported both widely diverging numerical findings and substantive conclusions despite identical start conditions. Researchers’ expertise, prior beliefs, and expectations barely predict the wide variation in research outcomes. More than 95% of the total variance in numerical results remains unexplained even after qualitative coding of all identifiable decisions in each team’s workflow. This reveals a universe of uncertainty that remains hidden when considering a single study in isolation. The idiosyncratic nature of how researchers’ results and conclusions varied is a previously underappreciated explanation for why many scientific hypotheses remain contested. These results call for greater epistemic humility and clarity in reporting scientific findings

The Crowdsourced Replication Initiative: Investigating Immigration and Social Policy Preferences. Executive Report.

In an era of mass migration, social scientists, populist parties and social movements raise concerns over the future of immigration-destination societies. What impacts does this have on policy and social solidarity? Comparative cross-national research, relying mostly on secondary data, has findings in different directions. There is a threat of selective model reporting and lack of replicability. The heterogeneity of countries obscures attempts to clearly define data-generating models. P-hacking and HARKing lurk among standard research practices in this area.This project employs crowdsourcing to address these issues. It draws on replication, deliberation, meta-analysis and harnessing the power of many minds at once. The Crowdsourced Replication Initiative carries two main goals, (a) to better investigate the linkage between immigration and social policy preferences across countries, and (b) to develop crowdsourcing as a social science method. The Executive Report provides short reviews of the area of social policy preferences and immigration, and the methods and impetus behind crowdsourcing plus a description of the entire project. Three main areas of findings will appear in three papers, that are registered as PAPs or in process