Reliability in the Rasch model

summary:This paper deals with the reliability of composite measurement consisting of true-false items obeying the Rasch model. A definition of reliability in the Rasch model is proposed and the connection to the classical definition of reliability is shown. As a modification of the classical estimator Cronbach's alpha, a new estimator logistic alpha is proposed. Finally, the properties of the new estimator are studied via simulations in the Rasch model

Rasch model item response theory (IRT) to analyze the quality of mathematics final semester exam test on system of linear equations in two variables (SLETV)

A high-quality test has a balanced level of difficulty and can be completed by the respondent with their level of abilities. This study analyzed the test instrument used to measure students' mathematics abilities in the semester final exam on System of Linear Equations in Two-Variables. The purposive sampling technique was applied to select the respondent students (N=195). The test items were twenty multiple-choice questions. The researchers performed the data analysis using Rasch model Item Response Theory (IRT) approach with the QUEST program. The analysis revealed that the twenty items’ validity matched the Rasch model with a range of INFIT MNSQ values between 0.89 – 1.17. Items on the final semester exam can be used based on the estimated OUTFIT t-value less than equal to 2.00. The OUTFIT t analysis obtained nineteen qualified items and one unqualified item.

Application of the Rasch Model to Measure Five Dimensions of Wellness in Community-Dwelling Older Adults

Background and Purpose: Nurse researchers and practicing nurses need reliable and valid instruments to measure key clinical concepts. The purpose of this research was to develop an innovative method to measure dimensions of wellness among older adults. Method: A sample of 5,604 community-dwelling older adults was drawn from members of the COLLAGE consortium. The Wellness Assessment Tool (WEL) of the COLLAGE assessment system provided the data used to create the scores. Application of the Rasch analysis and Masters\u27 partial credit method resulted in logit values for each item within the five dimensions of wellness as well as logit values for each person in the sample. Results: The items fit the Rasch model, and the composite scores for each dimension demonstrated high reliability (1.00). The person reliability was low: social (.19), intellectual (.33), physical (.29), emotional (.20), and spiritual (.29). The small number of items within each dimension and the homogenous sample appear to have contributed to this low reliability. Conclusion: Ongoing research using multidimensional tools to measure dimensions of wellness among older adults is needed to advance wellness science and wellness promotion in nursing practice

The relationship between students’ mathematics attitude and their mathematical thinking

This research aims to describe students’ mathematical thinking based on their attitude towards mathematics. In addition, this study also aims to investigate the relationship between students’ mathematics attitude and their mathematical thinking and to know how much influence mathematics attitude among the students with their mathematical thinking. There are two indicators of students’ mathematics attitude which were used in this study, namely Affective Engagement which means emotions or thoughts toward mathematics and Cognitive Engagement which defines as the judgements and beliefs about mathematics. This study employed a descriptive research design and 75 Form Four students from a public school in Segamat, Johor, Malaysia have been chosen as the sample of this study. A Mathematical Thinking Test (MTT) and Mathematics Attitude Questionnaire were used as the instruments in this study to collect the data. Based on the finding of this research, generally, there is a positive correlation between students’ mathematics attitude and mathematical thinking skill. However, according to the Pearson Correlation test results, students’ attitude towards mathematics has little influence on their mathematical thinking ability

Blue-Collar Workers Entrepreneurial Intentions and The Extended Theory of Reasoned Action: Incorporating SEM and Person-Item Map Analysis

Introduction/Main Objectives: Blue-collar workers face major threats related to the development and application of Industry 4.0. Unfortunately, research on how they deal with this situation is scarce in the behavioral science literature. In this study, we attempt to fill this gap by emphasizing a methodological aspect of combining structural equation modeling (SEM) and person-item map analysis to the extended model of the theory of reasoned action. Novelty and Methods: We offer the notion of combining SEM and Rasch model analysis to explain the extended of theory of reasoned action. The respondents were blue-collar workers from Indonesia who have not yet started a business. Finding/Results: In line with our goal of applying intersubjective certification to the extended theory of reasoned action (TRA) model in the context of blue-collar workers, our results suggest that religiosity affects entrepreneurial intentions, both directly and indirectly, through attitude. The subjective norms have also been shown to influence the intention of blue-collar workers to become entrepreneurs. The extended TRA model has been proven empirically to have good predictive power, with a total effect of 83%. Conclusion: Regarding the sample issue, the person-item map is excellent for explaining our SEM-based findings. The idea of combining the Rasch model property, which is a persons-items map, requires more empirical support to promote its ability to illuminate SEM-based research explanations

Assessment of posttraumatic stress disorder among women after childbirth using the City Birth Trauma Scale in Spain

OBJECTIVE: Postpartum posttraumatic stress disorder (PP-PTSD) affects 3.1-6.3% of women after childbirth. The City Birth Trauma Scale (City-BiTS) is a questionnaire designed to evaluate and diagnose this disorder, according to the Diagnostic and Statistical Manual of Mental Disorders (5th ed.) criteria, including the following groups of symptoms characteristic of posttraumatic stress: reexperiencing, avoidance, negative cognitions and mood, and hyperarousal. The aim of the present study was to evaluate the psychometric properties of the Spanish-language version of this questionnaire (City-BiTS-S), based on a community sample of Spanish women. METHOD: A total of 207 mothers, recruited at three health centers in southern Spain, completed the City-BiTS-S questionnaire and provided sociodemographic and obstetric data. RESULTS: Exploratory factor analysis of the data replicated the two-factor structure reported in previous studies that explained 47.9% of the variance: Factor 1 of general symptoms and Factor 2 of birth-related symptoms. Both City-BiTS-S (Cronbach's alpha = .90) and the two factors (Cronbach's alpha for Factor 1 = 0.89; Cronbach's alpha for Factor 2 = 0.82) presented high internal consistency. Rasch analysis confirmed the unidimensionality of the two factors as valid subscales of the PP-PTSD. Results suggested reducing response options, reviewing Item 8, and rewording Item 3 in the Spanish version. CONCLUSIONS: The City-BiTS-S presents appropriate psychometric properties to measure symptoms of PP-PTSD. Nevertheless, further research is recommended to confirm its validity in a clinical population and in different medical approaches to the birth process

Validation of The Indonesian Version of The Partner’s Emotional Dependency Scale (SED): a Rasch Model Approach

This research aimed to verify the Indonesian version of the Partner’s Emotional Dependency Scale (SED), originally developed by Camarillo et al. (2020) in Spanish. The SED is a unidimensional scale with 22 items designed to measure emotional dependence on a partner. This is the first time the instrument has been adapted, particularly for use in Indonesia. The validation process employed the Rasch model to ensure the scale’s reliability and accuracy in the Indonesian context. The study involved 229 participants (F = 185, M = 44), aged between 18 and 35 (MAge = 25.5, SD = 4.23). The results confirmed the unidimensionality of the SED scale, with 2 – 3 items contributing to the second dimension. The 5-point Likert scale functioned effectively, enabling respondents to distinguish between different response options. The person-item interaction reliability was found to be .85, and the item reliability was .98, indicating excellent reliability of the SED scale items. All items were evenly distributed across the scale from high to low difficulty. However, items number 1, 3, and 7 (SED 1, SED 3, SED 7) were identified as less suitable for measuring emotional dependence and respondents found it difficult to answer these specific questions. Furthermore, three items (SED 5, SED 6, and SED 19) exhibited Differential Item Functioning (DIF) due to gender bias. The study concludes that, with some revisions and improving the translation, the Indonesian version of the SED scale could be a valuable tool for measuring the level of emotional dependence on a partner in the Indonesian populatio

Development of Critical Thinking Ability Test using the Rasch Model on Substance Pressure Materials

This study aims to produce an instrument to measure students' critical thinking skills on pressure equipment in class VIII SMP. The type of research used is Research and Development (R&D). The type of research used is research and development (R&D). The test instrument developed was in the form of multiple choice based on eight development steps, namely potential and problems, information gathering, product design, product validation, product revision, product testing, product revision, and final product. Validation of the instrument was developed by 3 validators, namely 2 lecturers of Physics Education at FKIP Tanjungpura University and 1 science teacher at the school. The validation given to the validator is in the form of a material validation questionnaire and question validation. The results of material expert validation data obtained 84.72%, validation obtained from content validation with an Aiken index of 0.78, included in the high category and suitable for use. The results of the data analysis of teacher and student response questionnaires obtained 92.60% and 86.26% categorized as very feasible. The results of construct validity using the Rasch model with the application of Winstep 3.73 obtained result 3, which was declared inappropriate because it did not meet the MNSQ, ZSTD, and Pt Measure Corr criteria. The reliability of the Cronbach alpha test value obtained a value of 0.73 with a fairly high category. Regarding the level of difficulty of the item, 4 question items must be discarded because they are included in the very easy and very easy categories. The overall discriminatory power is in the good category with a discriminatory index of 0.40 DP < 70. The results of the study show that out of 30 things, 23 things are appropriate to be used to measure students' critical thinking skills with good-quality items

SCORE EQUATING BETWEEN AEPS-2 AND AEPS-3 FOR 0-3 YEAR OLDS

Over the past two decades, the emphasis on educational equity in early childhood education (ECE) and early childhood special education (ECSE) has highlighted the importance of assessment through policies and regulations. Ensuring accurate assessment scores is a fundamental aspect of this trend. The release of the Assessment, Evaluation, and Programming System for Infants and Children, Third Edition (AEPS-3) in December 2021 led to a shift from the Second Edition (AEPS-2) in child development scoring. In order to harmonize the previous and updated assessment versions for children aged 0-3 across six developmental domains, a common item non-equivalent design, featuring fixed parameter calibration equating (known as \u27anchoring\u27), is utilized within the Rasch framework. A total of 18,411 cases from the AEPS-2 Test Level I and 317 cases from the AEPS-3 Test were utilized to assess scale quality. The psychometric properties of both assessment versions were evaluated using the rating scale Rasch model, revealing a good model-data fit. Two sets of anchor items, selected based on either identical or functional matching methods, were determined using the cosine similarity coefficient and subsequently validated through expert content analysis. These anchor item sets demonstrated acceptable quality. The research then examined the impact of different anchor sets on person parameter estimation during the anchoring process. Ultimately, the study produced person measure and observed score conversion tables between AEPS-2 and AEPS-3. The resulting conversion tables provide valuable insights into the relationship between the old and updated assessment versions. These findings contribute to equating methodology, ECE/ECSE, and education policy. As an early implementation of functional matching anchoring equating in the ECSE field, this study provides a practical model for score equating transformation that can be applied across both early childhood education and special education sectors. In the early childhood education area, it supports the ongoing refinement of assessment tools in early childhood education, helping practitioners make more informed decisions about child development. By leveraging the psychometric model, the research contributes to improving the quality of assessment tools for early childhood education practitioners, leading to better outcomes for children in these critical developmental stages. Another important contribution of this study is that it reflects the assessment requirements in special education and connects education policy with research goals. This ensures that assessments remain consistent, fair, and accurate, enabling educators and specialists to effectively track and support children\u27s development over time, ultimately improving educational equity

An argument-based validation study of the English Placement Test (EPT) – Focusing on the inferences of extrapolation and ramification

English placement tests have been widely used in higher education as post-admission assessment instruments to measure admitted English as a second language (ESL) students’ English proficiency or readiness in academic English, usually upon their arrival at universities in English-speaking countries. Unlike commercial standardized English proficiency tests, many English placement tests are locally developed with comparatively limited resources and are relatively under-investigated in the field of language testing. Even less attention has been directed to the score interpretation and the impact of placement decisions on ESL students’ English learning and academic achievement. Undoubtedly, this scarcity of research on English placement tests is inappropriate in view of their status as one of the most frequently used language testing instruments, which may exert immediate and strong impact on ESL students’ learning in general. By employing a mixed-methods approach, this dissertation project investigates the validity of test score interpretation and use of the English Placement Test (EPT) used at Iowa State University (ISU) under an argument-based validity framework. More specifically, this study started with an interpretation and use argument for the EPT, which states the score meaning and intended impact of the EPT explicitly, and focused on the last two inferences in the interpretation and use argument, namely extrapolation and ramification. The extrapolation inference links expected scores of the EPT (scores that exhibit adequate test reliability) to target scores or actual performance in the target domain. In this study, the extrapolation inference requires investigation of the relationship between ESL students’ English placement test performance and two external criteria of English performance, including the TOEFL iBT and a self-assessment. The ramification inference links the use of the EPT results to its actual impact and in this study the ramification inference requires investigation of the impact of the placement decisions in a specific educational context. For the extrapolation inference, quantitative data such as test performance data on the EPT, the TOEFL iBT, and the self-assessment were collected and analyzed using multitrait-multimethod (MTMM) analysis techniques. The findings indicated that the EPT was found to have moderate relationships with the TOEFL iBT and weak to moderate relationships with the self-assessment. The EPT showed some of the expected convergent correlations as well as discriminant correlations based on the MTMM correlation coefficient matrix as well as the factor loading parameters in a correlated trait-correlated uniqueness (CTCU) model. For the ramification inference, three types of analyses were conducted to seek support with regard to 1) test stakeholders’ perceptions of the EPT placement decisions, 2) the impact of the EPT placement on ESL students’ English learning, and 3) the relationship between ESL students’ EPT performance and their first-semester academic achievement. The interviews with test stakeholders were coded and analyzed to identify statements indicating their perceptions of the impact of the placement decisions. The qualitative findings are also utilized to help interpret the quantitative findings. Multiple paired-samples t-tests are used to investigate ESL students’ progress in the ESL courses that they were placed into. In addition, a structural equation modeling (SEM) approach was used to model the relationship among students’ performance on the EPT, ESL courses, and their first-semester GPA, mediated by individual difference constructs, such as learning motivation, academic self-efficacy, and self-regulated learning strategies. The qualitative analyses of the interviews with four groups of test stakeholders show that the interviewed ESL students in general experienced initial frustration regarding the placement decisions, in retrospect, they understood why they were placed into ESL courses and appreciated the benefits of taking the required courses, especially ESL writing courses. The ESL course instructors were satisfied with the placement accuracy, even though occasionally they identified a few cases of potentially misplaced students in the ESL courses. The interviewed undergraduate academic advisors showed positive perceptions of the EPT and the placement decisions. They also reported observing that the majority of the ESL advisees were receptive to the EPT placement decisions. The analyses of ESL course performance data collected at the beginning and the end of the course indicate that ESL students in Engl99L, an ESL listening course focusing on listening strategies, made statistically significant progress in terms of score gain on the same listening test administered at two time points. However, only nine out of 38 ESL students made satisfactory progress with reference to the course standard. Students in Engl101B (a lower-level ESL academic English writing course) and Engl101C (a higher-level ESL academic English writing course) did not show much progress in terms of lexical complexity, syntactic complexity, and grammatical accuracy. However, the Engl101C students on average wrote longer essays at the end of the course. Based on the ratings of the essays written in the final exams using the EPT scoring rubric, 14 out 18 Engl101B students (77.8%) and eight out of 16 Engl101C students (50%) showed satisfactory progress in these classes and were deemed ready for the next level of English study. The SEM analysis results indicate that ESL students’ EPT performance had significant and direct impact on their academic achievement. What’s more, students’ EPT performance predicted their academic self-efficacy and affected extrinsic goal orientation. However, these motivational factors did not have direct impact on academic achievement. The findings in this study contribute to building the validity argument for the EPT with two of the assumptions underlying the warrant for the extrapolation inference and ramification inference found supported and the other three partially supported. This findings in this study contributed to a better understanding of the score interpretation and use of the EPT at Iowa State University through constructing a validity argument. These findings shed light on the future development of the EPT and other similar English placement tests. The findings in this study as well as the research methodology can be informative for other institutions where English placement tests are used