What Affects the Quality of Score Transformations? Potential Issues in True-Score Equating Using the Partial Credit Model

This simulation study investigated to what extent departures from construct similarity as well as differences in the difficulty and targeting of scales impact the score transformation when scales are equated by means of concurrent calibration using the partial credit model with a common person design. Practical implications of the simulation results are discussed with a focus on scale equating in health-related research settings. The study simulated data for two scales, varying the number of items and the sample sizes. The factor correlation between scales was used to operationalize construct similarity. Targeting of the scales was operationalized through increasing departure from equal difficulty and by varying the dispersion of the item and person parameters in each scale. The results show that low similarity between scales goes along with lower transformation precision. In cases with equal levels of similarity, precision improves in settings where the range of the item parameters is encompassing the person parameters range. With decreasing similarity, score transformation precision benefits more from good targeting. Difficulty shifts up to two logits somewhat increased the estimation bias but without affecting the transformation precision. The observed robustness against difficulty shifts supports the advantage of applying a true-score equating methods over identity equating, which was used as a naive baseline method for comparison. Finally, larger sample size did not improve the transformation precision in this study, longer scales improved only marginally the quality of the equating. The insights from the simulation study are used in a real-data example

All metrics are equal, but some metrics are more equal than others: A systematic search and review on the use of the term ‘metric’

Objective: To examine the use of the term 'metric' in health and social sciences' literature, focusing on the interval scale implication of the term in Modern Test Theory (MTT). Materials and methods: A systematic search and review on MTT studies including 'metric' or 'interval scale' was performed in the health and social sciences literature. The search was restricted to 2001-2005 and 2011-2015. A Text Mining algorithm was employed to operationalize the eligibility criteria and to explore the uses of 'metric'. The paradigm of each included article (Rasch Measurement Theory (RMT), Item Response Theory (IRT) or both), as well as its type (Theoretical, Methodological, Teaching, Application, Miscellaneous) were determined. An inductive thematic analysis on the first three types was performed. Results: 70.6% of the 1337 included articles were allocated to RMT, and 68.4% were application papers. Among the number of uses of 'metric', it was predominantly a synonym of 'scale'; as adjective, it referred to measurement or quantification. Three incompatible themes 'only RMT/all MTT/no MTT models can provide interval measures' were identified, but 'interval scale' was considerably more mentioned in RMT than in IRT. Conclusion: 'Metric' is used in many different ways, and there is no consensus on which MTT metric has interval scale properties. Nevertheless, when using the term 'metric', the authors should specify the level of the metric being used (ordinal, ordered, interval, ratio), and justify why according to them the metric is at that level

Creating a common metric based on existing activities of daily living tools to enable standardized reporting of functioning outcomes achieved during rehabilitation

Objective: Many different assessment tools are used to assess functioning in rehabilitation; this limits the comparability and aggregation of respective data. The aim of this study was to outline the development of an International Classification of Functioning, Disability and Health (ICF)-based interval-scaled common metric for 2 assessment tools assessing activities of daily living: the Functional Independence Measure (FIMTM) and the Extended Barthel Index (EBI), used in Swiss national rehabilitation quality reports. Methods: The conceptual equivalence of the 2 tools was assessed through their linking to the ICF. The Rasch measurement model was then applied to create a common metric including FIMTM and EBI. Subjects: Secondary analysis of a sample of 265 neurological patients from 5 Swiss clinics. Results: ICF linking found conceptual coherency of the tools. An interval-scaled common metric, including FIMTM and EBI, could be established, given fit to the Rasch model in the related analyses. Conclusion: The ICF-based and interval-scaled common metric enables comparison of patients� and clinics� functioning outcomes when different activities of daily living tools are used. The common metric can be included in a Standardized Assessment and Reporting System for functioning information in order to enable data aggregation and comparability

Change in environmental barriers experienced over a 5-year period by people living with spinal cord injury in Switzerland:a prospective cohort study

STUDY DESIGN: Cohort study with two measurement occasions. OBJECTIVES: To investigate change in environmental barriers experienced by people living with spinal cord injury (SCI) over a 5-year period. SETTING: Community, Switzerland. METHODS: Data were from the Swiss spinal cord injury (SwiSCI) survey. Main outcome measure was the Nottwil Environmental Factors Inventory-Short Form. Random-effects Poisson regression featuring between-within estimation was used to examine predictors of the number of environmental barriers and of its change over time. RESULTS: One thousand five hundred and forty-nine persons participated in Survey 2012 and 1530 participated in Survey 2017; 761 participated in both surveys. In both surveys most participants reported at least three barriers. Leading issues were unfavorable climate, inaccessibility of buildings and public spaces, and lack of or insufficiently adapted means of transportation. Reporting of barriers related to climate, finances, and state services declined over time. Between subjects, having more health problems, lesser physical independence, poorer mental health, and a lower household income were related to a higher number of barriers experienced. Within subjects, improvements in income, physical independence, and mental health over time were related to a reduction in barriers. CONCLUSIONS: Inaccessibility of buildings and places and problems with transportation remained major barriers over a 5-year period and should be priorities of Swiss disability policy. People with reduced mental and physical health, and those with lower income are vulnerable groups deserving specific attention. Policies targeting income and life-long rehabilitation targeting health promotion and maintenance may be suitable means to reduce the experience of environmental barriers

Demographic and environmental factors associated with disability in India, Laos, and Tajikistan: a population-based cross-sectional study

Background The number of people experiencing functional limitations due to health conditions (capacity) is expected to increase in low and middle-income countries as populations age and rates of non-communicable disease rise. This trend could raise the prevalence and levels of disability worldwide. Understanding the demographic and environmental factors associated with disability can inform the design of policy interventions to make societies more accessible and inclusive for all. Methods Approximately 2,500–3,000 participants in each of India, Laos, and Tajikistan responded to the Gallup World Poll and the World Health Organization’s Brief Model Disability Survey through face-to-face interviews. For each country, random forest regression was performed to explore the associations of demographic and environmental factors with disability while controlling for capacity. Using the variable importance measures generated by the random forest models, linear regression models were built in a stepwise manner for each country to predict disability level based on these contextual factors. Results Capacity was strongly associated with disability in all three countries. Most of the variance in disability was explained by minimally adjusted linear models that included only capacity, sex, and age. Inclusion of additional demographic factors and environmental factors explained slightly more of the variance in disability score. Across all three countries, the level of basic infrastructure, public services, and financial stability were moderately associated with disability. Age, sex, employment status, the use of assistive technologies, and other factors had associations with disability that were highly variable across countries. Conclusion While capacity was the main determinant of disability, individual demographic and environmental factors were associated with disability in a country-specific manner while controlling for the effects of capacity

The extended Barthel Index (EBI) can be reported as a unidimensional interval-scaled metric: a psychometric study

Background: The Extended Barthel Index (EBI), consisting of the original Barthel Index plus 6 cognitive items, provides a tool to monitor patients' outcomes in rehabilitation. Whether the EBI provides a unidimensional metric, thus can be reported as a valid sum-score, remains to be examined. Objective: To examine whether the EBI can be reported as unidimensional interval-scaled metric for neurological and musculoskeletal rehabilitation. Methods: Rasch analysis of a calibration sample of 800 cases from neurological or musculoskeletal rehabilitation in 2016 in Switzerland. Results: In the baseline analysis no fit to the Rasch Model was achieved. When accommodating local dependencies with a testlet approach satisfactory fit to the Rasch Model was achieved, and an interval scale transformation table was created. Conclusion: The results support the reporting of adapted EBI total scores for both rehabilitation groups by applying the interval scaled transformation table presented in this study.+ ID der Publikation: unilu_51329 + Sprache: Englisch + Letzte Aktualisierung: 2020-12-17 14:42:28 Abstract auf Englisch und Deutsc

WHO Functioning and Disability Disaggregation (FDD11) tool: a reliable approach for disaggregating data by disability

Background There is a global scarcity of good quality disability data, which has contributed to a lack of political will to address the challenges that persons with disabilities face. The current paper proposes a way forward to overcome this gap by demonstrating the psychometric properties of the World Health Organization Functioning and Disability Disaggregation Tool (FDD11) - a brief disability disaggregation instrument that countries can use. Results The study demonstrated that FDD11 is a valid and reliable tool. Unidimensionality of the scale produced by each calibration was supported by the factor analysis performed. The analysis indicated good fit of the items, and targeting of the items was deemed to be sufficient. The person separation index was 0.82, indicating good reliability of the final scale. Conclusion FDD11 provides a good opportunity to researchers and governments to capture good quality disability data and to disaggregate existing data by disability. The tool can facilitate low- and middle-income countries in their efforts to develop evidenced-based policies to address any barriers faced by persons with disabilities, to monitor the implementation of the Convention on the Rights of Persons with Disabilities and the Sustainable Development Goals, and to take stock of the challenges that still remain

Measuring functioning and disability using household surveys: metric properties of the brief version of the WHO and World Bank model disability survey

Background The Model Disability Survey (MDS) is the current standard recommended by WHO to collect functioning and disability data. Answering calls from countries requesting a version to be implemented as a module that could be integrated into existing surveys and be used for monitoring disability trends and for data disaggregation, WHO developed the brief MDS. The objectives of this paper are to evaluate the metric properties of the disability metrics generated with the Brief MDS and the precision of the Brief MDS in comparison with the full MDS. Results The partial credit model, a unidimensional model for polytomous data from the Rasch family, was applied to evaluate psychometric properties using data from national MDS implementations in Chile (N = 12,265) and in Sri Lanka (N = 3000). The Brief MDS generates valid metrics for measuring disability, from the perspectives of capacity and performance, thereby achieving good levels of measurement precision in comparison with its full counterpart. Conclusion Given the scarcity of valid functioning and disability modules for household surveys, the Brief MDS represents a milestone in disability measurement. The Brief MDS is currently used by countries to monitor disability trends over time, which is especially important to evaluate the impact of health policies and public health interventions, to disaggregate indicators of the Sustainable Development Goals, and to monitor the implementation of the UN Convention on the Rights of Persons with Disabilities (CRPD)

Rasch Analysis of the International Quality of Life Basic Data Set Version 2.0

Objective: To examine the internal construct validity of the International Spinal Cord Injury Quality of Life Basic Data Set Version 2.0 (QoL-BDS V2.0) and compare this with the internal construct validity of the original version of the QoL-BDS. Design: International cross-sectional psychometric study. Setting: Spinal rehabilitation units, clinics, and community. Participants: The study involved 5 sites and 4 countries, 2 of whose primary language is not English. Each site included a consecutive sample of inpatients with spinal cord injury or disease (SCI/D) and a convenience sample of individuals with SCI/D living in the community (N=565). Main outcome measures: The QoL-BDS V2.0 consists of the 3 original items on satisfaction with life as a whole, physical health, psychological health of the QoL-BDS, and an additional item on satisfaction with social life. All 4 items are answered on a 0-10 numeric rating scale. Rasch analysis was performed on versions 1.0 and 2.0 of the QoL-BDS to examine the ordering of the items' response options, item scaling, reliability, item fit, local item independence, differential item functioning, and unidimensionality. Results: The sample included 565 participants with 57% outpatients and 43% inpatients. Mean age was 51.4 years; 71% were male; 65% had a traumatic injury, 40% had tetraplegia, and 67% were wheelchair users. Item thresholds were collapsed for ordering, and subsequent analyses showed good internal construct validity for the QoL-BDS V2.0 with a person separation reliability of 0.76 and Cronbach α of 0.81. Infit and outfit statistics ranged 0.62-0.91. No local dependencies and multidimensionality were found. Differential item functioning was observed only for country and inpatients vs outpatients but not for other participants' characteristics. Differences in internal construct validity between the 3-item and 4-item versions were minimal. Conclusions: The results of this Rasch analysis support the internal construct validity of the QoL-BDS V2.0

Validation of PARADISE 24 and Development of PARADISE-EDEN 36 in Patients with Dementia

Dementia was one of the conditions focused on in an EU (European Union) project called “PARADISE” (Psychosocial fActors Relevant to brAin DISorders in Europe) that later produced a measure called PARADISE 24, developed within the biopsychosocial model proposed in the International Classification of Functioning Disability and Health (ICF). The aims of this study are to validate PARADISE 24 on a wider sample of patients with mild to moderate dementia to expand PARADISE 24 by defining a more specific scale for dementia, by adding 18 questions specifically selected for dementia, which eventually should be reduced to 12. We enrolled 123 persons with dementia, recruited between July 2017 and July 2019 in home care and long-term care facilities, in Italy, and 80 participants were recruited in Warsaw between January and July 2012 as part of a previous cross-sectional study. The interviews with the patient and/or family were conducted by health professionals alone or as a team by using the Paradise data collection protocol. The psychometric analysis with the Rasch analysis has shown that PARADISE 24 and the selection of 18 additional condition-specific items can be expected to have good measurement properties to assess the functional state in persons with dementia