Missing data and parameters estimates in multidimensional item response models

Statistical analyses of data based on surveys usually face the problem of missing data. However, some statistical methods require a complete data matrix to be applicable, hence the need to cope with such missingness. Literature on imputation abounds with contributions concerning quantitative responses, but seems to be poor with respect to the handling of categorical data. The present work aims at evaluating the impact of different imputation methods on multidimensional IRT models estimation for dichotomous data

Imputation of missing values in the INFORM Global Risk Index

Although they have been selected on the basis of their reliability, consistency, continuity and completeness, most of indicators used in INFORM Global Risk Index do not have global coverage and neither are issued regularly every year. This results in a significant number of missing values, irregularly distributed among countries, time and indicators. The main motivations for imputing missing values arise from the need to create consistent trends that would otherwise not be possible due to the lack of data in the indicator’s time series, and to increase the reliability of the single compound release. In the presented study we focus on better understanding the patterns and mechanisms of missing values in the INFORM GRI model, and on evaluating their impact on the model’s outputs. The scope is to develop a missing data imputation strategy to be implemented in the INFORM GRI that will strongly depend on the reason why data is missing.JRC.E.1-Disaster Risk Managemen

CleanML: A Study for Evaluating the Impact of Data Cleaning on ML Classification Tasks

Data quality affects machine learning (ML) model performances, and data scientists spend considerable amount of time on data cleaning before model training. However, to date, there does not exist a rigorous study on how exactly cleaning affects ML -- ML community usually focuses on developing ML algorithms that are robust to some particular noise types of certain distributions, while database (DB) community has been mostly studying the problem of data cleaning alone without considering how data is consumed by downstream ML analytics. We propose a CleanML study that systematically investigates the impact of data cleaning on ML classification tasks. The open-source and extensible CleanML study currently includes 14 real-world datasets with real errors, five common error types, seven different ML models, and multiple cleaning algorithms for each error type (including both commonly used algorithms in practice as well as state-of-the-art solutions in academic literature). We control the randomness in ML experiments using statistical hypothesis testing, and we also control false discovery rate in our experiments using the Benjamini-Yekutieli (BY) procedure. We analyze the results in a systematic way to derive many interesting and nontrivial observations. We also put forward multiple research directions for researchers.Comment: published in ICDE 202

Reporting and dealing with missing quality of life data in RCTs : has the picture changed in the last decade?

Peer reviewedPublisher PD

A review of RCTs in four medical journals to assess the use of imputation to overcome missing data in quality of life outcomes

Background: Randomised controlled trials (RCTs) are perceived as the gold-standard method for evaluating healthcare interventions, and increasingly include quality of life (QoL) measures. The observed results are susceptible to bias if a substantial proportion of outcome data are missing. The review aimed to determine whether imputation was used to deal with missing QoL outcomes. Methods: A random selection of 285 RCTs published during 2005/6 in the British Medical Journal, Lancet, New England Journal of Medicine and Journal of American Medical Association were identified. Results: QoL outcomes were reported in 61 (21%) trials. Six (10%) reported having no missing data, 20 (33%) reported ≤ 10% missing, eleven (18%) 11%–20% missing, and eleven (18%) reported >20% missing. Missingness was unclear in 13 (21%). Missing data were imputed in 19 (31%) of the 61 trials. Imputation was part of the primary analysis in 13 trials, but a sensitivity analysis in six. Last value carried forward was used in 12 trials and multiple imputation in two. Following imputation, the most common analysis method was analysis of covariance (10 trials). Conclusion: The majority of studies did not impute missing data and carried out a complete-case analysis. For those studies that did impute missing data, researchers tended to prefer simpler methods of imputation, despite more sophisticated methods being available.The Health Services Research Unit is funded by the Chief Scientist Office of the Scottish Government Health Directorate. Shona Fielding is also currently funded by the Chief Scientist Office on a Research Training Fellowship (CZF/1/31)

Quantifying critical thinking: Development and validation of the Physics Lab Inventory of Critical thinking (PLIC)

Introductory physics lab instruction is undergoing a transformation, with increasing emphasis on developing experimentation and critical thinking skills. These changes present a need for standardized assessment instruments to determine the degree to which students develop these skills through instructional labs. In this article, we present the development and validation of the Physics Lab Inventory of Critical thinking (PLIC). We define critical thinking as the ability to use data and evidence to decide what to trust and what to do. The PLIC is a 10-question, closed-response assessment that probes student critical thinking skills in the context of physics experimentation. Using interviews and data from 5584 students at 29 institutions, we demonstrate, through qualitative and quantitative means, the validity and reliability of the instrument at measuring student critical thinking skills. This establishes a valuable new assessment instrument for instructional labs.Comment: 16 pages, 4 figure

A review of RCTs in four medical journals to assess the use of imputation to overcome missing data in quality of life outcomes

Peer reviewedPublisher PD