In defense of Pratt's variable importance axioms: A response to Gromping

In a recent paper Gromping provided a wide-ranging review of metrics for assessing variable importance in regression analysis. There are, however, several flaws in Gromping's criticism of the well-known metric attributed to Pratt. Among the metrics she reviewed, Pratt's metric stands out because it is the only one that provides both a theoretically based definition of variable importance, and a simple method of estimation and inference. Our response is an effort to re-evaluate this unique metric. We give a simplified and abbreviated account of Pratt's original derivation, based on which we address the flaws in Gromping's presentation. We also discuss heuristic interpretations of Pratt's metric, and suggest a new approach for selecting one from among the many available metrics for assessing importance. This approach is intended to supplement that suggested by Gromping. Accordingly, the goal of this response is to help practitioners better understand and choose between available metrics for assessing variable importance. This article is categorized under: Statistical and Graphical Methods of Data Analysis > Multivariate Analysis Statistical Learning and Exploratory Methods of the Data Sciences > Exploratory Data Analysis

Embedding IRT in structural equation models: A comparison with regression based on IRT scores

This article reviews the problems associated with using item response theory (IRT)-based latent variable scores for analytical modeling, discusses the connection between IRT and structural equation modeling (SEM)-based latent regression modeling for discrete data, and compares regression parameter estimates obtained using predicted IRT scores and standardized number-right scores in Ordinary Least Squares (OLS) regression with regression estimates obtained using the combined IRT-SEM approach. The Monte Carlo results show the expected a posteriori (EA approach is insensitive to sample size as expected but leads to appreciable attenuation in regression parameter estimates. Standardized number-right estimates and EAP regression estimates were found to be highly comparable. On the other hand, the IRT-SEM method produced smaller finite sample bias, and as expected, generated consistent regression estimates for suitably large sample sizes. Copyrigh

An Anthropologist Among the Psychometricians: Assessment Events, Ethnography, and Differential Item Functioning in the Mongolian Gobi

This article explores the potential for ethnographic observations to inform the analysis of test item performance. In 2010, a standardized, large-scale adult literacy assessment took place in Mongolia as part of the United Nations Educational, Scientific and Cultural Organization Literacy Assessment and Monitoring Programme (LAMP). In a novel form of interdisciplinary collaboration an ethnographer worked closely with psychometric researchers to investigate the sources and explanations of differential item functioning and test item performance. The research involved detailed ethnographic observations of literacy assessment events. The results illustrate the potential for ethnography to provide insights into testing situations, group and test item performance, and to combine with psychometric analysis in cross-cultural assessment

On Johnson's (2000) Relative Weights Method for Assessing Variable Importance: A Reanalysis

This article provides a reanalysis of J. W. Johnson's (2000) "relative weights" method for assessing variable importance in multiple regression. The primary conclusion of the reanalysis is that the derivation of the method is theoretically flawed and has no more validity than the discredited method of Green, Carroll, and DeSarbo (1978) on which it is based. By means of 2 examples, supplemented by other results from the literature, it is also shown that the method can result in materially distorted inferences when it is compared with another widely used importance metric, namely, general dominance (Azen & Budescu, 2003; Budescu, 1993). Our primary recommendation is that J. W. Johnson's (2000) relative weights method should no longer be used as a variable importance metric for multiple linear regression. In the final section of the article, 2 additional recommendations are made based on our analysis, examples, and discussion

A retrospective case-control analysis of 2002 running injuries

Etude épidémiologique sur les lésions des coureurs. Typologie, facteurs de risque identifiés à partir d'un modèle incluant les caractéristiques anthropométriques, les informations sur l'entraînement des sujets et certaines variables biomécaniques liées aux blessures en course à pied