A large body of empirical research in accounting investigates the causes and consequences of accruals quality, reaching numerous influential conclusions. Yet little work has been done to systematically evaluate the validity of the underlying measures of accruals quality. We evaluate these measures using three criteria: (i) Is the measure unaffected by the underlying economic determinants of accruals? (ii) Does the measure consistently reflect errors in accruals? and (iii) Does the measure facilitate tests with sufficient power to detect plausible variation in accrual errors? Using a combination of theoretical modelling and numerical simulations, we show that all measures fail at least one of these criteria. Our evaluation provides new interpretations of existing research and guides the choice of measures and the interpretation of results in future research