Testing and correcting sample selection in academic achievement comparisons

Abstract

Country comparisons using standardized test scores may in some cases be misleading unless we make sure that the potential sample selection bias created by drop-outs and non-enrollment patterns does not alter the analysis. In this paper, I propose an answer to this issue which consists in comparing the counterfactual distribution of achievement (I mean the distribution of achievement if there was hypothetically no selection) and the observed distribution of achievements. If the difference is statistically significant, international comparison measures like means, quantiles, and inequality measures have to be computed using that counterfactual distribution. I identify the quantiles of that latent distribution by readjusting the percentile levels of the observed quantile function of achievement. Because the data on test scores is by nature truncated, I have to rely on auxiliary data to borrow identification power. I finally applied my method to 6 sub-Saharan countries using 6th-grade test scores