The mean and standard deviation of data, some of which are below the detection limit: An introduction to maximum likelihood estimation

In this article, the principle of maximum likelihood estimation (MLE) is introduced. It is illustrated by an application of the maximum likelihood method for the estimation of the mean and standard deviation of a single censored data set. This is a data set for which some data are only known to be below a lower limit (left-censored) or above an upper limit. An example of the determination of an impurity in a raw material, where the measurements are carried out around the detection limit and some fall below it, is given to illustrate the application of MLE to a single censored data set

Comparison of methods for the estimation of statistical parameters of censored data

Approaches based on the maximum likelihood (ML) method and on the order statistics are described and evaluated for the estimation of the mean and standard deviation of a normal population from a left-singly censored sample, i.e. a sample for which some measurement results fall below the reporting limit of the analytical method. The performance of the methods is evaluated by means of data simulations. The sample size considered is small to moderate: N=6–18. Simulation data show that the ML method performs better than the method based on order statistics, especially in difficult situations, e.g. large expected censored proportion hex (hex≥50%) and for small sample size (N=6). The reliability of the estimates depends on the censored proportion. The larger the censored proportion, the poorer the quality of the estimates. When the expected censored proportion does not exceed 50%, i.e. when the true mean μ of the measurement results is above the reporting limit, the performance of the ML method in the estimation of the mean of a censored sample is very acceptable, i.e. it is comparable to that using classical moment calculation on a complete (non-censored) sample. When the expected censored proportion is very high (e.g. 83%) the estimates are, as expected, largely biased. The performance of the ML method in the estimation of the standard deviation of censored data is not as good as in the estimation of the mean. A formula is given for the approximate sample size required to have a specified confidence level that a ML estimated mean for the censored sample will not differ from the true mean by a certain magnitude