Estimation of genewise variance arises from two important applications in
microarray data analysis: selecting significantly differentially expressed
genes and validation tests for normalization of microarray data. We approach
the problem by introducing a two-way nonparametric model, which is an extension
of the famous Neyman--Scott model and is applicable beyond microarray data. The
problem itself poses interesting challenges because the number of nuisance
parameters is proportional to the sample size and it is not obvious how the
variance function can be estimated when measurements are correlated. In such a
high-dimensional nonparametric problem, we proposed two novel nonparametric
estimators for genewise variance function and semiparametric estimators for
measurement correlation, via solving a system of nonlinear equations. Their
asymptotic normality is established. The finite sample property is demonstrated
by simulation studies. The estimators also improve the power of the tests for
detecting statistically differentially expressed genes. The methodology is
illustrated by the data from microarray quality control (MAQC) project.Comment: Published in at http://dx.doi.org/10.1214/10-AOS802 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org