This EM review article focuses on parameter expansion, a simple technique
introduced in the PX-EM algorithm to make EM converge faster while maintaining
its simplicity and stability. The primary objective concerns the connection
between parameter expansion and efficient inference. It reviews the statistical
interpretation of the PX-EM algorithm, in terms of efficient inference via bias
reduction, and further unfolds the PX-EM mystery by looking at PX-EM from
different perspectives. In addition, it briefly discusses potential
applications of parameter expansion to statistical inference and the broader
impact of statistical thinking on understanding and developing other iterative
optimization algorithms.Comment: Published in at http://dx.doi.org/10.1214/10-STS348 the Statistical
Science (http://www.imstat.org/sts/) by the Institute of Mathematical
Statistics (http://www.imstat.org