We describe a method for removing the effect of confounders in order to
reconstruct a latent quantity of interest. The method, referred to as
half-sibling regression, is inspired by recent work in causal inference using
additive noise models. We provide a theoretical justification and illustrate
the potential of the method in a challenging astronomy application.Comment: Extended version of a paper appearing in the Proceedings of the 32nd
International Conference on Machine Learning, Lille, France, 201