We provide a rigorous definition of the visual cause of a behavior that is
broadly applicable to the visually driven behavior in humans, animals, neurons,
robots and other perceiving systems. Our framework generalizes standard
accounts of causal learning to settings in which the causal variables need to
be constructed from micro-variables. We prove the Causal Coarsening Theorem,
which allows us to gain causal knowledge from observational data with minimal
experimental effort. The theorem provides a connection to standard inference
techniques in machine learning that identify features of an image that
correlate with, but may not cause, the target behavior. Finally, we propose an
active learning scheme to learn a manipulator function that performs optimal
manipulations on the image to automatically identify the visual cause of a
target behavior. We illustrate our inference and learning algorithms in
experiments based on both synthetic and real data.Comment: Accepted at UAI 201