A key challenge in modern data analysis is understanding connections between
complex and differing modalities of data. For example, two of the main
approaches to the study of breast cancer are histopathology (analyzing visual
characteristics of tumors) and genetics. While histopathology is the gold
standard for diagnostics and there have been many recent breakthroughs in
genetics, there is little overlap between these two fields. We aim to bridge
this gap by developing methods based on Angle-based Joint and Individual
Variation Explained (AJIVE) to directly explore similarities and differences
between these two modalities. Our approach exploits Convolutional Neural
Networks (CNNs) as a powerful, automatic method for image feature extraction to
address some of the challenges presented by statistical analysis of
histopathology image data. CNNs raise issues of interpretability that we
address by developing novel methods to explore visual modes of variation
captured by statistical algorithms (e.g. PCA or AJIVE) applied to CNN features.
Our results provide many interpretable connections and contrasts between
histopathology and genetics