1 research outputs found
An Empirical Framework for Domain Generalization in Clinical Settings
Clinical machine learning models experience significantly degraded
performance in datasets not seen during training, e.g., new hospitals or
populations. Recent developments in domain generalization offer a promising
solution to this problem by creating models that learn invariances across
environments. In this work, we benchmark the performance of eight domain
generalization methods on multi-site clinical time series and medical imaging
data. We introduce a framework to induce synthetic but realistic domain shifts
and sampling bias to stress-test these methods over existing non-healthcare
benchmarks. We find that current domain generalization methods do not achieve
significant gains in out-of-distribution performance over empirical risk
minimization on real-world medical imaging data, in line with prior work on
general imaging datasets. However, a subset of realistic induced-shift
scenarios in clinical time series data exhibit limited performance gains. We
characterize these scenarios in detail, and recommend best practices for domain
generalization in the clinical setting.Comment: Accepted at ACM CHIL 202