1 research outputs found
Instability in clinical risk stratification models using deep learning
While it has been well known in the ML community that deep learning models
suffer from instability, the consequences for healthcare deployments are under
characterised. We study the stability of different model architectures trained
on electronic health records, using a set of outpatient prediction tasks as a
case study. We show that repeated training runs of the same deep learning model
on the same training data can result in significantly different outcomes at a
patient level even though global performance metrics remain stable. We propose
two stability metrics for measuring the effect of randomness of model training,
as well as mitigation strategies for improving model stability.Comment: Accepted for publication in Machine Learning for Health (ML4H) 202