Machine Learning is proving invaluable across disciplines. However, its
success is often limited by the quality and quantity of available data, while
its adoption by the level of trust that models afford users. Human vs. machine
performance is commonly compared empirically to decide whether a certain task
should be performed by a computer or an expert. In reality, the optimal
learning strategy may involve combining the complementary strengths of man and
machine. Here we present Expert-Augmented Machine Learning (EAML), an automated
method that guides the extraction of expert knowledge and its integration into
machine-learned models. We use a large dataset of intensive care patient data
to predict mortality and show that we can extract expert knowledge using an
online platform, help reveal hidden confounders, improve generalizability on a
different population and learn using less data. EAML presents a novel framework
for high performance and dependable machine learning in critical applications