Development of a Novel Ulcerative Colitis Endoscopic Mayo Score Prediction Model Using Machine Learning

Background and Aims: Endoscopic assessment is a co-primary end point in inflammatory bowel disease registration trials, yet it is subject to inter- and intraobserver variability. We present an original machine learning approach to Endoscopic Mayo Score (eMS) prediction in ulcerative colitis and report the model’s performance in differentiating key levels of endoscopic disease activity on full-length procedure videos. Methods: Seven hundred ninety-three full-length videos with centrally-read eMS were obtained from 249 patients with ulcerative colitis, who participated in a phase II trial evaluating mirikizumab (NCT02589665). A video annotation approach was established to extract mucosal features and associated eMS classification labels from each video to be used as inputs for model training. The primary objective of the model was a categorical prediction of inactive vs active endoscopic disease evaluated against 2 independent test sets: a full set with a baseline single human expert read and a consensus subset in which 2 human reads agreed. Results: On the full test set of 147 videos, the model predicted inactive vs active endoscopic disease via the eMS with an area under the curve of 89%, accuracy of 84%, positive predictive value of 80%, and negative predictive value of 85%. In the consensus test set of 94 videos, the model predicted inactive vs active endoscopic disease with an area under the curve of 92%, accuracy of 89%, positive predictive value of 87%, and negative predictive value of 90%. Conclusion: We have demonstrated that this machine learning model supervised by mucosal features and eMS video annotations accurately differentiates key levels of endoscopic disease activity