Newman, Phil

Shaw, Angus

Witchalls, Jeremy

PURPOSE: Medial Tibial Stress Syndrome (MTSS) has been identified as the most costly musculoskeletal injury (MSKI) to the British army. There is no reliable treatment for MTSS and reoccurrence rates are high. Prevention of MTSS is critical to reducing operational burden. Typically injury prediction is complex, multivariate, and has not been capable of discerning individual level risks. This study investigated the accuracy of a machine learning approach that combines best known risk factors into an individual risk profiling tool for a common MSKI. METHODS: Using a prospective design, this study combined 10 risk factors identified in 2 previous systematic reviews, to determine the predictive accuracy of an ensemble of machine learners. Data was obtained from 123 recruits (28 females and 95 males). Follow-up was conducted at 3 months to determine those in the group that had developed MTSS. Four ensemble learning algorithms- logistic regression (LR), k- nearest neighbours (kNN), Naïve Bayes (NB) and Decision Tree (Tree) were deployed and trained 5 times on random stratified samples of 75% of the dataset. The resultant algorithms were tested on the remaining 25% of the dataset and the models were compared for classification accuracy, precision and recall. Where possible, visualisation tools were created to determine the interpretability of the resultant algorithms.RESULTS: Ranked classification accuracy was (Tree= 0.987, NB=0.897, LR=0.800, kNN=0.755).Models ranked by precision were (LR=1.000, Tree=0.952, NB=0.740, kNN=0.556).Models ranked by recall were (Tree= 0.987, NB=0.925, kNN=0.250, LR=0.225).Tree visualisation tool provided useful cut points to classify likely MTSS sufferers. NB visualization tool demonstrated useful capacity to model the effects of risk interventions in MTSS, allowing for a context of modifiable and non-modifiable factors. CONCLUSIONS: Tree and NB model analyses offer accurate individual level risk predictions as well as the capacity to model the effects of risk modifications for MTSS. These predictive models provide military institutions, clinicians and instructors with a strong and accurate calculator for predicting an individual recruit’s risk of MTSS. Further research must determine the generalizability of these findings. OPERATIONAL RELEVANCE: Accurate identification of individuals at risk of MTSS is an important advance in the management of this difficult and costly problem.The ability to mitigate occupational risk is increasingly a responsibility of commanders and trainers. MSKI is often complex and multifactorial, making prediction and management arduous. Machine learning methodologies can provide decision makers with better tools for MSKI control.<br/

Garnock, Cameron

University of Canberra Research Repository

A Machine Learning Algorithm to enhance decision making in the management of Medial Tibial Stress Syndrome

Background: Medial Tibial Stress Syndrome (MTSS) has been identified as the most costly musculoskeletal injury (MSKI) to the British army (Sharma, Greeves et al. 2015), affecting 35-80% of military trainees. There is no reliable treatment for MTSS and reoccurrence rates are high. Prevention of MTSS is critical to reducing operational burden. Typically injury prediction is complex, multivariate, and has not been capable of discerning individual level risks.Aim: This study investigated the accuracy of a machine learning approach that combines best known risk factors into an individual risk profiling tool for a common MSKI.Method: Using a prospective design, this study collected a suite of key variables, determined in a previous study of navy recruits (Garnock, Witchalls et al. 2018), in a new population from the Australian Defence Force Academy. Data was obtained from 107 recruits (35 females and 75 males). Follow-up was conducted at 3 months to determine those in the group that had developed MTSS, when a total of 99 recruits (69 males, 30 females) remained for inclusion in statistical analysis. Six ensemble learning algorithms- Decision Tree, Support Vector Machine, Logistic Regression, K-Nearest Neighbour, Random Forest and Naïve Bayes were deployed and trained 5 times on random stratified samples of 75% of the dataset. The resultant algorithms were tested on the remaining 25% of the dataset and the models were compared for classification accuracy, precision and recall, and Predictive Lift. The most accurate new algorithm was additionally deployed on the previous dataset of navy recruits in order to further validate the accuracy of the model.Results: Random Forest modelling was the most accurate with a classification accuracy of 92%, Area Under the Curve 98%, precision 91% and recall 92%. When this model was applied to an unrelated dataset it performed with similarly high classification accuracy of 93%, Area Under the Curve 93%, precision 93% and recall 93%. Lift curve analysis showed that treating the top 30% of at risk individuals would result in an 80% reduction in MTSS injuries.Conclusions: This model is highly accurate in predicting those who will develop the debilitating condition of MTSS. The model provides important preventive capacity which should be trialled as an intervention. References: Garnock, C., J. Witchalls and P. Newman (2018). "Predicting individual risk for medial tibial stress syndrome in navy recruits." Journal of Science and Medicine in Sport 21(6): 586-590.Sharma, J., J. P. Greeves, M. Byers, A. N. Bennett and I. R. Spears (2015). "Musculoskeletal injuries in British Army recruits: a prospective study of diagnosis-specific incidence and rehabilitation times." BMC Musculoskeletal Disorders 16(1): 1-7.<br/

Validation of a Machine Learning Algorithm to enhance decision making in the management of Medial Tibial Stress Syndrome

Abstract

Similar works

Full text

Available Versions

University of Canberra Research Repository

University of Canberra Research Repository