Handling limited datasets with neural networks in medical applications : a small-data approach

Motivation: Single-centre studies in medical domain are often characterised by limited samples due to the complexity and high costs of patient data collection. Machine learning methods for regression modelling of small datasets (less than 10 observations per predictor variable) remain scarce. Our work bridges this gap by developing a novel framework for application of artificial neural networks (NNs) for regression tasks involving small medical datasets. Methods: In order to address the sporadic fluctuations and validation issues that appear in regression NNs trained on small datasets, the method of multiple runs and surrogate data analysis were proposed in this work. The approach was compared to the state-of-the-art ensemble NNs; the effect of dataset size on NN performance was also investigated. Results: The proposed framework was applied for the prediction of compressive strength (CS) of femoral trabecular bone in patients suffering from severe osteoarthritis. The NN model was able to estimate the CS of osteoarthritic trabecular bone from its structural and biological properties with a standard error of 0.85 MPa. When evaluated on independent test samples, the NN achieved accuracy of 98.3%, outperforming an ensemble NN model by 11%. We reproduce this result on CS data of another porous solid (concrete) and demonstrate that the proposed framework allows for an NN modelled with as few as 56 samples to generalise on 300 independent test samples with 86.5% accuracy, which is comparable to the performance of an NN developed with 18 times larger dataset (1030 samples). Conclusion: The significance of this work is two-fold: the practical application allows for non-destructive prediction of bone fracture risk, while the novel methodology extends beyond the task considered in this study and provides a general framework for application of regression NNs to medical problems characterised by limited dataset sizes

Decision tree and random forest models for outcome prediction in antibody incompatible kidney transplantation

Clinical datasets are commonly limited in size, thus restraining applications of Machine Learning (ML)techniques for predictive modelling in clinical research and organ transplantation. We explored thepotential of Decision Tree (DT) and Random Forest (RF) classification models, in the context of smalldataset of 80 samples, for outcome prediction in high-risk kidney transplantation. The DT and RF modelsidentified the key risk factors associated with acute rejection: the levels of the donor specific IgG anti-bodies, the levels of IgG4 subclass and the number of human leucocyte antigen mismatches betweenthe donor and recipient. Furthermore, the DT model determined dangerous levels of donor-specific IgGsubclass antibodies, thus demonstrating the potential of discovering new properties in the data whentraditional statistical tools are unable to capture them. The DT and RF classifiers developed in this workpredicted early transplant rejection with accuracy of 85%, thus offering an accurate decision supporttool for doctors tasked with predicting outcomes of kidney transplantation in advance of the clinicalintervention

Subclass analysis of donor HLA-specific IgG in antibody-incompatible renal transplantation reveals a significant association of IgG4 with rejection and graft failure

Donor HLA-specific antibodies (DSAs) can cause rejection and graft loss after renal transplantation, but their levels measured by the current assays are not fully predictive of outcomes. We investigated whether IgG subclasses of DSA were associated with early rejection and graft failure. DSA levels were determined pretreatment, at the day of peak pan-IgG level and at 30 days post-transplantation in eighty HLA antibody-incompatible kidney transplant recipients using a modified microbead assay. Pretreatment IgG4 levels were predictive of acute antibody-mediated rejection (P = 0.003) in the first 30 days post-transplant. Pre-treatment presence of IgG4 DSA (P = 0.008) and day 30 IgG3 DSA (P = 0.03) was associated with poor graft survival. Multivariate regression analysis showed that in addition to pan-IgG levels, total IgG4 levels were an independent risk factor for early rejection when measured pretreatment, and the presence of pretreatment IgG4 DSA was also an independent risk factor for graft failure. Pretreatment IgG4 DSA levels correlated independently with higher risk of early rejection episodes and medium-term death-censored graft survival. Thus, pretreatment IgG4 DSA may be used as a biomarker to predict and risk stratify cases with higher levels of pan-IgG DSA in HLA antibody-incompatible transplantation. Further investigations are needed to confirm our results

Neural networks for analysis of trabecular bone in osteoarthritis

This study investigated the correlation of age in male and female specimens with physico-mechanical properties of trabecular bone including compressive strength, bone volume fraction, structural model index, trabecular thickness factor, level of inter-connectivity and pore morphology. An artificial neural network was designed to analyse 35 available samples in order to account for complex inter-dependencies of the key parameters in multi-dimensional space. Trained by using Levenberg-Marquardt back propagation algorithm, the network achieved regression factor of 0·96 by optimisation and showed that age correlates strongly with the physical properties of the bone affected by severe osteoarthritis. In addition, the compressive strength was found to be the most important factor for predicting the bone aging. Within the limitations of the input data set, the model developed provides a reliable predictive tool to tissue engineering applications

Artificial neural networks in hard tissue engineering : another look at age-dependence of trabecular bone properties in osteoarthritis

Artificial Neural Network (ANN) model has been developed to correlate age of severely osteoarthritic male and female specimens with key mechanical and structural characteristics of their trabecular bone. The complex interdependency between age, gender, compressive strength, porosity, morphology and level of interconnectivity was analysed in multi-dimensional space using a two-layer feedforward ANN. Trained by Levenberg-Marquardt back propagation algorithm, the ANN achieved regression factor of R = 96.3% between the predicted and target age when optimised for the experimental dataset. Results indicate a strong correlation of the 5-dimensional vector of physical properties of the bone with the age of the specimens. The inverse problem of estimating compressive strength as the key bone fracture risk was also investigated. The outcomes yield correlation between predicted and target compressive strength with the regression factor of R = 97.4%. Within the limitations of the input data set, the ANNs provide robust predictive models for hard tissue engineering decision support

Machine learning with limited information : risk stratification and predictive modelling for clinical applications.

The high cost, complexity and multimodality of clinical data collection restrain the datasets available for predictive modelling using machine learning (ML), thus necessitating new data-efficient approaches specifically for limited datasets. This interdisciplinary thesis focuses on clinical outcome modelling using a range of ML techniques, including artificial neural networks (NNs) and their ensembles, decision trees (DTs) and random forests (RFs), as well as classical logistic regression (LR) and Cox proportional hazards (Cox PH) models. The utility of ML for data-efficient regression, classification and survival analyses was investigated in three clinical applications, whereby exposing the common limitations inherent in patient data, such as class imbalance, incomplete samples, and, in particular, limited dataset size. The latter problem was addressed by developing a methodological framework for learning from datasets with less than 10 observations per predictor variable. A novel method of multiple runs overcame the volatility of NN and DT models due to limited training samples, while a surrogate data test allowed for regression model evaluation in the presence of noise due to limited dataset size. When applied to hard tissue engineering for predicting femoral fracture risk, the framework resulted in 98.3% accurate regression NN. The framework was used to detect early rejection in antibody- incompatible kidney transplantation, achieving 85% accurate classification DT. The third clinical task – that of predicting 10-year incidence of type 2 diabetes in the UK population – resulted in 70-85% accurate classification and survival models, whilst highlighting the challenges of learning with the limited information characteristic of routinely collected data. By discovering unintuitive patterns, supporting existing hypotheses and generating novel insight, the ML models developed in this research contributed meaningfully to clinical research and paved the way for data-efficient applications of ML in engineering and clinical practice

Data for Decision tree and random forest models for outcome prediction in antibody incompatible kidney transplantation

Clinical datasets are commonly limited in size, thus restraining applications of Machine Learning (ML)techniques for predictive modelling in clinical research and organ transplantation. We explored thepotential of Decision Tree (DT) and Random Forest (RF) classification models, in the context of smalldataset of 80 samples, for outcome prediction in high-risk kidney transplantation. The DT and RF modelsidentified the key risk factors associated with acute rejection: the levels of the donor specific IgG anti-bodies, the levels of IgG4 subclass and the number of human leucocyte antigen mismatches betweenthe donor and recipient. Furthermore, the DT model determined dangerous levels of donor-specific IgGsubclass antibodies, thus demonstrating the potential of discovering new properties in the data whentraditional statistical tools are unable to capture them. The DT and RF classifiers developed in this workpredicted early transplant rejection with accuracy of 85%, thus offering an accurate decision supporttool for doctors tasked with predicting outcomes of kidney transplantation in advance of the clinicalintervention

Machine learning for predictive modelling based on small data in biomedical engineering

Experimental datasets in bioengineering are commonly limited in size, thus rendering Machine Learning (ML) impractical for predictive modelling. Novel techniques of multiple runs for model development and surrogate data analysis for model validation are suggested for prediction of biomedical outcomes based on small datasets for classification and regression tasks. The proposed framework was applied to designing a Neural Network model for osteoarthritic bone fracture risk stratification, and a Decision Tree model for prediction of antibody-mediated kidney transplant rejection. Despite the small datasets (35 bone specimens and 80 kidney transplants), the two models achieved high accuracy of 98.3% and 85%, respectively