Comparing Robustness of Pairwise and Multiclass Neural-Network Systems for Face Recognition

Noise, corruptions and variations in face images can seriously hurt the performance of face recognition systems. To make such systems robust, multiclass neuralnetwork classifiers capable of learning from noisy data have been suggested. However on large face data sets such systems cannot provide the robustness at a high level. In this paper we explore a pairwise neural-network system as an alternative approach to improving the robustness of face recognition. In our experiments this approach is shown to outperform the multiclass neural-network system in terms of the predictive accuracy on the face images corrupted by noise

Classification of newborn EEG maturity with Bayesian averaging over decision trees

EEG experts can assess a newborn’s brain maturity by visual analysis of age-related patterns in sleep EEG. It is highly desirable to make the results of assessment most accurate and reliable. However, the expert analysis is limited in capability to provide the estimate of uncertainty in assessments. Bayesian inference has been shown providing the most accurate estimates of uncertainty by using Markov Chain Monte Carlo (MCMC) integration over the posterior distribution. The use of MCMC enables to approximate the desired distribution by sampling the areas of interests in which the density of distribution is high. In practice, the posterior distribution can be multimodal, and so that the existing MCMC techniques cannot provide the proportional sampling from the areas of interest. The lack of prior information makes MCMC integration more difficult when a model parameter space is large and cannot be explored in detail within a reasonable time. In particular, the lack of information about EEG feature importance can affect the results of Bayesian assessment of EEG maturity. In this paper we explore how the posterior information about EEG feature importance can be used to reduce a negative influence of disproportional sampling on the results of Bayesian assessment. We found that the MCMC integration tends to oversample the areas in which a model parameter space includes one or more features, the importance of which counted in terms of their posterior use is low. Using this finding, we proposed to cure the results of MCMC integration and then described the results of testing the proposed method on a set of sleep EEG recordings

Bayesian averaging over Decision Tree models for trauma severity scoring

Health care practitioners analyse possible risks of misleading decisions and need to estimate and quantify uncertainty in predictions. We have examined the “gold” standard of screening a patient's conditions for predicting survival probability, based on logistic regression modelling, which is used in trauma care for clinical purposes and quality audit. This methodology is based on theoretical assumptions about data and uncertainties. Models induced within such an approach have exposed a number of problems, providing unexplained fluctuation of predicted survival and low accuracy of estimating uncertainty intervals within which predictions are made. Bayesian method, which in theory is capable of providing accurate predictions and uncertainty estimates, has been adopted in our study using Decision Tree models. Our approach has been tested on a large set of patients registered in the US National Trauma Data Bank and has outperformed the standard method in terms of prediction accuracy, thereby providing practitioners with accurate estimates of the predictive posterior densities of interest that are required for making risk-aware decisions

Bayesian averaging over decision tree models: an application for estimating uncertainty in trauma severity scoring

Introduction For making reliable decisions, practitioners need to estimate uncertainties that exist in data and decision models. In this paper we analyse uncertainties of predicting survival probability for patients in trauma care. The existing prediction methodology employs logistic regression modelling of Trauma and Injury Severity Score(external) (TRISS), which is based on theoretical assumptions. These assumptions limit the capability of TRISS methodology to provide accurate and reliable predictions. Methods We adopt the methodology of Bayesian model averaging and show how this methodology can be applied to decision trees in order to provide practitioners with new insights into the uncertainty. The proposed method has been validated on a large set of 447,176 cases registered in the US National Trauma Data Bank in terms of discrimination ability evaluated with receiver operating characteristic (ROC) and precision–recall (PRC) curves. Results Areas under curves were improved for ROC from 0.951 to 0.956 (p = 3.89 × 10−18) and for PRC from 0.564 to 0.605 (p = 3.89 × 10−18). The new model has significantly better calibration in terms of the Hosmer–Lemeshow Hˆ" role="presentation"> statistic, showing an improvement from 223.14 (the standard method) to 11.59 (p = 2.31 × 10−18). Conclusion The proposed Bayesian method is capable of improving the accuracy and reliability of survival prediction. The new method has been made available for evaluation purposes as a web application

НОВЫЙ МЕТОД ПРОГНОЗИРОВАНИЯ ВЕРОЯТНОСТИ ВЫЖИВАНИЯ И ОЦЕНКИ НЕОПРЕДЕЛЕННОСТИ ДЛЯ ПАЦИЕНТОВ С ТРАВМАМИ

The Trauma and Injury Severity Score (TRISS) is the current “gold” standard of screening patient’s condition for purposes of predicting survival probability. More than 40 years of TRISS practice revealed a number of problems, particularly, 1) unexplained fluctuation of predicted values caused by aggregation of screening tests, and 2) low accuracy of uncertainty intervals estimations. We developed a new method made it available for practitioners as a web calculator to reduce negative effect of factors given above. The method involves Bayesian methodology of statistical inference which, being computationally expensive, in theory provides most accurate predictions. We implemented and tested this approach on a data set including 571,148 patients registered in the US National Trauma Data Bank (NTDB) with 1–20 injuries. These patients were distributed over the following categories: (1) 174,647 with 1 injury, (2) 381,137 with 2–10 injuries, and (3) 15,364 with 11–20 injuries. Survival rates in each category were 0.977, 0.953, and 0.831, respectively. The proposed method has improved prediction accuracy by 0.04%, 0.36%, and 3.64% (p-value <0.05) in the categories 1, 2, and 3, respectively. Hosmer-Lemeshow statistics showed a significant improvement of the new model calibration. The uncertainty 2σ intervals were reduced from 0.628 to 0.569 for patients of the second category and from 1.227 to 0.930 for patients of the third category, both with p-value <0.005. The new method shows the statistically significant improvement (p-value <0.05) in accuracy of predicting survival and estimating the uncertainty intervals. The largest improvement has been achieved for patients with 11–20 injuries. The method is available for practitioners as a web calculator http://www.traumacalc.org.Для оценки тяжести травм и повреждений в настоящее время используется «золотой» стандарт Trauma and Injury Severity Score (TRISS), предназначенный для скрининга состояния пациента с целью предсказания вероятности выживания. Однако использование этого стандарта в течение более чем 40 лет выявило ряд проблем: первое — необъяснимую флуктуацию предсказанных значений, вызванную агрегированием скрининговых тестов, второе — недостаточную точность оценки интервалов неопределенности, в которых распределены предсказания. Для снижения негативного влияния этих факторов нами разработан новый метод и сделан доступным для практиков в виде web-калькулятора. Метод использует байесовскую методологию статистического вывода, которая теоретически позволяет достичь максимальной точности предсказаний, являясь, однако, вычислительно сложной в реализации. Метод был реализован и верифицирован нами на выборке данных, включающей 571 148 пациентов, зарегистрированных в US National Trauma Data Bank (NTDB), с числом травм от 1 до 20. Распределение пациентов по группам по числу травм: 1-я группа — 174 647 пациентов имели 1 травму, 2-я группа — 381 137— от 2 до 10 травм и 3-я группа 15 364 — от 11 до 20 травм. Доли выживших в каждой категории были 0,977, 0,953 и 0,831 соответственно. Предложенный нами метод улучшил точность предсказаний на 0,04%, 0,36% и 3,64% (значимость p<0,05) для каждой указанной группы. Критерий Хосмер-Лемешоу показал значительное улучшение калибрации новой модели. Интервалы неопределенности 2σ были снижены с 0,628 до 0,569 для пациентов 2-й группы и с 1,227 до 0,930 для пациентов 3-й группы (p<0,005). Новый метод показал статистически значимое улучшение как точности предсказания выживания, так и точности оценки интервалов неопределенности. Наибольшее улучшение достигается для пациентов 3-й группы. Метод сделан доступным для практиков как web-калькулятор http://www.traumacalc.org

Bayesian Predictive Modelling: Application to Aircraft Short-Term Conflict Alert System

Abstract Bayesian Model Averaging (BMA), computationally feasible using Markov Chain Monte Carlo (MCMC), is a well-known method for reliable estimation of predictive distributions. The use of decision tree (DT) models for the averaging enables experts not only to estimate a predictive posterior but also to interpret models of interest and estimate the importance of predictor factors that are assumed to contribute to the prediction. The MCMC method generates parameters of DT models in order to explore their posterior distributions and to draw samples from the models. However, these samples can often overrepresent DT models of an excessive size, which in cases of real-world applications affects the results of BMA. When this happens, it is unlikely for a DT model that provides Maximum a Posteriori probability to explain the observed data with high accuracy. We propose a new technology in order to estimate and interpret predictive posteriors. In our experiments with aircraft short-term conflict alerts, we show how this technology can be used for analysing uncertainties in detections of conflicts

A NEW METHOD FOR PREDICTING SURVIVAL AND ESTIMATING UNCERTAINTY IN TRAUMA PATIENTS

The Trauma and Injury Severity Score (TRISS) is the current “gold” standard of screening patient’s condition for purposes of predicting survival probability. More than 40 years of TRISS practice revealed a number of problems, particularly, 1) unexplained fluctuation of predicted values caused by aggregation of screening tests, and 2) low accuracy of uncertainty intervals estimations. We developed a new method made it available for practitioners as a web calculator to reduce negative effect of factors given above. The method involves Bayesian methodology of statistical inference which, being computationally expensive, in theory provides most accurate predictions. We implemented and tested this approach on a data set including 571,148 patients registered in the US National Trauma Data Bank (NTDB) with 1–20 injuries. These patients were distributed over the following categories: (1) 174,647 with 1 injury, (2) 381,137 with 2–10 injuries, and (3) 15,364 with 11–20 injuries. Survival rates in each category were 0.977, 0.953, and 0.831, respectively. The proposed method has improved prediction accuracy by 0.04%, 0.36%, and 3.64% (p-value <0.05) in the categories 1, 2, and 3, respectively. Hosmer-Lemeshow statistics showed a significant improvement of the new model calibration. The uncertainty 2σ intervals were reduced from 0.628 to 0.569 for patients of the second category and from 1.227 to 0.930 for patients of the third category, both with p-value <0.005. The new method shows the statistically significant improvement (p-value <0.05) in accuracy of predicting survival and estimating the uncertainty intervals. The largest improvement has been achieved for patients with 11–20 injuries. The method is available for practitioners as a web calculator http://www.traumacalc.org