A unifying view for performance measures in multi-class prediction

In the last few years, many different performance measures have been introduced to overcome the weakness of the most natural metric, the Accuracy. Among them, Matthews Correlation Coefficient has recently gained popularity among researchers not only in machine learning but also in several application fields such as bioinformatics. Nonetheless, further novel functions are being proposed in literature. We show that Confusion Entropy, a recently introduced classifier performance measure for multi-class problems, has a strong (monotone) relation with the multi-class generalization of a classical metric, the Matthews Correlation Coefficient. Computational evidence in support of the claim is provided, together with an outline of the theoretical explanation

Deep Learning Based Human Emotional State Recognition in a Video

Human emotions play significant role in everyday life. There are a lot of applications of automatic emotion recognition in medicine, e-learning, monitoring, marketing etc. In this paper the method and neural network architecture for real-time human emotion recognition by audio-visual data are proposed. To classify one of seven emotions, deep neural networks, namely, convolutional and recurrent neural networks are used. Visual information is represented by a sequence of 16 frames of 96 × 96 pixels, and audio information - by 140 features for each of a sequence of 37 temporal windows. To reduce the number of audio features autoencoder was used. Audio information in conjunction with visual one is shown to increase recognition accuracy up to 12%. The developed system being not demanding to be computing resources is dynamic in terms of selection of parameters, reducing or increasing the number of emotion classes, as well as the ability to easily add, accumulate and use information from other external devices for further improvement of classification accuracy

Вариация пигментации радужки глаз белорусской популяции в связи с полиморфизмом генов HERC2 и OCA2

The human genetic phenotyping is one of the most intensely developing area of forensic genetics. Externally visible traits, including eye color, can be predicted by analyzing single nucleotide polymorphisms with a high predictive rate. We studied the polymorphisms rs12913832 and rs1800407 in the HERC2 and OCA2 genes, respectively, to evaluate its prognostic availability in relation to the iris pigmentation of the Belarusian population. For this, both eye images and DNA samples were collected from 314 individuals to analyze the key polymorphisms by the TaqMan assay. Our data confirmed a relevance of rs12913832:A>G and rs1800407:G>A in the prediction context. The highest values of the sensitivity (SE = 0.94) and the specificity (SP = 0.90) were obtained for rs12913832, demonstrating the high efficiency of this marker as a classifier of phenotypic groups. The presence of the ancestral dominant allele rs12913832-A causes a dark (brown) iris pigmentation, how- ever, the heterozygous state rs12913832:GA includes a range of mixed variants. The predictive value of rs1800407 for the genetic phenotyping is highly significant (SE = 0.98), but has a low specificity (SP = 0.14), thus rs1800407, not being an effective classifier, can be used as an auxiliary in the eye color predictive model. The analysis of a cumulative impact of the both poly- morphisms on the iris color variation shows their high prospects for the genetic phenotyping of the Belarusian population.Генетическое фенотипирование человека – новое, интенсивно развивающееся направление криминалистической генетики. Исследование генетических основ цветовой вариации глаз является одним из наиболее перспективных среди подходов, нацеленных на установление облика неизвестного индивида по характеристикам его ДНК. В данной работе проведено изучение полиморфизмов rs12913832 и rs1800407 в генах HERC2 и ОСA2 соответственно в связи с пигментацией радужки глаз белорусской популяции и дана оценка их прогностической эффективности для генетического фенотипирования. Полученные данные подтвердили значимый вклад в цветовую вариацию радужки глаз rs12913832:A>G и rs1800407:G>A. Высокие значения чувствительности (SE = 0,94) и специфичности (SP = 0,90) были получены для rs12913832, подтвердив эффективность данного маркера для использования в качестве классификатора фенотипических групп. Наличие предкового доминантного аллеля rs12913832-A обусловливает темную пигментацию радужки, однако гетерозиготное носительство rs12913832:GA включает значительный спектр смешанных вариантов. Однонуклеотидный полиморфизм rs1800407 характеризуется высокой чувствительностью (SE = 0,98), однако имеет низкое значение специфичности (SP = 0,14), следовательно, данный маркер, не являясь эффективным классификатором, может использоваться только как вспомогательный инструмент для предсказания цвета глаз. Оценка совокупного вклада изученных полиморфизмов в цветовую вариацию радужки глаз белорусской популяции показывает их высокий прогностический потенциал для генетического фенотипирования

Usefulness of the SF-36 Health Survey questionnaire in screening for health-related quality of life among parents of children with cancer: Latent profile analysis

Background: Poor health-related quality of life (HRQOL) of parents of children with cancer as their main caregivers can adversely affect child’s HRQOL. Short Form-36 Health Survey (SF-36) is a widely used instrument to measure HRQOL. However, there are no clearly defined cut-off points for screening for parents with poor HRQOL. This study aimed to find appropriate cut-off points for the SF-36 questionnaire in a sample of parents of children with cancer using latent profile analysis to add another possibility to use it. Methods: In this cross-sectional study, 110 couples (110 mothers and 110 fathers) of children diagnosed with cancer selected by simple random sampling method from the patients' files were included. The study was conducted at two settings, pediatric hematology ward and pediatric hematology clinic of a university hospital in 2016-2017. Latent Profile analysis method was used to determine appropriate cut-off points for the SF-36 questionnaire. Data was analyzed by Mplus and R3.3.0 software. Results:  Based on the results, scores ≤44, 45-63 and ≥64 for mental health, and scores ≤43, 44-59 and ≥60 for physical health classes indicate weak, medium, and good, respectively. These cut-off points showed acceptable accuracy in classification of individuals. For the total quality of life, correct classification rates were 88%, 65% and 53% for each class respectively. For mental health (physical health), they were 79 (63), 50 (62) and 52 (63) for each class respectively.    Conclusion: The cut-off points for the classes identified here can be useful in screening parents of children with cancer in clinical setting to provide clinical interventions to protect vulnerable parents from negative outcomes

Quantifying Explainability of Saliency Methods in Deep Neural Networks

One way to achieve eXplainable artificial intelligence (XAI) is through the use of post-hoc analysis methods. In particular, methods that generate heatmaps have been used to explain black-box models, such as deep neural network. In some cases, heatmaps are appealing due to the intuitive and visual ways to understand them. However, quantitative analysis that demonstrates the actual potential of heatmaps have been lacking, and comparison between different methods are not standardized as well. In this paper, we introduce a synthetic dataset that can be generated adhoc along with the ground-truth heatmaps for better quantitative assessment. Each sample data is an image of a cell with easily distinguishable features, facilitating a more transparent assessment of different XAI methods. Comparison and recommendations are made, shortcomings are clarified along with suggestions for future research directions to handle the finer details of select post-hoc analysis methods

DiagTest3Grp: An R Package for Analyzing Diagnostic Tests with Three Ordinal Groups

Medical researchers endeavor to identify potentially useful biomarkers to develop markerbased screening assays for disease diagnosis and prevention. Useful summary measures which properly evaluate the discriminative ability of diagnostic markers are critical for this purpose. Literature and existing software, for example, R packages nicely cover summary measures for diagnostic markers used for the binary case (e.g., healthy vs. diseased). An intermediate population at an early disease stage usually exists between the healthy and the fully diseased population in many disease processes. Supporting utilities for threegroup diagnostic tests are highly desired and important for identifying patients at the early disease stage for timely treatments. However, application packages which provide summary measures for three ordinal groups are currently lacking. This paper focuses on two summary measures of diagnostic accuracy—volume under the receiver operating characteristic surface and the extended Youden index, with three diagnostic groups. We provide the R package DiagTest3Grp to estimate, under both parametric and nonparametric assumptions, the two summary measures and the associated variances, as well as the optimal cut-points for disease diagnosis. An omnibus test for multiple markers and a Wald test for two markers, on independent or paired samples, are incorporated to compare diagnostic accuracy across biomarkers. Sample size calculation under the normality assumption can be performed in the R package to design future diagnostic studies. A real world application evaluating the diagnostic accuracy of neuropsychological markers for Alzheimer’s disease is used to guide readers through step-by-step implementation of DiagTest3Grp to demonstrate its utility

Multi-objective optimisation for receiver operating characteristic analysis

Copyright © 2006 Springer-Verlag Berlin Heidelberg. The final publication is available at link.springer.comBook title: Multi-Objective Machine LearningSummary Receiver operating characteristic (ROC) analysis is now a standard tool for the comparison of binary classifiers and the selection operating parameters when the costs of misclassification are unknown. This chapter outlines the use of evolutionary multi-objective optimisation techniques for ROC analysis, in both its traditional binary classification setting, and in the novel multi-class ROC situation. Methods for comparing classifier performance in the multi-class case, based on an analogue of the Gini coefficient, are described, which leads to a natural method of selecting the classifier operating point. Illustrations are given concerning synthetic data and an application to Short Term Conflict Alert