Accuracy Measures for the Comparison of Classifiers

The selection of the best classification algorithm for a given dataset is a very widespread problem. It is also a complex one, in the sense it requires to make several important methodological choices. Among them, in this work we focus on the measure used to assess the classification performance and rank the algorithms. We present the most popular measures and discuss their properties. Despite the numerous measures proposed over the years, many of them turn out to be equivalent in this specific case, to have interpretation problems, or to be unsuitable for our purpose. Consequently, classic overall success rate or marginal rates should be preferred for this specific task.Comment: The 5th International Conference on Information Technology, amman : Jordanie (2011

A Statistical Modeling Approach to Computer-Aided Quantification of Dental Biofilm

Biofilm is a formation of microbial material on tooth substrata. Several methods to quantify dental biofilm coverage have recently been reported in the literature, but at best they provide a semi-automated approach to quantification with significant input from a human grader that comes with the graders bias of what are foreground, background, biofilm, and tooth. Additionally, human assessment indices limit the resolution of the quantification scale; most commercial scales use five levels of quantification for biofilm coverage (0%, 25%, 50%, 75%, and 100%). On the other hand, current state-of-the-art techniques in automatic plaque quantification fail to make their way into practical applications owing to their inability to incorporate human input to handle misclassifications. This paper proposes a new interactive method for biofilm quantification in Quantitative light-induced fluorescence (QLF) images of canine teeth that is independent of the perceptual bias of the grader. The method partitions a QLF image into segments of uniform texture and intensity called superpixels; every superpixel is statistically modeled as a realization of a single 2D Gaussian Markov random field (GMRF) whose parameters are estimated; the superpixel is then assigned to one of three classes (background, biofilm, tooth substratum) based on the training set of data. The quantification results show a high degree of consistency and precision. At the same time, the proposed method gives pathologists full control to post-process the automatic quantification by flipping misclassified superpixels to a different state (background, tooth, biofilm) with a single click, providing greater usability than simply marking the boundaries of biofilm and tooth as done by current state-of-the-art methods.Comment: 10 pages, 7 figures, Journal of Biomedical and Health Informatics 2014. keywords: {Biomedical imaging;Calibration;Dentistry;Estimation;Image segmentation;Manuals;Teeth}, http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6758338&isnumber=636350

Personality structure and social style in macaques.

Why regularities in personality can be described with particular dimensions is a basic question in differential psychology. Nonhuman primates can also be characterized in terms of personality structure. Comparative approaches can help reveal phylogenetic constraints and social and ecological patterns associated with the presence or absence of specific personality dimensions. We sought to determine how different personality structures are related to interspecific variation in social style. Specifically, we examined this question in 6 different species of macaques, because macaque social style is well characterized and can be categorized on a spectrum of despotic (Grade 1) versus tolerant (Grade 4) social styles. We derived personality structures from adjectival ratings of Japanese (Macaca fuscata; Grade 1), Assamese (M. assamensis; Grade 2), Barbary (M. sylvanus; Grade 3), Tonkean (M. tonkeana; Grade 4), and crested (M. nigra; Grade 4) macaques and compared these species with rhesus macaques (M. mulatta; Grade 1) whose personality was previously characterized. Using a nonparametric method, fuzzy set analysis, to identify commonalities in personality dimensions across species, we found that all but 1 species exhibited consistently defined Friendliness and Openness dimensions, but that similarities in personality dimensions capturing aggression and social competence reflect similarities in social styles. These findings suggest that social and phylogenetic relationships contribute to the origin, maintenance, and diversification of personality

Image Perceptual Similarity Metrics for the Assessment of Basal Cell Carcinoma.

Efficient management of basal cell carcinomas (BCC) requires reliable assessments of both tumors and post-treatment scars. We aimed to estimate image similarity metrics that account for BCC's perceptual color and texture deviation from perilesional skin. In total, 176 clinical photographs of BCC were assessed by six physicians using a visual deviation scale. Internal consistency and inter-rater agreement were estimated using Cronbach's α, weighted Gwet's AC2, and quadratic Cohen's kappa. The mean visual scores were used to validate a range of similarity metrics employing different color spaces, distances, and image embeddings from a pre-trained VGG16 neural network. The calculated similarities were transformed into discrete values using ordinal logistic regression models. The Bray-Curtis distance in the YIQ color model and rectified embeddings from the 'fc6' layer minimized the mean squared error and demonstrated strong performance in representing perceptual similarities. Box plot analysis and the Wilcoxon rank-sum test were used to visualize and compare the levels of agreement, conducted on a random validation round between the two groups: 'Human-System' and 'Human-Human.' The proposed metrics were comparable in terms of internal consistency and agreement with human raters. The findings suggest that the proposed metrics offer a robust and cost-effective approach to monitoring BCC treatment outcomes in clinical settings

Fully automated deep learning-based localization and segmentation of the locus coeruleus in aging and Parkinson's disease using neuromelanin-sensitive MRI

Purpose: Development and performance measurement of a fully automated pipeline that localizes and segments the locus coeruleus in so-called neuromelanin-sensitive magnetic resonance imaging data for the derivation of quantitative biomarkers of neurodegenerative diseases such as Alzheimer’s disease and Parkinson’s disease. Methods: We propose a pipeline composed of several 3D-Unet-based convolutional neural networks for iterative multi-scale localization and multi-rater segmentation and non-deep learning-based components for automated biomarker extraction. We trained on the healthy aging cohort and did not carry out any adaption or fine-tuning prior to the application to Parkinson’s disease subjects. Results: The localization and segmentation pipeline demonstrated sufficient performance as measured by Euclidean distance (on average around 1.3mm on healthy aging subjects and 2.2mm in Parkinson’s disease subjects) and Dice similarity coefficient (overall around 71% on healthy aging subjects and 60% for subjects with Parkinson’s disease) as well as promising agreement with respect to contrast ratios in terms of intraclass correlation coefficient of ≥0.80 for healthy aging subjects compared to a manual segmentation procedure. Lower values (≥0.48) for Parkinson’s disease subjects indicate the need for further investigation and tests before the application to clinical samples. Conclusion: These promising results suggest the usability of the proposed algorithm for data of healthy aging subjects and pave the way for further investigations using this approach on different clinical datasets to validate its practical usability more conclusively

Evaluation of Performance Measures for Classifiers Comparison

The selection of the best classification algorithm for a given dataset is a very widespread problem, occuring each time one has to choose a classifier to solve a real-world problem. It is also a complex task with many important methodological decisions to make. Among those, one of the most crucial is the choice of an appropriate measure in order to properly assess the classification performance and rank the algorithms. In this article, we focus on this specific task. We present the most popular measures and compare their behavior through discrimination plots. We then discuss their properties from a more theoretical perspective. It turns out several of them are equivalent for classifiers comparison purposes. Futhermore. they can also lead to interpretation problems. Among the numerous measures proposed over the years, it appears that the classical overall success rate and marginal rates are the more suitable for classifier comparison task

Scoring divergent thinking tests: A review and systematic framework

Divergent thinking tests are often used in creativity research as measures of creative potential. However, measurement approaches across studies vary to a great extent. One facet of divergent thinking measurement that contributes strongly to differences across studies is the scoring of participants’ responses. Most commonly, responses are scored for fluency, flexibility, and originality. However, even with respect to only one dimension (e.g., originality), scoring decisions vary extensively. In the current work, a systematic framework for practical scoring decisions was developed. Scoring dimensions, instruction-scoring fit, adequacy of responses, objectivity (vs. subjectivity), level of scoring (response vs. ideational pool level), and the method of aggregation were identified as determining factors of divergent thinking test scoring. In addition, recommendations and guidelines for making these decisions and reporting the information in papers have been provided

Configurations of Leadership Traits and Their Relation to Performance Ratings: A Person-Oriented Approach

The study of traits has re-emerged in the leadership literature despite its checkered past. There is now ample evidence that a variety of individual traits consistently relate to leadership effectiveness. Nonetheless, enormous ambiguity remains regarding the patterning of these traits within leaders and the implications of the various interactions among traits. A major contributor to these issues has been the failure to examine these traits within their founding theoretical context, as elements operating simultaneously as a configural system within the individual. Thus, this study examines the configurations of leadership traits in a sample of middle and upper-level managers. The main purposes of this paper are: 1) to describe clusters of within-person trait patterns in a sample of managers, and 2) to evaluate the extent to which these cluster profiles are related to performance ratings from a 360-degree feedback instrument and an assessment center. Results identified four stable clusters of managers based on the similarity of their leader trait patterns. The profile of each cluster was described and the following labels were provided: Action-Oriented Drivers, Interpersonal Achievers, Steadfast Introverts, and Apathetic Stoics. As hypothesized, these clustered displayed differences in both assessment center and multisource feedback ratings of leadership performance. For the most part, Interpersonal Achievers and Steadfast Introverts had the highest performance ratings across all dimensions and sources; however, a few interesting exceptions were revealed. Overall, results support the general premises of the person-oriented approach based on holistic interactionism theory. That is, a limited number of common trait patterns can be identified and used to describe individuals in leadership positions. In addition, based on the results of this study trait patterns assessed via a person-oriented approach are related to leadership performance and often provide a more precise explanation of leadership ratings than do individual or additive trait effects