83 research outputs found
Agreement Between Experts and an Untrained Crowd for Identifying Dermoscopic Features Using a Gamified App: Reader Feasibility Study
Background
Dermoscopy is commonly used for the evaluation of pigmented lesions, but agreement between experts for identification of dermoscopic structures is known to be relatively poor. Expert labeling of medical data is a bottleneck in the development of machine learning (ML) tools, and crowdsourcing has been demonstrated as a cost- and time-efficient method for the annotation of medical images.
Objective
The aim of this study is to demonstrate that crowdsourcing can be used to label basic dermoscopic structures from images of pigmented lesions with similar reliability to a group of experts.
Methods
First, we obtained labels of 248 images of melanocytic lesions with 31 dermoscopic âsubfeaturesâ labeled by 20 dermoscopy experts. These were then collapsed into 6 dermoscopic âsuperfeaturesâ based on structural similarity, due to low interrater reliability (IRR): dots, globules, lines, network structures, regression structures, and vessels. These images were then used as the gold standard for the crowd study. The commercial platform DiagnosUs was used to obtain annotations from a nonexpert crowd for the presence or absence of the 6 superfeatures in each of the 248 images. We replicated this methodology with a group of 7 dermatologists to allow direct comparison with the nonexpert crowd. The Cohen Îș value was used to measure agreement across raters.
Results
In total, we obtained 139,731 ratings of the 6 dermoscopic superfeatures from the crowd. There was relatively lower agreement for the identification of dots and globules (the median Îș values were 0.526 and 0.395, respectively), whereas network structures and vessels showed the highest agreement (the median Îș values were 0.581 and 0.798, respectively). This pattern was also seen among the expert raters, who had median Îș values of 0.483 and 0.517 for dots and globules, respectively, and 0.758 and 0.790 for network structures and vessels. The median Îș values between nonexperts and thresholded averageâexpert readers were 0.709 for dots, 0.719 for globules, 0.714 for lines, 0.838 for network structures, 0.818 for regression structures, and 0.728 for vessels.
Conclusions
This study confirmed that IRR for different dermoscopic features varied among a group of experts; a similar pattern was observed in a nonexpert crowd. There was good or excellent agreement for each of the 6 superfeatures between the crowd and the experts, highlighting the similar reliability of the crowd for labeling dermoscopic images. This confirms the feasibility and dependability of using crowdsourcing as a scalable solution to annotate large sets of dermoscopic images, with several potential clinical and educational applications, including the development of novel, explainable ML tools
Statistical techniques applied to the automatic diagnosis of dermoscopic images
An image based system implementing a wellâknown diagnostic method is disclosed for the automatic detection of melanomas as
support to clinicians. The software procedure is able to recognize automatically the skin lesion within the digital image, measure
morphological and chromatic parameters, carry out a suitable classification for the detection of structural dermoscopic criteria
provided by the 7âPoint Check. Original contribution is referred to advanced statistical techniques, which are introduced at different
stages of the image processing, including the border detection, the extraction of lowâlevel features and scoring of high order features
(namely dermoscopic criteria). The proposed approach is experimentally tested with reference to a large image set of pigmented
lesions
Automatic Detection of Blue-White Veil and Related Structures in Dermoscopy Images
Dermoscopy is a non-invasive skin imaging technique, which permits
visualization of features of pigmented melanocytic neoplasms that are not
discernable by examination with the naked eye. One of the most important
features for the diagnosis of melanoma in dermoscopy images is the blue-white
veil (irregular, structureless areas of confluent blue pigmentation with an
overlying white "ground-glass" film). In this article, we present a machine
learning approach to the detection of blue-white veil and related structures in
dermoscopy images. The method involves contextual pixel classification using a
decision tree classifier. The percentage of blue-white areas detected in a
lesion combined with a simple shape descriptor yielded a sensitivity of 69.35%
and a specificity of 89.97% on a set of 545 dermoscopy images. The sensitivity
rises to 78.20% for detection of blue veil in those cases where it is a primary
feature for melanoma recognition
- âŠ