2 research outputs found
Graph-Based Intercategory and Intermodality Network for Multilabel Classification and Melanoma Diagnosis of Skin Lesions in Dermoscopy and Clinical Images
The identification of melanoma involves an integrated analysis of skin lesion
images acquired using the clinical and dermoscopy modalities. Dermoscopic
images provide a detailed view of the subsurface visual structures that
supplement the macroscopic clinical images. Melanoma diagnosis is commonly
based on the 7-point visual category checklist (7PC). The 7PC contains
intrinsic relationships between categories that can aid classification, such as
shared features, correlations, and the contributions of categories towards
diagnosis. Manual classification is subjective and prone to intra- and
interobserver variability. This presents an opportunity for automated methods
to improve diagnosis. Current state-of-the-art methods focus on a single image
modality and ignore information from the other, or do not fully leverage the
complementary information from both modalities. Further, there is not a method
to exploit the intercategory relationships in the 7PC. In this study, we
address these issues by proposing a graph-based intercategory and intermodality
network (GIIN) with two modules. A graph-based relational module (GRM)
leverages intercategorical relations, intermodal relations, and prioritises the
visual structure details from dermoscopy by encoding category representations
in a graph network. The category embedding learning module (CELM) captures
representations that are specialised for each category and support the GRM. We
show that our modules are effective at enhancing classification performance
using a public dataset of dermoscopy-clinical images, and show that our method
outperforms the state-of-the-art at classifying the 7PC categories and
diagnosis