A Query-by-Example Content-Based Image Retrieval System of Non-melanoma Skin Lesions

Abstract. This paper proposes a content-based image retrieval system for skin lesion images as a diagnostic aid. The aim is to support decision making by retrieving and displaying relevant past cases visually similar to the one under examination. Skin lesions of five common classes, including two non-melanoma cancer types are used. Colour and texture features are extracted from lesions. Feature selection is achieved by optimising a similarity matching function. Experiments on our database of 208 images are performed and results evaluated.

Content-Based Image Retrieval of Skin Lesions by Evolutionary Feature Synthesis

Abstract. This paper gives an example of evolved features that improve image retrieval performance. A content-based image retrieval system for skin lesion images is presented. The aim is to support decision making by retrieving and displaying relevant past cases visually similar to the one under examination. Skin lesions of five common classes, including two non-melanoma cancer types, are used. Colour and texture features are extracted from lesions. Evolutionary algorithms are used to create composite features that optimise a similarity matching function. Experiments on our database of 533 images are performed and results are compared to those obtained using simple features. The use of the evolved composite features improves the precision by about 7%.

Saliency-Enhanced Content-Based Image Retrieval for Diagnosis Support in Dermatology Consultation: Reader Study.

BACKGROUND Previous research studies have demonstrated that medical content image retrieval can play an important role by assisting dermatologists in skin lesion diagnosis. However, current state-of-the-art approaches have not been adopted in routine consultation, partly due to the lack of interpretability limiting trust by clinical users. OBJECTIVE This study developed a new image retrieval architecture for polarized or dermoscopic imaging guided by interpretable saliency maps. This approach provides better feature extraction, leading to better quantitative retrieval performance as well as providing interpretability for an eventual real-world implementation. METHODS Content-based image retrieval (CBIR) algorithms rely on the comparison of image features embedded by convolutional neural network (CNN) against a labeled data set. Saliency maps are computer vision-interpretable methods that highlight the most relevant regions for the prediction made by a neural network. By introducing a fine-tuning stage that includes saliency maps to guide feature extraction, the accuracy of image retrieval is optimized. We refer to this approach as saliency-enhanced CBIR (SE-CBIR). A reader study was designed at the University Hospital Zurich Dermatology Clinic to evaluate SE-CBIR's retrieval accuracy as well as the impact of the participant's confidence on the diagnosis. RESULTS SE-CBIR improved the retrieval accuracy by 7% (77% vs 84%) when doing single-lesion retrieval against traditional CBIR. The reader study showed an overall increase in classification accuracy of 22% (62% vs 84%) when the participant is provided with SE-CBIR retrieved images. In addition, the overall confidence in the lesion's diagnosis increased by 24%. Finally, the use of SE-CBIR as a support tool helped the participants reduce the number of nonmelanoma lesions previously diagnosed as melanoma (overdiagnosis) by 53%. CONCLUSIONS SE-CBIR presents better retrieval accuracy compared to traditional CBIR CNN-based approaches. Furthermore, we have shown how these support tools can help dermatologists and residents improve diagnosis accuracy and confidence. Additionally, by introducing interpretable methods, we should expect increased acceptance and use of these tools in routine consultation

Saliency-Enhanced Content-Based Image Retrieval for Diagnosis Support in Dermatology Consultation: Reader Study

BACKGROUND Previous research studies have demonstrated that medical content image retrieval can play an important role by assisting dermatologists in skin lesion diagnosis. However, current state-of-the-art approaches have not been adopted in routine consultation, partly due to the lack of interpretability limiting trust by clinical users. OBJECTIVE This study developed a new image retrieval architecture for polarized or dermoscopic imaging guided by interpretable saliency maps. This approach provides better feature extraction, leading to better quantitative retrieval performance as well as providing interpretability for an eventual real-world implementation. METHODS Content-based image retrieval (CBIR) algorithms rely on the comparison of image features embedded by convolutional neural network (CNN) against a labeled data set. Saliency maps are computer vision-interpretable methods that highlight the most relevant regions for the prediction made by a neural network. By introducing a fine-tuning stage that includes saliency maps to guide feature extraction, the accuracy of image retrieval is optimized. We refer to this approach as saliency-enhanced CBIR (SE-CBIR). A reader study was designed at the University Hospital Zurich Dermatology Clinic to evaluate SE-CBIR's retrieval accuracy as well as the impact of the participant's confidence on the diagnosis. RESULTS SE-CBIR improved the retrieval accuracy by 7% (77% vs 84%) when doing single-lesion retrieval against traditional CBIR. The reader study showed an overall increase in classification accuracy of 22% (62% vs 84%) when the participant is provided with SE-CBIR retrieved images. In addition, the overall confidence in the lesion's diagnosis increased by 24%. Finally, the use of SE-CBIR as a support tool helped the participants reduce the number of nonmelanoma lesions previously diagnosed as melanoma (overdiagnosis) by 53%. CONCLUSIONS SE-CBIR presents better retrieval accuracy compared to traditional CBIR CNN-based approaches. Furthermore, we have shown how these support tools can help dermatologists and residents improve diagnosis accuracy and confidence. Additionally, by introducing interpretable methods, we should expect increased acceptance and use of these tools in routine consultation

Artificial Intelligence in Skin Cancer: A Literature Review from Diagnosis to Prevention and Beyond

Artificial Intelligence (AI) in medicine is quickly expanding, offering significant potential benefits in diagnosis and prognostication. While concerns may exist regarding its implementation, it is important for dermatologists and dermatopathologists to collaborate with technical specialists to embrace AI as a tool for enhancing medical decision-making and improving healthcare accessibility. This is particularly relevant in melanocytic neoplasms, which continue to present challenges despite years of experience. Dermatology, with its extensive medical data and images, provides an ideal field for training AI algorithms to enhance patient care. Collaborative efforts between medical professionals and technical specialists are crucial in harnessing the power of AI while ensuring it complements and enhances the existing healthcare framework. By staying informed about AI concepts and ongoing research, dermatologists can remain at the forefront of this emerging field and leverage its potential to improve patient outcomes. In conclusion, AI holds great promise in dermatology, especially in the management and analysis of Skin cancer (SC). In this review we strive to introduce the concepts of AI and its association with dermatology, providing an overview of recent studies in the field, such as existing applications and future potential in dermatology

Towards an Effective Imaging-Based Decision Support System for Skin Cancer

The usage of expert systems to aid in medical decisions has been employed since 1980s in distinct ap plications. With the high demands of medical care and limited human resources, these technologies are required more than ever. Skin cancer has been one of the pathologies with higher growth, which suf fers from lack of dermatology experts in most of the affected geographical areas. A permanent record of examination that can be further analyzed are medical imaging modalities. Most of these modalities were also assessed along with machine learning classification methods. It is the aim of this research to provide background information about skin cancer types, medical imaging modalities, data mining and machine learning methods, and their application on skin cancer imaging, as well as the disclosure of a proposal of a multi-imaging modality decision support system for skin cancer diagnosis and treatment assessment based in the most recent available technology. This is expected to be a reference for further implementation of imaging-based clinical support systems.info:eu-repo/semantics/publishedVersio

Human-computer collaboration for skin cancer recognition

The rapid increase in telemedicine coupled with recent advances in diagnostic artificial intelligence (AI) create the imperative to consider the opportunities and risks of inserting AI-based support into new paradigms of care. Here we build on recent achievements in the accuracy of image-based AI for skin cancer diagnosis to address the effects of varied representations of AI-based support across different levels of clinical expertise and multiple clinical workflows. We find that good quality AI-based support of clinical decision-making improves diagnostic accuracy over that of either AI or physicians alone, and that the least experienced clinicians gain the most from AI-based support. We further find that AI-based multiclass probabilities outperformed content-based image retrieval (CBIR) representations of AI in the mobile technology environment, and AI-based support had utility in simulations of second opinions and of telemedicine triage. In addition to demonstrating the potential benefits associated with good quality AI in the hands of non-expert clinicians, we find that faulty AI can mislead the entire spectrum of clinicians, including experts. Lastly, we show that insights derived from AI class-activation maps can inform improvements in human diagnosis. Together, our approach and findings offer a framework for future studies across the spectrum of image-based diagnostics to improve human-computer collaboration in clinical practice

New Opportunities and Challenges for Health Professionals in the era of Artificial Intelligence – Review

Introduction and purpose: Modern medical knowledge has grown to a vastness incomprehensible for a single health professional to learn and accommodate. The usage of modern information technologies comes to help, one of them being artificial intelligence, a branch of computer science aimed at developing solutions to perform tasks similar to the human brain, but more efficient and complex, without actual human intervention.  The goal of this review is to provide reader with the knowledge how artificial intelligence is applied in various branches of medicine. Brief description of the state of knowledge: In the fields of infectious diseases, including COVID-19 diagnostics, radiology, dermatology and surgery, works lean toward the statement, which suspect application of AI is beneficial for medical practitioners. Programs help to develop statistical models for virus spreading and the creation of antiviral solutions. The radiological application involves the analysis of images to aid radiologists in diagnosing certain features, similarly to dermatology, where eg. AI can identify malignancy of skin nevi. In the department of surgery, predictive algorithms can help in choosing operation methods and improve outcomes. Conclusions: Usage of AI assistance in the medical field has proven to be successful, but it is yet to be commonly encountered in everyday work. Programs need to be further developed and made more approachable to users without expertise in the IT field. AI may also prove useful in the process of education of health professionals