Deep learning applied to the classification of skin lesions

Mestrado de dupla diplomação com a UTFPR - Universidade Tecnológica Federal do ParanáSkin cancer has been a global health issue and its diagnosis is a challenge in the medical field. Among all the types of skin cancer, melanoma is the worst and can be lethal if not early treated. The use of deep learning techniques, specifically, convolutional neural networks can help to improve the accuracy and speed up the classification of skin lesions. In this work, we aim to employ different image preprocessing techniques, various convolutional neural network models, data augmentation, and ensemble techniques to compare their results and provide an analysis of the data obtained. To achieve that, it was performed several experiments combining different image preprocessing techniques, which, paired with data augmentation strategies, aim to enhance the accuracy and reliability of the classification models. Additionally, three ensemble methods were tested to improve the classification systems’ robustness and reliability by gathering the strengths of each model. Our best result was the ensemble of EfficientNet-B2, EfficientNet-B5, and ResNeSt101 models with the application of data augmentation, and the combination of color constancy and hair removal techniques. This combined approach achieved a balanced accuracy of0.8132. By offering insights into the challenges faced, methodologies employed, and results obtained, this story aims to serve as a guide for researchers and practitioners aiming to advance the field of skin lesion classification using deep learning. Keywords: Deep Learning; Skin Lesion Classification; Image preprocessing.O câncer de pele é um problema de saúde global e seu diagnóstico é um desafio na área médica. Entre todos os tipos de câncer de pele, o melanoma é o pior e pode ser letal se não tratado precocemente. O uso de técnicas de deep learning, especificamente, redes neurais convolucionais, pode ajudar a melhorar a precisão e acelerar a classificação de lesões de pele. Neste trabalho, buscamos empregar diferentes técnicas de pré-processamento de imagens, vários modelos de redes neurais convolucionais, data augmentation e técnicas de ensemble para comparar seus resultados e fornecer uma análise dos dados obtidos. Para isso, foram realizados vários experimentos combinando diferentes técnicas de préprocessamento de imagens, que, combinadas com estratégias de data augmentation, visam melhorar a precisão e confiabilidade dos modelos de classificação. Além disso, três métodos de ensemble foram testados para melhorar a robustez e confiabilidade dos sistemas de classificação, reunindo os pontos fortes de cada modelo. Nosso melhor resultado foi o ensemble dos modelos EfficientNet-B2, EfficientNet-B5 e ResNeSt101 com a aplicação de data augmentation e a combinação de técnicas de color constancy e remoção de pelos. Esta abordagem alcançou uma balanced accuracy de 0,8132. Ao oferecer insights sobre as metodologias empregadas e resultados obtidos, este estudo visa servir como um guia para pesquisadores e profissionais que buscam avançar no campo da classificação de lesões cutâneas usando aprendizado profundo

An Efficient Block-Based Algorithm for Hair Removal in Dermoscopic Images

Hair occlusion in dermoscopy images affects the diagnostic operation of the skin lesion. Segmentation and classification of skin lesions are two major steps of the diagnostic operation required by Dermatologists. We propose a new algorithm for hair removal in dermoscopy images that includes two main stages: hair detection and inpainting. In hair detection, a morphological bottom-hat operation is implemented on Y-channel image of YIQ color space followed by a binarization operation. In inpainting, the repaired Y-channel is partitioned into 256 nonoverlapped blocks and for each block, white pixels are replaced by locating the highest peak of using a histogram function and a morphological close operation. Our proposed algorithm reports a true positive rate (sensitivity) of 97.36%, a false positive rate (fall-out) of 4.25%, and a true negative rate (specificity) of 95.75%. The diagnostic accuracy achieved is recorded at a high level of 95.78%

An efficient block-based algorithm for hair removal in dermoscopic images

Hair occlusion in dermoscopy images affects the diagnostic operation of the skin lesion. Segmentation and classification of skin lesions are two major steps of the diagnostic operation required by dermatologists. We propose a new algorithm for hair removal in dermoscopy images that includes two main stages: hair detection and inpainting. In hair detection, a morphological bottom-hat operation is implemented on Y-channel image of YIQ color space followed by a binarization operation. In inpainting, the repaired Y-channel is partitioned into 256 non-overlapped blocks and for each block, white pixels are replaced by locating the highest peak, using a histogram function and a morphological close operation. The proposed algorithm reports a true positive rate (sensitivity) of 97.36 %, a false positive rate (fall-out) of 4.25 %, and a true negative rate (specificity) of 95.75 %. The diagnostic accuracy achieved is recorded at a high level of 95.78 %

Ex vivo dermis microdialysis: A tool for bioequivalence testing of topical dermatological drug product (Demonstration of proof of concept and testing)

Clinical response to most topical dermatological drug products (TDDP) depends on the availability of the drug in the dermis. Dermal Microdialysis (dMD) is a sampling technique that permits measuring the concentration of a drug over time, in vivo, directly into the target tissue, the dermis. The pharmacokinetic parameters obtained from such studies may help to optimize the development of TDDP and potentially can be applied to the assessment of TDDP bioequivalence. However, these studies require several hours or even days of continuous sampling that makes it often stressful and unpractical for human subjects as well as animals. The goal of this dissertation was to develop a reliable and consistent ex-vivo dMD method to complement and assist in vivo dMD experiments. In the first part of the project, we have developed and tested the ex-vivo dermal microdialysis method on two different experimental skin models using freshly excised porcine skin. Porcine skin was selected due to the close resemblance to human skin, it is advantageous in terms of availability and expense. For the microdialysis study, in-house dermal microdialysis probes were conveniently manufactured with controlled specifications and the microdialysis recovery process was screened with an in vitro setup to match the intended use. The in vitro microdialysis method was optimized for probe specification, analyte suitability, perfusion flow rate, and perfusate composition. A maximized, rapid, and steady recovery was demonstrated within a wide range of concentrations. For the ex vivo dermal microdialysis study, the two different skin models developed were: M1-- Full-thickness skin (≈0.25 cm) without subcutaneous fat layer placed on a hydrated 0.5 cm cellulose backing support, and M2 -- Full-thickness skin with subcutaneous fat layer (total thickness = 1.0 cm) placed directly on an aluminum boat, avoiding any kind of hydration. Both setups were tested on TDDP cream and gel of metronidazole (MTZ) for which both in vivo and IVPT data are available for comparison. The two different formulations, Metronidazole cream and gel, were compared side-by-side for the rate and extent of delivery to the dermis. The latter skin model was found suitable, manifesting data comparable to the available data from in vivo pig and IVPT (human cadaver) study. The selection of the best-fit-model was based on the comparative bioavailability response from the negative control, Metronidazole gel, resulting in a lower bioavailability profile (90% CI). Using this suitable ex vivo dMD model, site-specific results of the drug can be conveniently monitored in the dermis leading to dose-dependent rate and extent in concentration-time exposure. The M2 model was further tested for the effect of temperature of the skin on the bioavailability profile of the drug. As reported in several pieces of literature, an increase in dermal exposure is expected with the rise in skin temperature. Superficial addition of heat to the skin was not feasible as it may change the thermodynamics of the formulation leading to alteration in permeation kinetics. Thus, the physiological temperature of ex vivo pig skin explant was achieved by providing continuous heat from the ventral side using a closed water-bath system. The process of supplementation of temperature did not impact the bioavailability profile, rather unfavorable damages to the skin microstructure due to thermal degradation was observed. Further studies with the proposed model ex vivo dMD model were conducted at ambient lab temperature. Yet another aspect of the proposed model was to test its capability to determine bioequivalence (BE). The potential of using this model for BE testing was validated by comparing the BA of MetroCream with its USFDA-approved generic Metronidazole 0.75% cream. The overall BE estimation resulted in an ln-AUC of 91.65 (80.93, 104.88) and an ln-Cmax value of 87.56 (74.87,102.39). The fact that reference and test formulations can be tested simultaneously at multiple sites on a skin sample harvested from a single animal subject reduces the burden of vii-inter-subject variability. The experimental population size required to establish bioequivalence for topically applied drugs can be reduced. Yet another aspect of the study was to design a mathematical model, based on the ex vivo dMD findings, to extend its predictability to in vivo outcomes. A first-of-its-kind unit impulse response method was applied in dermis tissue of skin explant to measure the absorption independent elimination parameters. The estimated parameters were employed to calculate the cumulative absorption of the drug from different topical formulations. The absorption profile of the developed model was time-scaled and absolute-scaled with a permeation scaling factor to map with available literature data on in vivo pig. The levy point-to-point regression coefficient was employed to predict the in vivo PK profile thereafter. With the current intervention, we propose a mathematical possibility to predict in vivo outcomes for a given topical dose. The studies presented were limited internal predictions only, and external validation with a different set of data is yet to be performed and to be undertaken at another time. Overall, the studies presented in this work provides a foundation stone for an elaborate field of work that can be undertaken to minimize the use of animal in pharmacokinetic evaluations for topical and transdermal products. Ex vivo dermal microdialysis warrants testing on a plethora of drug molecules of different polarities to decide on the future of the technique. Regardless, the technique holds unmet potential and needs to be nurtured over time

Analysis of the ISIC image datasets: Usage, benchmarks and recommendations

The International Skin Imaging Collaboration (ISIC) datasets have become a leading repository for researchers in machine learning for medical image analysis, especially in the field of skin cancer detection and malignancy assessment. They contain tens of thousands of dermoscopic photographs together with gold-standard lesion diagnosis metadata. The associated yearly challenges have resulted in major contributions to the field, with papers reporting measures well in excess of human experts. Skin cancers can be divided into two major groups - melanoma and non-melanoma. Although less prevalent, melanoma is considered to be more serious as it can quickly spread to other organs if not treated at an early stage. In this paper, we summarise the usage of the ISIC dataset images and present an analysis of yearly releases over a period of 2016 - 2020. Our analysis found a significant number of duplicate images, both within and between the datasets. Additionally, we also noted duplicates spread across testing and training sets. Due to these irregularities, we propose a duplicate removal strategy and recommend a curated dataset for researchers to use when working on ISIC datasets. Given that ISIC 2020 focused on melanoma classification, we conduct experiments to provide benchmark results on the ISIC 2020 test set, with additional analysis on the smaller ISIC 2017 test set. Testing was completed following the application of our duplicate removal strategy and an additional data balancing step. As a result of removing 14,310 duplicate images from the training set, our benchmark results show good levels of melanoma prediction with an AUC of 0.80 for the best performing model. As our aim was not to maximise network performance, we did not include additional steps in our experiments. Finally, we provide recommendations for future research by highlighting irregularities that may present research challenges. A list of image files with reference to the original ISIC dataset sources for the recommended curated training set will be shared on our GitHub repository (available at www.github.com/mmu-dermatology-research/isic_duplicate_removal_strategy)

Beauty surveillance: the digital self-monitoring cultures of neoliberalism

This paper argues that ‘beauty apps’ are transforming the arena of appearance politics and foregrounds a theoretical architecture for critically understanding them. Informed by a feminist-Foucaultian framework, it argues that beauty apps offer a technology of gender which brings together digital self-monitoring and postfeminist modalities of subjecthood to produce an hitherto unprecedented regulatory gaze upon women that is marked by the intensification, extensification and psychologization of surveillance. The paper is divided into four sections. First it introduces the literature on digital self-tracking. Secondly it sets out our understanding of neoliberalism and postfeminism. Thirdly it looks at beauty and surveillance, before offering, in the final section, a typology of appearance apps. This is followed by a discussion of the modes of address/authority deployed in these apps – especially what we call ‘surveillant sisterhood’ - and the kinds of entrepreneurial subjectivity they constitute. The paper seeks to make a contribution to feminist surveillance studies and argues that much more detailed research is needed to critically examine beauty apps

From scans to models: Registration of 3D human shapes exploiting texture information

New scanning technologies are increasing the importance of 3D mesh data, and of algorithms that can reliably register meshes obtained from multiple scans. Surface registration is important e.g. for building full 3D models from partial scans, identifying and tracking objects in a 3D scene, creating statistical shape models. Human body registration is particularly important for many applications, ranging from biomedicine and robotics to the production of movies and video games; but obtaining accurate and reliable registrations is challenging, given the articulated, non-rigidly deformable structure of the human body. In this thesis, we tackle the problem of 3D human body registration. We start by analyzing the current state of the art, and find that: a) most registration techniques rely only on geometric information, which is ambiguous on flat surface areas; b) there is a lack of adequate datasets and benchmarks in the field. We address both issues. Our contribution is threefold. First, we present a model-based registration technique for human meshes that combines geometry and surface texture information to provide highly accurate mesh-to-mesh correspondences. Our approach estimates scene lighting and surface albedo, and uses the albedo to construct a high-resolution textured 3D body model that is brought into registration with multi-camera image data using a robust matching term. Second, by leveraging our technique, we present FAUST (Fine Alignment Using Scan Texture), a novel dataset collecting 300 high-resolution scans of 10 people in a wide range of poses. FAUST is the first dataset providing both real scans and automatically computed, reliable ground-truth correspondences between them. Third, we explore possible uses of our approach in dermatology. By combining our registration technique with a melanocytic lesion segmentation algorithm, we propose a system that automatically detects new or evolving lesions over almost the entire body surface, thus helping dermatologists identify potential melanomas. We conclude this thesis investigating the benefits of using texture information to establish frame-to-frame correspondences in dynamic monocular sequences captured with consumer depth cameras. We outline a novel approach to reconstruct realistic body shape and appearance models from dynamic human performances, and show preliminary results on challenging sequences captured with a Kinect

Abnormal Brain Connectivity in the Primary Visual Pathway in Human Albinism

In albinism, the ipsilateral projection of retinal axons is significantly reduced, and most fibres project contralaterally. The retina and optic chiasm have been proposed as sites for misrouting. The number of lateral geniculate nucleus (LGN) relay neurons has been linked to LGN volume, suggesting a correlation between LGN size and the number of tracts traveling through the optic radiation (OR) to the primary visual cortex (V1). Using diffusion data and both deterministic and probabilistic tractography, we studied differences in OR between albinism and controls. Statistical analyses measured white matter integrity in areas corresponding to the OR, as well as LGN to V1 connectivity. Results revealed reduced white matter integrity and connectivity in the OR region in albinism compared to controls, suggesting altered structural development. Previous reports of smaller LGN and the altered thalamo-cortical connectivity reported here demonstrate the effect of misrouting on structural organization of the visual pathway in albinism

On the Recognition of Emotion from Physiological Data

This work encompasses several objectives, but is primarily concerned with an experiment where 33 participants were shown 32 slides in order to create ‗weakly induced emotions‘. Recordings of the participants‘ physiological state were taken as well as a self report of their emotional state. We then used an assortment of classifiers to predict emotional state from the recorded physiological signals, a process known as Physiological Pattern Recognition (PPR). We investigated techniques for recording, processing and extracting features from six different physiological signals: Electrocardiogram (ECG), Blood Volume Pulse (BVP), Galvanic Skin Response (GSR), Electromyography (EMG), for the corrugator muscle, skin temperature for the finger and respiratory rate. Improvements to the state of PPR emotion detection were made by allowing for 9 different weakly induced emotional states to be detected at nearly 65% accuracy. This is an improvement in the number of states readily detectable. The work presents many investigations into numerical feature extraction from physiological signals and has a chapter dedicated to collating and trialing facial electromyography techniques. There is also a hardware device we created to collect participant self reported emotional states which showed several improvements to experimental procedure

Deep human face analysis and modelling

Human face appearance and motion play a significant role in creating the complex social environments of human civilisation. Humans possess the capacity to perform facial analysis and come to conclusion such as the identity of individuals, understanding emotional state and diagnosing diseases. The capacity though is not universal for the entire population, where there are medical conditions such prosopagnosia and autism which can directly affect facial analysis capabilities of individuals, while other facial analysis tasks require specific traits and training to perform well. This has lead to the research of facial analysis systems within the computer vision and machine learning fields over the previous decades, where the aim is to automate many facial analysis tasks to a level similar or surpassing humans. While breakthroughs have been made in certain tasks with the emergence of deep learning methods in the recent years, new state-of-the-art results have been achieved in many computer vision and machine learning tasks. Within this thesis an investigation into the use of deep learning based methods for facial analysis systems takes place, following a review of the literature specific facial analysis tasks, methods and challenges are found which form the basis for the research findings presented. The research presented within this thesis focuses on the tasks of face detection and facial symmetry analysis specifically for the medical condition facial palsy. Firstly an initial approach to face detection and symmetry analysis is proposed using a unified multi-task Faster R-CNN framework, this method presents good accuracy on the test data sets for both tasks but also demonstrates limitations from which the remaining chapters take their inspiration. Next the Integrated Deep Model is proposed for the tasks of face detection and landmark localisation, with specific focus on false positive face detection reduction which is crucial for accurate facial feature extraction in the medical applications studied within this thesis. Evaluation of the method on the Face Detection Dataset and Benchmark and Annotated Faces in-the-Wild benchmark data sets shows a significant increase of over 50% in precision against other state-of-the-art face detection methods, while retaining a high level of recall. The task of facial symmetry and facial palsy grading are the focus of the finals chapters where both geometry-based symmetry features and 3D CNNs are applied. It is found through evaluation that both methods have validity in the grading of facial palsy. The 3D CNNs are the most accurate with an F1 score of 0.88. 3D CNNs are also capable of recognising mouth motion for both those with and without facial palsy with an F1 score of 0.82