On the spectral signature of melanoma: a non-parametric classification framework for cancer detection in hyperspectral imaging of melanocytic lesions

Early detection and diagnosis is a must in secondary prevention of melanoma and other cancerous lesions of the skin. In this work, we present an online, reservoir-based, non-parametric estimation and classification model that allows for this functionality on pigmented lesions, such that detection thresholding can be tuned to maximize accuracy and/or minimize overall false negative rates. This system has been tested in a dataset consisting of 116 patients and a total of 124 hyperspectral images of nevi, raised nevi and melanomas, detecting up to 100% of the suspicious lesions at the expense of some false positives.MINECO (Ministerio de Economía y Competitividad), Instituto de Salud Carlos III (ISCIII) (DTS15/00238, DTS17/00055, TEC2016-76021-C2-2-R); CIBER-BBN; IDIVAL (INNVAL 16/02); MECD (Ministerio de Educación, Cultura y Deporte) (FPU16/05705)

Comparison of preprocessing techniques to reduce nontissue-related variations in hyperspectral reflectance imaging

Significance: Hyperspectral reflectance imaging can be used in medicine to identify tissue types, such as tumor tissue. Tissue classification algorithms are developed based on, e.g., machine learning or principle component analysis. For the development of these algorithms, data are generally preprocessed to remove variability in data not related to the tissue itself since this will improve the performance of the classification algorithm. In hyperspectral imaging, the measured spectra are also influenced by reflections from the surface (glare) and height variations within and between tissue samples.Aim: To compare the ability of different preprocessing algorithms to decrease variations in spectra induced by glare and height differences while maintaining contrast based on differences in optical properties between tissue types.Approach: We compare eight preprocessing algorithms commonly used in medical hyperspectral imaging: standard normal variate, multiplicative scatter correction, min-max normalization, mean centering, area under the curve normalization, single wavelength normalization, first derivative, and second derivative. We investigate conservation of contrast stemming from differences in: blood volume fraction, presence of different absorbers, scatter amplitude, and scatter slope-while correcting for glare and height variations. We use a similarity metric, the overlap coefficient, to quantify contrast between spectra. We also investigate the algorithms for clinical datasets from the colon and breast.Conclusions: Preprocessing reduces the overlap due to glare and distance variations. In general, the algorithms standard normal variate, min-max, area under the curve, and single wavelength normalization are the most suitable to preprocess data used to develop a classification algorithm for tissue classification. The type of contrast between tissue types determines which of these four algorithms is most suitable

Automatic Recognition of Colon and Esophagogastric Cancer with Machine Learning and Hyperspectral Imaging

There are approximately 1.8 million diagnoses of colorectal cancer, 1 million diagnoses of stomach cancer, and 0.6 million diagnoses of esophageal cancer each year globally. An automatic computer-assisted diagnostic (CAD) tool to rapidly detect colorectal and esophagogastric cancer tissue in optical images would be hugely valuable to a surgeon during an intervention. Based on a colon dataset with 12 patients and an esophagogastric dataset of 10 patients, several state-of-the-art machine learning methods have been trained to detect cancer tissue using hyperspectral imaging (HSI), including Support Vector Machines (SVM) with radial basis function kernels, Multi-Layer Perceptrons (MLP) and 3D Convolutional Neural Networks (3DCNN). A leave-one-patient-out cross-validation (LOPOCV) with and without combining these sets was performed. The ROC-AUC score of the 3DCNN was slightly higher than the MLP and SVM with a difference of 0.04 AUC. The best performance was achieved with the 3DCNN for colon cancer and esophagogastric cancer detection with a high ROC-AUC of 0.93. The 3DCNN also achieved the best DICE scores of 0.49 and 0.41 on the colon and esophagogastric datasets, respectively. These scores were significantly improved using a patient-specific decision threshold to 0.58 and 0.51, respectively. This indicates that, in practical use, an HSI-based CAD system using an interactive decision threshold is likely to be valuable. Experiments were also performed to measure the benefits of combining the colorectal and esophagogastric datasets (22 patients), and this yielded significantly better results with the MLP and SVM models

Segmentation in dermatological hyperspectral images: dedicated methods

Background: Segmentation of hyperspectral medical images is one of many image segmentation methods which require profiling. This profiling involves either the adjustment of existing, known image segmentation methods or a proposal of new dedicated methods of hyperspectral image segmentation. Taking into consideration the size of analysed data, the time of analysis is of major importance. Therefore, the authors proposed three new dedicated methods of hyperspectral image segmentation with special reference to the time of analysis. Methods: The segmentation methods presented in this paper were tested and profiled to the images acquired from different hyperspectral cameras including SOC710 Hyperspectral Imaging System, Specim sCMOS-50-V10E. Correct functioning of the method was tested for over 10,000 2D images constituting the sequence of over 700 registrations of the areas of the left and right hand and the forearm. Results: As a result, three new methods of hyperspectral image segmentation have been proposed: fast analysis of emissivity curves (SKE), 3D segmentation (S3D) and hierarchical segmentation (SH). They have the following features: are fully automatic; allow for implementation of fast segmentation methods; are profiled to hyperspectral image segmentation; use emissivity curves in the model form, can be applied in any type of objects not necessarily biological ones, are faster (SKE-2.3 ms, S3D-1949 ms, SH-844 ms for the computer with Intel® Core i7 4960X CPU 3.6 GHz) and more accurate (SKE-accuracy 79 %, S3D-90 %, SH-92 %) in comparison with typical methods known from the literature. Conclusions: Profiling and/or proposing new methods of hyperspectral image segmentation is an indispensable element of developing software. This ensures speed, repeatability and low sensitivity of the algorithm to changing parameters

Development of a new spectral imaging system for the diagnosis of skin cancer

The incidence of skin cancer in Europe, US and Australia has been rising rapidly. Skin cancer accounts for one in three cancers worldwide and a person has 1:25 chance to develop a melanoma, the most aggressive form. Visual inspection followed by histological examination is, still today, the gold standard for clinicians, which is carried out through a dermoscope, a handheld device with a magnifying lens and a white and uniform illumination field. The dermoscopic technique requires considerable training in the interpretation of what is seen and is highly dependent on subjective impressions. In consequence, a large number of unnecessary surgical procedures are performed. For this reason, in this thesis a spectral imaging system to improve skin cancer diagnosis has been developed. This work has been carried out in the framework of the European project DIAGNOPTICS "Diagnosis of skin cancer using optics", which aimed to launch a hospital service based on a multiphotonic platform to improve skin cancer with the combination of four non-invasive novel techniques: 3D and multispectral imaging, optical feedback interferometry and confocal microscopy. The handheld system built included a monochromatic CCD camera attached to an objective lens and a light source containing 32 light emitting diodes (LEDs) with 8 spectral bands from 400 nm to 1000 nm. An acquisition software to control all the components of the multispectral system was programmed as well as a simplest version for physicians. The changes over time of the emission of the LEDs was analysed, and also the linear response of the camera at each wavelength, the uniformity of the LED emission and the short and long-term repeatability of the system in acquiring images, to ensure the good performance of the system. In order to proceed with the Ethical Committee approval and to launch the systems in both hospitals, irradiance and radiance measurements were done according to the standard UNE-EN 62471. A Graphical User Interface (GUI) was developed for the spectral images processing and corresponding analysis, allowing spectral and colorimetric features to be computed in terms of reflectance, absorbance and colour parameters. Furthermore, a segmentation algorithm was also implemented to extract the isolated information from the lesion. For all images calculated in terms of any of the parameters, conventional statistical descriptors were obtained. As a first approach to extracting textural information we also used the analysis of the statistical properties of the histogram. An inclusion criteria and a measurement protocol were established. From all lesions analysed, 620 were measured with the multispectral system, 572 of them had a clinical or histopathological diagnosis, and 502 could be properly segmented. Therefore, 429 skin lesions were finally included in the study: 290 nevi, 95 melanomas and 44 basal cell carcinomas. A classification algorithm was developed in order to decide whether the lesions were malignant (melanomas and basal cell carcinomas) or not (nevi), splitting previously the data into training and validations set of the same size. 15 parameters from 1309 were found to be not redundant providing a 91.3% of sensitivity and 54.5% of specificity. Accordingly, the addition of textural information was shown to be useful for the diagnosis of malignant lesions than the sole use of averaged spectral and colour information. The same steps were carried out for the 3D imaging system also included in the multiphotonic platform. In this case, 3 parameters were found to be useful for the classification providing values of 55.6% and 83.7% of sensitivity and specificity, respectively. Finally, the combination of both system was also tested as a first attempt to improve the detection of melanomas, providing 100% and 72.2% of sensitivity and specificity, respectively. However, the conclusions reached in this case should be taken with caution due to the limited number of lesions.La incidència del càncer de pell a Europa, Estats Units i Austràlia ha anat augmentant ràpidament. Representa un de cada tres càncers a tot el món i una persona té 1:25 oportunitats de desenvolupar un melanoma, la forma més agressiva. Actualment, la inspecció visual amb un dermoscopi seguida d'un examen histològic és l'estàndard utilitzat pels metges a l'hora de diagnosticar-lo. La dermoscòpia requereix una formació considerable per interpretar el que es veu i depèn de les impressions subjectives dels clínics. En conseqüència, es realitzen una gran quantitat de procediments quirúrgics innecessaris. Per aquest motiu, en aquesta tesi s'ha desenvolupat una sistema d'imatge espectral per millorar el diagnòstic del càncer de pell. Aquest treball s'ha realitzat dins el marc del projecte Europeu DIAGNOPTICS ¿Diagnosis del càncer de pell utilitzant òptica?, el qual ha posat a punt un servei hospitalari basat en un plataforma multifotònica que combina quatre tècniques òptiques innovadores: sistemes d'imatge multiespectral 3D, interferometria de retroalimentació i microscòpia confocal. El sistema portàtil desenvolupat inclou una càmera monocromàtica CCD, un objectiu i una font de llum formada per 32 díodes emissors de llum (LED) amb 8 bandes espectrals diferents que emeten des de 400 nm fins a 1000 nm. S'ha preparat un programa d'adquisició per controlar tots els components del sistema així com una versió més simple del mateix pels metges. Per tal d'assegurar el bon funcionament del sistema, es van analitzar els canvis temporals en l'emissió dels LEDs així com la seva uniformitat d'emissió, la resposta lineal de la càmera per cada longitud d'ona i la repetibilitat del sistema pel què fa a l'adquisició d'imatges. Per tal d'obtenir l'aprovació del Comitè Ètic i poder realitzar l'estudi clínic en els hospitals, es van dur a terme mesures d'irradiància i radiància d'acord amb la norma UNE-EN 62471. També es va implementar una interfície gràfica d'usuari (GUI) per al processament de les imatges espectrals i la seva corresponent anàlisi. Aquest algorisme permet calcular paràmetres espectrals i colorimètrics de la pell en termes de reflectància, absorbància i d'altres basats en el color. A més, inclús es va desenvolupar un algorisme de segmentació per extreure informació aïllada de cada lesió. Per a totes les imatges calculades en termes de qualsevol paràmetre, es van obtenir descriptors estadístics convencionals i també es van utilitzar propietats estadístiques dels histogrames com una primera aproximació d'extreure informació de textura de la lesió. Finalment, es van establir els criteris d'inclusió i un protocol de mesura. De totes les lesions analitzades, se'n van mesurar 620, de les quals 572 tenien un diagnòstic clínic o histopatològic; 502 es van poder segmentar adequadament. D'aquesta manera es van incloure 429 lesions cutànies a l'estudi: 290 nevus, 95 melanomes i 44 carcinomes de cèl·lules basals. Es va desenvolupar un algorisme de classificació per determinar si les lesions eren malignes (melanomes i carcinomes de cèl·lules basals) o no (nevus), dividint prèviament les dades en un grup d'entrenament i un altre de validació de la mateixa mida. Es va observar que 15 del 1309 paràmetres proporcionaven informació rellevant per a la classificació, obtenint una sensibilitat del 91,3% i una especificitat del 54,5%. Així doncs, es demostra que la incorporació d'informació de textura és molt útil per al diagnòstic del càncer de pell més enllà de la informació espectral i de color. Aquests mateixos passos es van seguir pel sistema 3D també inclòs en la plataforma multifotònica, tot i que en aquest cas el número de lesions de què es disposava era més limitat. En aquest cas, es van seleccionar 3 paràmetres i es va obtenir una sensibilitat del 55,6% i una especificitat del 83,7%. Finalment, amb la combinació d'ambdós sistemes la sensibilitat obtinguda va ser de100% i l'especificitat del 72,2%.Postprint (published version

Development of a new spectral imaging system for the diagnosis of skin cancer

The incidence of skin cancer in Europe, US and Australia has been rising rapidly. Skin cancer accounts for one in three cancers worldwide and a person has 1:25 chance to develop a melanoma, the most aggressive form. Visual inspection followed by histological examination is, still today, the gold standard for clinicians, which is carried out through a dermoscope, a handheld device with a magnifying lens and a white and uniform illumination field. The dermoscopic technique requires considerable training in the interpretation of what is seen and is highly dependent on subjective impressions. In consequence, a large number of unnecessary surgical procedures are performed. For this reason, in this thesis a spectral imaging system to improve skin cancer diagnosis has been developed. This work has been carried out in the framework of the European project DIAGNOPTICS "Diagnosis of skin cancer using optics", which aimed to launch a hospital service based on a multiphotonic platform to improve skin cancer with the combination of four non-invasive novel techniques: 3D and multispectral imaging, optical feedback interferometry and confocal microscopy. The handheld system built included a monochromatic CCD camera attached to an objective lens and a light source containing 32 light emitting diodes (LEDs) with 8 spectral bands from 400 nm to 1000 nm. An acquisition software to control all the components of the multispectral system was programmed as well as a simplest version for physicians. The changes over time of the emission of the LEDs was analysed, and also the linear response of the camera at each wavelength, the uniformity of the LED emission and the short and long-term repeatability of the system in acquiring images, to ensure the good performance of the system. In order to proceed with the Ethical Committee approval and to launch the systems in both hospitals, irradiance and radiance measurements were done according to the standard UNE-EN 62471. A Graphical User Interface (GUI) was developed for the spectral images processing and corresponding analysis, allowing spectral and colorimetric features to be computed in terms of reflectance, absorbance and colour parameters. Furthermore, a segmentation algorithm was also implemented to extract the isolated information from the lesion. For all images calculated in terms of any of the parameters, conventional statistical descriptors were obtained. As a first approach to extracting textural information we also used the analysis of the statistical properties of the histogram. An inclusion criteria and a measurement protocol were established. From all lesions analysed, 620 were measured with the multispectral system, 572 of them had a clinical or histopathological diagnosis, and 502 could be properly segmented. Therefore, 429 skin lesions were finally included in the study: 290 nevi, 95 melanomas and 44 basal cell carcinomas. A classification algorithm was developed in order to decide whether the lesions were malignant (melanomas and basal cell carcinomas) or not (nevi), splitting previously the data into training and validations set of the same size. 15 parameters from 1309 were found to be not redundant providing a 91.3% of sensitivity and 54.5% of specificity. Accordingly, the addition of textural information was shown to be useful for the diagnosis of malignant lesions than the sole use of averaged spectral and colour information. The same steps were carried out for the 3D imaging system also included in the multiphotonic platform. In this case, 3 parameters were found to be useful for the classification providing values of 55.6% and 83.7% of sensitivity and specificity, respectively. Finally, the combination of both system was also tested as a first attempt to improve the detection of melanomas, providing 100% and 72.2% of sensitivity and specificity, respectively. However, the conclusions reached in this case should be taken with caution due to the limited number of lesions.La incidència del càncer de pell a Europa, Estats Units i Austràlia ha anat augmentant ràpidament. Representa un de cada tres càncers a tot el món i una persona té 1:25 oportunitats de desenvolupar un melanoma, la forma més agressiva. Actualment, la inspecció visual amb un dermoscopi seguida d'un examen histològic és l'estàndard utilitzat pels metges a l'hora de diagnosticar-lo. La dermoscòpia requereix una formació considerable per interpretar el que es veu i depèn de les impressions subjectives dels clínics. En conseqüència, es realitzen una gran quantitat de procediments quirúrgics innecessaris. Per aquest motiu, en aquesta tesi s'ha desenvolupat una sistema d'imatge espectral per millorar el diagnòstic del càncer de pell. Aquest treball s'ha realitzat dins el marc del projecte Europeu DIAGNOPTICS ¿Diagnosis del càncer de pell utilitzant òptica?, el qual ha posat a punt un servei hospitalari basat en un plataforma multifotònica que combina quatre tècniques òptiques innovadores: sistemes d'imatge multiespectral 3D, interferometria de retroalimentació i microscòpia confocal. El sistema portàtil desenvolupat inclou una càmera monocromàtica CCD, un objectiu i una font de llum formada per 32 díodes emissors de llum (LED) amb 8 bandes espectrals diferents que emeten des de 400 nm fins a 1000 nm. S'ha preparat un programa d'adquisició per controlar tots els components del sistema així com una versió més simple del mateix pels metges. Per tal d'assegurar el bon funcionament del sistema, es van analitzar els canvis temporals en l'emissió dels LEDs així com la seva uniformitat d'emissió, la resposta lineal de la càmera per cada longitud d'ona i la repetibilitat del sistema pel què fa a l'adquisició d'imatges. Per tal d'obtenir l'aprovació del Comitè Ètic i poder realitzar l'estudi clínic en els hospitals, es van dur a terme mesures d'irradiància i radiància d'acord amb la norma UNE-EN 62471. També es va implementar una interfície gràfica d'usuari (GUI) per al processament de les imatges espectrals i la seva corresponent anàlisi. Aquest algorisme permet calcular paràmetres espectrals i colorimètrics de la pell en termes de reflectància, absorbància i d'altres basats en el color. A més, inclús es va desenvolupar un algorisme de segmentació per extreure informació aïllada de cada lesió. Per a totes les imatges calculades en termes de qualsevol paràmetre, es van obtenir descriptors estadístics convencionals i també es van utilitzar propietats estadístiques dels histogrames com una primera aproximació d'extreure informació de textura de la lesió. Finalment, es van establir els criteris d'inclusió i un protocol de mesura. De totes les lesions analitzades, se'n van mesurar 620, de les quals 572 tenien un diagnòstic clínic o histopatològic; 502 es van poder segmentar adequadament. D'aquesta manera es van incloure 429 lesions cutànies a l'estudi: 290 nevus, 95 melanomes i 44 carcinomes de cèl·lules basals. Es va desenvolupar un algorisme de classificació per determinar si les lesions eren malignes (melanomes i carcinomes de cèl·lules basals) o no (nevus), dividint prèviament les dades en un grup d'entrenament i un altre de validació de la mateixa mida. Es va observar que 15 del 1309 paràmetres proporcionaven informació rellevant per a la classificació, obtenint una sensibilitat del 91,3% i una especificitat del 54,5%. Així doncs, es demostra que la incorporació d'informació de textura és molt útil per al diagnòstic del càncer de pell més enllà de la informació espectral i de color. Aquests mateixos passos es van seguir pel sistema 3D també inclòs en la plataforma multifotònica, tot i que en aquest cas el número de lesions de què es disposava era més limitat. En aquest cas, es van seleccionar 3 paràmetres i es va obtenir una sensibilitat del 55,6% i una especificitat del 83,7%. Finalment, amb la combinació d'ambdós sistemes la sensibilitat obtinguda va ser de100% i l'especificitat del 72,2%

Hyperspectral imaging and robust statistics in non-melanoma skin cancer analysis

Non-Melanoma skin cancer is one of the most frequent types of cancer. Early detection is encouraged so as to ensure the best treatment, Hyperspectral imaging is a promising technique for non-invasive inspection of skin lesions, however, the optimal wavelengths for these purposes are yet to be conclusively determined. A visible-near infrared hyperspectral camera with an ad-hoc built platform was used for image acquisition in the present study. Robust statistical techniques were used to conclude an optimal range between 573.45 and 779.88 nm to distinguish between healthy and non-healthy skin. Wavelengths between 429.16 and 520.17 nm were additionally found to be optimal for the differentiation between cancer types.Gerencia Regional de Salud de Castilla y León (GRS 2139/A/20); Spanish Ministry of Science, Innovation and Universities (PRE2019-089411); Instituto de Salud Carlos III (PI18/00587); Ibderdrola Spain; Junta de Castilla y León (GRS 1837/A/18). This project was funded by the Junta de Castilla y Leon, under the title project HYPERSKINCARE (Ref. GRS 1837/A/18). Lloyd Austin Courtenay is funded by the Spanish Ministry of Science, Innovation and Universities with an FPI Predoctoral Grant (Ref. PRE2019-089411) associated to project RTI2018-099850-B-I00 and the University of Salamanca. Susana Lagüela and Susana del Pozo are both funded by the Iberdrola Spain through the initiative Cátedra Iberdrola VIII Centenario of the University of Salamanca. Javier Cañueto is partially supported by the PI18/00587(Instituto de Salud Carlos III cofinanciado con fondos FEDER) and GRS 2139/A/20 (Gerencia Regional de Salud de Castilla y León

Information Extraction Techniques in Hyperspectral Imaging Biomedical Applications

Hyperspectral imaging (HSI) is a technology able to measure information about the spectral reflectance or transmission of light from the surface. The spectral data, usually within the ultraviolet and infrared regions of the electromagnetic spectrum, provide information about the interaction between light and different materials within the image. This fact enables the identification of different materials based on such spectral information. In recent years, this technology is being actively explored for clinical applications. One of the most relevant challenges in medical HSI is the information extraction, where image processing methods are used to extract useful information for disease detection and diagnosis. In this chapter, we provide an overview of the information extraction techniques for HSI. First, we introduce the background of HSI, and the main motivations of its usage for medical applications. Second, we present information extraction techniques based on both light propagation models within tissue and machine learning approaches. Then, we survey the usage of such information extraction techniques in HSI biomedical research applications. Finally, we discuss the main advantages and disadvantages of the most commonly used image processing approaches and the current challenges in HSI information extraction techniques in clinical applications