Application of artificial vision algorithms to images of microscopy and spectroscopy for the improvement of cancer diagnosis

El diagnóstico final de la mayoría de tipos de cáncer lo realiza un médico experto en anatomía patológica que examina muestras tisulares o celulares sospechosas extraídas del paciente. Actualmente, esta evaluación depende en gran medida de la experiencia del médico y se lleva a cabo de forma cualitativa mediante técnicas de imagen tradicionales como la microscopía óptica. Esta tarea tediosa está sujeta a altos grados de subjetividad y da lugar a niveles de discordancia inadecuados entre diferentes patólogos, especialmente en las primeras etapas de desarrollo del cáncer. La espectroscopía infrarroja por Transformada de Fourier (siglas FTIR en inglés) es una tecnología ampliamente utilizada en la industria que recientemente ha demostrado una capacidad creciente para mejorar el diagnóstico de diferentes tipos de cáncer. Esta técnica aprovecha las propiedades del infrarrojo medio para excitar los modos vibratorios de los enlaces químicos que forman las muestras biológicas. La principal señal generada consiste en un espectro de absorción que informa sobre la composición química de la muestra iluminada. Los microespectrómetros FTIR modernos, compuestos por complejos componentes ópticos y detectores matriciales de alta sensibilidad, permiten capturar en un laboratorio de investigación común imágenes hiperespectrales de alta calidad que aúnan información química y espacial. Las imágenes FTIR son estructuras de datos ricas en información que se pueden analizar individualmente o junto con otras modalidades de imagen para realizar diagnósticos patológicos objetivos. Por lo tanto, esta técnica de imagen emergente alberga un alto potencial para mejorar la detección y la graduación del riesgo del paciente en el cribado y vigilancia de cáncer. Esta tesis estudia e implementa diferentes metodologías y algoritmos de los campos interrelacionados de procesamiento de imagen, visión por ordenador, aprendizaje automático, reconocimiento de patrones, análisis multivariante y quimiometría para el procesamiento y análisis de imágenes hiperespectrales FTIR. Estas imágenes se capturaron con un moderno microscopio FTIR de laboratorio a partir de muestras de tejidos y células afectadas por cáncer colorrectal y de piel, las cuales se prepararon siguiendo protocolos alineados con la práctica clínica actual. Los conceptos más relevantes de la espectroscopía FTIR se investigan profundamente, ya que deben ser comprendidos y tenidos en cuenta para llevar a cabo una correcta interpretación y tratamiento de sus señales especiales. En particular, se revisan y analizan diferentes factores fisicoquímicos que influyen en las mediciones espectroscópicas en el caso particular de muestras biológicas y pueden afectar críticamente su análisis posterior. Todos estos conceptos y estudios preliminares entran en juego en dos aplicaciones principales. La primera aplicación aborda el problema del registro o alineación de imágenes hiperespectrales FTIR con imágenes en color adquiridas con microscopios tradicionales. El objetivo es fusionar la información espacial de distintas muestras de tejido medidas con esas dos modalidades de imagen y centrar la discriminación en las regiones seleccionadas por los patólogos, las cuales se consideran más relevantes para el diagnóstico de cáncer colorrectal. En la segunda aplicación, la espectroscopía FTIR se lleva a sus límites de detección para el estudio de las entidades biomédicas más pequeñas. El objetivo es evaluar las capacidades de las señales FTIR para discriminar de manera fiable diferentes tipos de células de piel que contienen fenotipos malignos. Los estudios desarrollados contribuyen a la mejora de métodos de decisión objetivos que ayuden al patólogo en el diagnóstico final del cáncer. Además, revelan las limitaciones de los protocolos actuales y los problemas intrínsecos de la tecnología FTIR moderna, que deberían abordarse para permitThe final diagnosis of most types of cancers is performed by an expert clinician in anatomical pathology who examines suspicious tissue or cell samples extracted from the patient. Currently, this assessment largely relies on the experience of the clinician and is accomplished in a qualitative manner by means of traditional imaging techniques, such as optical microscopy. This tedious task is subject to high degrees of subjectivity and gives rise to suboptimal levels of discordance between different pathologists, especially in early stages of cancer development. Fourier Transform infrared (FTIR) spectroscopy is a technology widely used in industry that has recently shown an increasing capability to improve the diagnosis of different types of cancer. This technique takes advantage of the ability of mid-infrared light to excite the vibrational modes of the chemical bonds that form the biological samples. The main generated signal consists of an absorption spectrum that informs of the chemical composition of the illuminated specimen. Modern FTIR microspectrometers, composed of complex optical components and high-sensitive array detectors, allow the acquisition of high-quality hyperspectral images with spatially-resolved chemical information in a common research laboratory. FTIR images are information-rich data structures that can be analysed alone or together with other imaging modalities to provide objective pathological diagnoses. Hence, this emerging imaging technique presents a high potential to improve the detection and risk stratification in cancer screening and surveillance. This thesis studies and implements different methodologies and algorithms from the related fields of image processing, computer vision, machine learning, pattern recognition, multivariate analysis and chemometrics for the processing and analysis of FTIR hyperspectral images. Those images were acquired with a modern benchtop FTIR microspectrometer from tissue and cell samples affected by colorectal and skin cancer, which were prepared by following protocols close to the current clinical practise. The most relevant concepts of FTIR spectroscopy are thoroughly investigated, which ought to be understood and considered to perform a correct interpretation and treatment of its special signals. In particular, different physicochemical factors are reviewed and analysed, which influence the spectroscopic measurements for the particular case of biological samples and can critically affect their later analysis. All these knowledge and preliminary studies come into play in two main applications. The first application tackles the problem of registration or alignment of FTIR hyperspectral images with colour images acquired with traditional microscopes. The aim is to fuse the spatial information of distinct tissue samples measured by those two imaging modalities and focus the discrimination on regions selected by the pathologists, which are meant to be the most relevant areas for the diagnosis of colorectal cancer. In the second application, FTIR spectroscopy is pushed to their limits of detection for the study of the smallest biomedical entities. The aim is to assess the capabilities of FTIR signals to reliably discriminate different types of skin cells containing malignant phenotypes. The developed studies contribute to the improvement of objective decision methods to support the pathologist in the final diagnosis of cancer. In addition, they reveal the limitations of current protocols and intrinsic problems of modern FTIR technology, which should be tackled in order to enable its transference to anatomical pathology laboratories in the future.El diagnòstic final de la majoria de tipus de càncer ho realitza un metge expert en anatomia patològica que examina mostres tissulars o cel¿lulars sospitoses extretes del pacient. Actualment, aquesta avaluació depèn en gran part de l'experiència del metge i es porta a terme de forma qualitativa mitjançant tècniques d'imatge tradicionals com la microscòpia òptica. Aquesta tasca tediosa està subjecta a alts graus de subjectivitat i dóna lloc a nivells de discordança inadequats entre diferents patòlegs, especialment en les primeres etapes de desenvolupament del càncer. L'espectroscòpia infraroja per Transformada de Fourier (sigles FTIR en anglès) és una tecnologia àmpliament utilitzada en la indústria que recentment ha demostrat una capacitat creixent per millorar el diagnòstic de diferents tipus de càncer. Aquesta tècnica aprofita les propietats de l'infraroig mitjà per excitar els modes vibratoris dels enllaços químics que formen les mostres biològiques. El principal senyal generat consisteix en un espectre d'absorció que informa sobre la composició química de la mostra il¿luminada. Els microespectrómetres FTIR moderns, compostos per complexos components òptics i detectors matricials d'alta sensibilitat, permeten capturar en un laboratori d'investigació comú imatges hiperespectrals d'alta qualitat que uneixen informació química i espacial. Les imatges FTIR són estructures de dades riques en informació que es poden analitzar individualment o juntament amb altres modalitats d'imatge per a realitzar diagnòstics patològics objectius. Per tant, aquesta tècnica d'imatge emergent té un alt potencial per a millorar la detecció i la graduació del risc del pacient en el cribratge i vigilància de càncer. Aquesta tesi estudia i implementa diferents metodologies i algoritmes dels camps interrelacionats de processament d'imatge, visió per ordinador, aprenentatge automàtic, reconeixement de patrons, anàlisi multivariant i quimiometria per al processament i anàlisi d'imatges hiperespectrals FTIR. Aquestes imatges es van capturar amb un modern microscopi FTIR de laboratori a partir de mostres de teixits i cèl¿lules afectades per càncer colorectal i de pell, les quals es van preparar seguint protocols alineats amb la pràctica clínica actual. Els conceptes més rellevants de l'espectroscòpia FTIR s'investiguen profundament, ja que han de ser compresos i tinguts en compte per dur a terme una correcta interpretació i tractament dels seus senyals especials. En particular, es revisen i analitzen diferents factors fisicoquímics que influeixen en els mesuraments espectroscòpiques en el cas particular de mostres biològiques i poden afectar críticament la seua anàlisi posterior. Tots aquests conceptes i estudis preliminars entren en joc en dues aplicacions principals. La primera aplicació aborda el problema del registre o alineació d'imatges hiperespectrals FTIR amb imatges en color adquirides amb microscopis tradicionals. L'objectiu és fusionar la informació espacial de diferents mostres de teixit mesurades amb aquestes dues modalitats d'imatge i centrar la discriminació en les regions seleccionades pels patòlegs, les quals es consideren més rellevants per al diagnòstic de càncer colorectal. En la segona aplicació, l'espectroscòpia FTIR es porta als seus límits de detecció per a l'estudi de les entitats biomèdiques més xicotetes. L'objectiu és avaluar les capacitats dels senyals FTIR per discriminar de manera fiable diferents tipus de cèl¿lules de pell que contenen fenotips malignes. Els estudis desenvolupats contribueixen a la millora de mètodes de decisió objectius que ajuden el patòleg en el diagnòstic final del càncer. A més, revelen les limitacions dels protocols actuals i els problemes intrínsecs de la tecnologia FTIR moderna, que haurien d'abordar per permetre la seva transferència als laboratoris d'anatomia patològica en el futur.Peñaranda Gómez, FJ. (2018). Application of artificial vision algorithms to images of microscopy and spectroscopy for the improvement of cancer diagnosis [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/99748TESI

Biomedical Applications of Mid-Infrared Spectroscopic Imaging and Multivariate Data Analysis: Contribution to the Understanding of Diabetes Pathogenesis

Diabetic retinopathy (DR) is a microvascular complication of diabetes and a leading cause of adult vision loss. Although a great deal of progress has been made in ophthalmological examinations and clinical approaches to detect the signs of retinopathy in patients with diabetes, there still remain outstanding questions regarding the molecular and biochemical changes involved. To discover the biochemical mechanisms underlying the development and progression of changes in the retina as a result of diabetes, a more comprehensive understanding of the bio-molecular processes, in individual retinal cells subjected to hyperglycemia, is required. Animal models provide a suitable resource for temporal detection of the underlying pathophysiological and biochemical changes associated with DR, which is not fully attainable in human studies. In the present study, I aimed to determine the nature of diabetes-induced, highly localized biochemical changes in the retinal tissue from Ins2Akita/+ (Akita/+; a model of Type I diabetes) male mice with different duration of diabetes. Employing label-free, spatially resolved Fourier transform infrared (FT-IR) imaging engaged with chemometric tools enabled me to identify temporal-dependent reproducible biomarkers of the diabetic retinal tissue from mice with 6 or 12 weeks, and 6 or 10 months of diabetes. I report, for the first time, the origin of molecular changes in the biochemistry of individual retinal layers with different duration of diabetes. A robust classification between distinctive retinal layers - namely photoreceptor layer (PRL), outer plexiform layer (OPL), inner nuclear layer (INL), and inner plexiform layer (IPL) - and associated temporal-dependent spectral biomarkers, were delineated. Spatially-resolved super resolution chemical images revealed oxidative stress-induced structural and morphological alterations within the nucleus of the photoreceptors. Comparison among the PRL, OPL, INL, and IPL suggested that the photoreceptor layer is the most susceptible layer to the oxidative stress with short-duration of diabetes. Moreover, for the first time, we present the temporal-dependent molecular alterations for the PRL, OPL, INL, and IPL from Akita/+ mice, with progression of diabetes. These findings are potentially important and may be of particular benefit in understanding the molecular and biological activity of retinal cells during oxidative stress in diabetes. Our integrating paradigm provides a new conceptual framework and a significant rationale for a better understanding of the molecular and cellular mechanisms underlying the development and progression of DR. This approach may yield alternative and potentially complimentary methods for the assessment of diabetes changes. It is expected that the conclusions drawn from this work will bridge the gap in our knowledge regarding the biochemical mechanisms of the DR and address some critical needs in the biomedical community

Rapid high-resolution mid-IR imaging for molecular spectral histopathological diagnosis of oesophageal cancers

This thesis is written as part of Marie-Curie international training network called Mid-TECH. Mid-TECH is devoted to improve mid-infrared (MIR) technologies and consists of 15 PhD projects across European universities. This thesis aims to evaluate new technologies and concepts developed by the project partners for their applicability in a biomedical setting. The clinical problem to diagnose oesophageal cancers serves as an example case for this. The thesis consists of three projects all aimed to further the understanding of MIR hyperspectral imaging. The first project discussed in chapter 5 demonstrates the use of an new design of the United States Airforce resolution test chart. The new test chart is developed to evaluate spatial resolution of MIR hyperspectral imaging systems. The use of different materials is discussed and the new iteration of the thes chart is evaluated using a state of the art MIR imaging system. The second project discussed in chapter 6 evaluates the technical differences and their practical implications of discrete frequency MIR imaging systems compared to continuum source systems. A comparison of the two system types is drawn for imaging paraffin embedded sections of oesophageal tissue. Furthermore the effect of chemically removing the paraffin from the sample is compared to a mathematical correction algorithm. The system performance is compared based on their ability to differentiate healthy from cancerous tissue. The third project discussed in chapter 7 evaluates the potential of a new MIR detection scheme called upconversion in combination with a novel MIR laser source. It is a prove of concept study demonstrating that those two technologies can be deployed to do hyperspectral imaging in the MIR.European Commissio

Variabilities in global DNA methylation and β\beta-sheet richness establish spectroscopic landscapes among subtypes of pancreatic cancer

Purpose: Knowledge about pancreatic cancer (PC) biology has been growing rapidly in recent decades. Nevertheless, the survival of PC patients has not greatly improved. The development of a novel methodology suitable for deep investigation of the nature of PC tumors is of great importance. Molecular imaging techniques, such as Fourier transform infrared (FTIR) spectroscopy and Raman hyperspectral mapping (RHM) combined with advanced multivariate data analysis, were useful in studying the biochemical composition of PC tissue. Methods: Here, we evaluated the potential of molecular imaging in differentiating three groups of PC tumors, which originate from different precursor lesions. Specifically, we comprehensively investigated adenocarcinomas (ACs): conventional ductal AC, intraductal papillary mucinous carcinoma, and ampulla of Vater AC. FTIR microspectroscopy and RHM maps of 24 PC tissue slides were obtained, and comprehensive advanced statistical analyses, such as hierarchical clustering and nonnegative matrix factorization, were performed on a total of 211,355 Raman spectra. Additionally, we employed deep learning technology for the same task of PC subtyping to enable automation. The so-called convolutional neural network (CNN) was trained to recognize spectra specific to each PC group and then employed to generate CNN-prediction-based tissue maps. To identify the DNA methylation spectral markers, we used differently methylated, isolated DNA and compared the observed spectral differences with the results obtained from cellular nuclei regions of PC tissues. Results: The results showed significant differences among cancer tissues of the studied PC groups. The main findings are the varying content of β-sheet-rich proteins within the PC cells and alterations in the relative DNA methylation level. Our CNN model efficiently differentiated PC groups with 94% accuracy. The usage of CNN in the classification task did not require Raman spectral data preprocessing and eliminated the need for extensive knowledge of statistical methodologies. Conclusions: Molecular spectroscopy combined with CNN technology is a powerful tool for PC detection and subtyping. The molecular fingerprint of DNA methylation and β-sheet cytoplasmic proteins established by our results is different for the main PC groups and allowed the subtyping of pancreatic tumors, which can improve patient management and increase their survival. Our observations are of key importance in understanding the variability of PC and allow translation of the methodology into clinical practice by utilizing liquid biopsy testing

Histopathological image analysis : a review

Over the past decade, dramatic increases in computational power and improvement in image analysis algorithms have allowed the development of powerful computer-assisted analytical approaches to radiological data. With the recent advent of whole slide digital scanners, tissue histopathology slides can now be digitized and stored in digital image form. Consequently, digitized tissue histopathology has now become amenable to the application of computerized image analysis and machine learning techniques. Analogous to the role of computer-assisted diagnosis (CAD) algorithms in medical imaging to complement the opinion of a radiologist, CAD algorithms have begun to be developed for disease detection, diagnosis, and prognosis prediction to complement the opinion of the pathologist. In this paper, we review the recent state of the art CAD technology for digitized histopathology. This paper also briefly describes the development and application of novel image analysis technology for a few specific histopathology related problems being pursued in the United States and Europe

Laser Based Mid-Infrared Spectroscopic Imaging – Exploring a Novel Method for Application in Cancer Diagnosis

A number of biomedical studies have shown that mid-infrared spectroscopic images can provide both morphological and biochemical information that can be used for the diagnosis of cancer. Whilst this technique has shown great potential it has yet to be employed by the medical profession. By replacing the conventional broadband thermal source employed in modern FTIR spectrometers with high-brightness, broadly tuneable laser based sources (QCLs and OPGs) we aim to solve one of the main obstacles to the transfer of this technology to the medical arena; namely poor signal to noise ratios at high spatial resolutions and short image acquisition times. In this thesis we take the first steps towards developing the optimum experimental configuration, the data processing algorithms and the spectroscopic image contrast and enhancement methods needed to utilise these high intensity laser based sources. We show that a QCL system is better suited to providing numerical absorbance values (biochemical information) than an OPG system primarily due to the QCL pulse stability. We also discuss practical protocols for the application of spectroscopic imaging to cancer diagnosis and present our spectroscopic imaging results from our laser based spectroscopic imaging experiments of oesophageal cancer tissue

Application of Terahertz Technology in Biomolecular Analysis and Medical Diagnosis

Terahertz technology is a nondestructive technique, which has progressed significantly in the scientific research and gains highly attention in the analysis of biological molecular, cellular, tissues and organs. In this decade, some studies were reported on the application of terahertz technology in medical testing and diagnosis. Here, we summarize the terahertz characters, terahertz spectroscopy, and terahertz imaging technology combined with chemometrics. This chapter focuses on introducing the research progress on analyzing the tissues of cancers using terahertz spectroscopy and terahertz imaging technology. Furthermore, the problems should be solved, and development directions of terahertz spectroscopy and terahertz imaging technology are discussed

Novel chemometric approaches towards handling biospectroscopy datasets

Chemometrics allows one to identify chemical patterns using spectrochemical information of biological materials, such as tissues and biofluids. This has fundamental importance to overcome limitations in traditional bioanalytical analysis, such as the need for laborious and extreme invasive procedures, high consumption of reagents, and expensive instrumentation. In biospectroscopy, a beam of light, usually in the infrared region, is projected onto the surface of a biological sample and, as a result, a chemical signature is generated containing the vibrational information of most of the molecules in that material. This can be performed in a single-spectra or hyperspectral imaging fashion, where a resultant spectrum is generated for each position (pixel) in the surface of a biological material segment, hence, allowing extraction of both spatial and spectrochemical information simultaneously. As an advantage, these methodologies are non-destructive, have a relatively low-cost, and require minimum sample preparation. However, in biospectroscopy, large datasets containing complex spectrochemical signatures are generated. These datasets are processed by computational tools in order to solve their signal complexity and then provide useful information that can be used for decision taking, such as the identification of clustering patterns distinguishing disease from healthy controls samples; differentiation of tumour grades; prediction of unknown samples categories; or identification of key molecular fragments (biomarkers) associated with the appearance of certain diseases, such as cancer. In this PhD thesis, new computational tools are developed in order to improve the processing of bio-spectrochemical data, providing better clinical outcomes for both spectral and hyperspectral datasets

Spectral imaging in preclinical research and clinical pathology.

Spectral imaging methods are attracting increased interest from researchers and practitioners in basic science, pre-clinical and clinical arenas. A combination of better labeling reagents and better optics creates opportunities to detect and measure multiple parameters at the molecular and cellular level. These tools can provide valuable insights into the basic mechanisms of life, and yield diagnostic and prognostic information for clinical applications. There are many multispectral technologies available, each with its own advantages and limitations. This chapter will present an overview of the rationale for spectral imaging, and discuss the hardware, software and sample labeling strategies that can optimize its usefulness in clinical settings

Applications of advanced spectroscopic imaging to biological tissues

The objectives of this research were to develop experimental approaches that can be applied to classify different stages of malignancy in routine formalin-fixed and paraffin-embedded tissues and to optimise the imaging approaches using novel implementations. It is hoped that the approach developed in this research may be applied for early cancer diagnostics in clinical settings in the future in order to increase cancer survival rates. Infrared spectroscopic imaging has recently shown to have great potential as a powerful method for the spatial visualization of biological tissues. This spectroscopic technique does not require sample labelling because its chemical specificity allows the differentiation of biocomponents to be achieved based on their chemical structures. Experiments were performed on 3-µm thick prostate and colon tissues that were deposited on 2 mm-calcium fluoride (CaF2) which were subsequently deparaffinised. The samples were measured under IR microscopes, in both transmission and attenuated total reflection (ATR) mode. In transmission, thermo-spectroscopic imaging of the prostate samples was first carried out to investigate the potential of thermography to complement the information obtained from IR spectral. Spectroscopic imaging has made the acquisition of chemical map of a sample possible within a short time span since this approach facilitates the simultaneous acquisition of thousands of spatially resolved infrared spectra. Spectral differences in the lipid region (3000 -2800 cm-1) were identified between cancer and benign regions within prostate tissues. The governing spectral band for classification was anti-symmetric stretching of CH2 (2921 cm-1) from PCA analysis. Nonetheless, the difference in tissue emissivity at room temperature was minimal, thus the contrast in the thermal image is low for intra-tissue classification. Besides, the thermal camera could only capture IR light between 3333-2000 cm-1. To record spectral data between 3900 - 900 cm-1 (mid-IR), Fourier transform infrared (FTIR) spectroscopic imaging was used to classify the different stages of colon disease. An automated processing framework was developed, that could achieve an overall classification accuracy of 92.7%. The processing steps included unsupervised k-means clustering of lipid bands, followed by Random Forest (RF) classification using the ‘fingerprint’ region of the data. The implementation of a correcting lens and the effect of the RMieS-EMSC correction on the tissue spectra were also investigated, which showed that computational RMieS-EMSC correction was more effective at removing spectral artefacts than the correcting lens. Furthermore, the effect of the fluctuations of surrounding humidity where the experiments were carried out was studied by using various supersaturated salt solutions. Significant peak changes of the phosphate band were observed, most notably the peak shift of the anti-symmetric stretching of phosphate bands from 1230 cm-1 to 1238 cm-1 was observed. By regulating and controlling humidity at its lowest, the classification accuracy of the colon specimens was improved without having to resort to alteration on the RF machine learning algorithm. In the ATR mode, additional apertures were introduced to the FTIR microscope, as a novel means of depth profiling the prostate tissue samples by changing the angle of incidence of IR light beam. Despite the successful attempts in capturing the qualitative information on the change of tissue morphology with the depth of penetration (dp), the spectral data were not suitable for further processing with machine learning as dp changes with wavelengths. Apart from the apertures, a ‘large-area’ germanium (Ge) crystal was introduced to enable simultaneous mapping and imaging of the colon tissue samples. Many advantages of this new implementation were observed, which included improvement in signal-to-noise ratio, uniform distribution, and no impression left on the sample. The research done in this thesis set a groundwork for clinical diagnosis and the novel implementations were transferable to studies of other samples.Open Acces