Multimodal endoscopic system based on multispectral and photometric stereo imaging and analysis

We propose a multimodal endoscopic system based on white light (WL), multispectral (MS), and photometric stereo (PS) imaging for the examination of colorectal cancer (CRC). Recently, the enhancement of the diagnostic accuracy of CRC colonoscopy has been reported; however, tumor diagnosis for a variety of lesion types remains challenging using current endoscopy. In this study, we demonstrate that our developed system can simultaneously discriminate tumor distributions and provide three-dimensional (3D) morphological information about the colon surface using the WL, MS, and PS imaging modalities. The results demonstrate that the proposed system has considerable potential for CRC diagnosis. © 2019, OSA - The Optical Society. All rights reserved.1

Multispectral image analysis in laparoscopy – A machine learning approach to live perfusion monitoring

Modern visceral surgery is often performed through small incisions. Compared to open surgery, these minimally invasive interventions result in smaller scars, fewer complications and a quicker recovery. While to the patients benefit, it has the drawback of limiting the physician’s perception largely to that of visual feedback through a camera mounted on a rod lens: the laparoscope. Conventional laparoscopes are limited by “imitating” the human eye. Multispectral cameras remove this arbitrary restriction of recording only red, green and blue colors. Instead, they capture many specific bands of light. Although these could help characterize important indications such as ischemia and early stage adenoma, the lack of powerful digital image processing prevents realizing the technique’s full potential. The primary objective of this thesis was to pioneer fluent functional multispectral imaging (MSI) in laparoscopy. The main technical obstacles were: (1) The lack of image analysis concepts that provide both high accuracy and speed. (2) Multispectral image recording is slow, typically ranging from seconds to minutes. (3) Obtaining a quantitative ground truth for the measurements is hard or even impossible. To overcome these hurdles and enable functional laparoscopy, for the first time in this field physical models are combined with powerful machine learning techniques. The physical model is employed to create highly accurate simulations, which in turn teach the algorithm to rapidly relate multispectral pixels to underlying functional changes. To reduce the domain shift introduced by learning from simulations, a novel transfer learning approach automatically adapts generic simulations to match almost arbitrary recordings of visceral tissue. In combination with the only available video-rate capable multispectral sensor, the method pioneers fluent perfusion monitoring with MSI. This system was carefully tested in a multistage process, involving in silico quantitative evaluations, tissue phantoms and a porcine study. Clinical applicability was ensured through in-patient recordings in the context of partial nephrectomy; in these, the novel system characterized ischemia live during the intervention. Verified against a fluorescence reference, the results indicate that fluent, non-invasive ischemia detection and monitoring is now possible. In conclusion, this thesis presents the first multispectral laparoscope capable of videorate functional analysis. The system was successfully evaluated in in-patient trials, and future work should be directed towards evaluation of the system in a larger study. Due to the broad applicability and the large potential clinical benefit of the presented functional estimation approach, I am confident the descendants of this system are an integral part of the next generation OR

Experimental and Model-based Terahertz Imaging and Spectroscopy for Mice, Human, and Phantom Breast Cancer Tissues

The goal of this work is to investigate terahertz technology for assessing the surgical margins of breast tumors through electromagnetic modeling and terahertz experiments. The measurements were conducted using a pulsed terahertz system that provides time and frequency domain signals. Three types of breast tissues were investigated in this work. The first was formalin-fixed, paraffin-embedded tissues from human infiltrating ductal and lobular carcinomas. The second was human tumors excised within 24-hours of lumpectomy or mastectomy surgeries. The third was xenograft and transgenic mice breast cancer tumors grown in a controlled laboratory environment to achieve more data for statistical analysis. Experimental pulsed terahertz imaging first used thin sections (10-30 μm thick) of fixed breast cancer tissue on slides. Electromagnetic inverse scattering models, in transmission and reflection modes, were developed to retrieve the tissue refractive index and absorption coefficient. Terahertz spectroscopy was utilized to experimentally collect data from breast tissues for these models. The results demonstrated that transmission mode is suitable for lossless materials while the reflection model is more suitable for biological materials where the skin depth of terahertz waves does not exceed 100 µm. The reflection model was implemented to estimate the polarization of the incident terahertz signal of the system, which was shown to be a hybridization of TE and TM modes. Terahertz imaging of three-dimensional human breast cancer blocks of tissue embedded in paraffin was achieved through the reflection model. The terahertz beam can be focused at depths inside the block to produce images in the x-y planes (z-scan). The time-of-flight analysis was applied to terahertz signals reflected at each depth demonstrating the margins of cancerous regions inside the block as validated with pathology images at each depth. In addition, phantom tissues that mimic freshly excised infiltrating ductal carcinoma human tumors were developed with and without embedded carbon nanometer-scale onion-like carbon particles. These particles exhibited a strong terahertz signal interaction with tissue demonstrating a potential to greatly improve the image contrast. The results presented in this work showed, in most cases, a significant differentiation in terahertz images between cancer and healthy tissue as validated with histopathology images

Surgical spectral imaging

Recent technological developments have resulted in the availability of miniaturised spectral imaging sensors capable of operating in the multi- (MSI) and hyperspectral imaging (HSI) regimes. Simultaneous advances in image-processing techniques and artificial intelligence (AI), especially in machine learning and deep learning, have made these data-rich modalities highly attractive as a means of extracting biological information non-destructively. Surgery in particular is poised to benefit from this, as spectrally-resolved tissue optical properties can offer enhanced contrast as well as diagnostic and guidance information during interventions. This is particularly relevant for procedures where inherent contrast is low under standard white light visualisation. This review summarises recent work in surgical spectral imaging (SSI) techniques, taken from Pubmed, Google Scholar and arXiv searches spanning the period 2013–2019. New hardware, optimised for use in both open and minimally-invasive surgery (MIS), is described, and recent commercial activity is summarised. Computational approaches to extract spectral information from conventional colour images are reviewed, as tip-mounted cameras become more commonplace in MIS. Model-based and machine learning methods of data analysis are discussed in addition to simulation, phantom and clinical validation experiments. A wide variety of surgical pilot studies are reported but it is apparent that further work is needed to quantify the clinical value of MSI/HSI. The current trend toward data-driven analysis emphasises the importance of widely-available, standardised spectral imaging datasets, which will aid understanding of variability across organs and patients, and drive clinical translation

POTENTIAL OF TERAHERTZ PULSED REFLECTOMETRY AND IMAGING FOR THE EARLY DIAGNOSIS OF CUTANEOUS MELANOMA

In the last two decades the incidence rate of cutaneous malignant melanoma have been risen faster than any other form of cancer worldwide in the white-Caucasian population. The mortality rates over time show that an early diagnosis is the key point for quick treatment, which increase survival rates. As a standard procedure dermatologists use a dermascope or the naked eye for evaluation of possible lesions, where experts have a higher chance of spotting infiltrated tissue than untrained persons. Multiple investigations on diagnostic imaging for the detection of melanoma have been conducted in the past, like Ultrasound, Near-Infrared spectroscopy or Optical Coherence Tomography, with mixed, but not sufficient results to date. In recent years terahertz radiation has shown to be a promising technology for the early detection of various types of cancers, i.e., colon ex-vivo}, breast ex-vivo and non-melanoma skin cancers ex-vivo andmin-vivo as terahertz radiation is able to penetrate slightly into the bio-tissue but also deemed to be a non-ionising and therefore safe method for diagnosis of lesions in-vivo. Investigations into the practicality and benefits of using terahertz reflectometry for the early diagnosis of melanoma has never been performed. Therefore, as a pilot study, an investigation into the modalities of utilising terahertz technology on freshly excised human cutaneous melanoma is anticipated, which includes a comparison of the collected 3D terahertz images with visuals, comparison of histopathologists findings but also investigations about modelling skin and abnormalities of the skin using terahertz radiation. Diverse and manifold results can be reported based on the study conducted, which show that there is a good potential of terahertz detecting abnormalities on a per patient basis of up to 78% sensitivity and 95% specificity respectively. However, skin is a very diverse medium and results of the modelling approach have to be seen very critically. As modality for a diagnostic tool, this investigation suggests that there is potential in detecting margins and active regions of cancerous region spreading, which may help to support the dermatologists to determine better margins for the excision of the lesion.This work has been sponsored by the Hope Againast Cancer Foundation, Leiceste

Prediction of Proapoptotic Anticancer Therapeutic Response Based on Visualization of Death Ligand-Receptor Interaction and Specific Marker of Cellular Proliferation

Emerging targeted therapeutics hold great promise for the treatment of human cancer. However there are still challenges for selecting patients that most likely will benefit from targeted drugs. One of the major limitations of classical imaging methods is the significant delay to provide quantifiable and objective evidence of response to cancer therapy. Molecular imaging may be useful in targeted drug development by assessing the target expression and drug-target interaction, and predicting therapeutic response in both preclinical and clinical settings. The apoptosis pathway triggered by the Tumor Necrosis Factor (TNF)-Related Apoptosis-Inducing Ligand (TRAIL) receptors is a potential target for therapeutic intervention. TRAIL and its proapoptotic receptor agonistic monoclonal antibodies are being developed as targeted therapeutics in the treatment of human cancer. It is our hypothesis that visualization of proapoptotic receptors and binding of their agonists to proapoptotic receptors can noninvasively predict proapoptotic response if the pathway is intact. Hence the objective of this work is to develop efficient multimodality molecular imaging methods to predict proapoptotic anticancer therapy response before or at the very early stage of treatment. Towards this goal, we have labeled proapoptotic receptor agonists (PARAs) with near-infrared (NIR) fluorescent dyes to image PARAs binding to their targets expressed on the cell surface in cultured cells and in human tumor xenografts grown subcutaneously in immunodeficient mice. Both in vitro and in vivo studies demonstrated that imaging PARAs binding to their targets was well correlated with proapoptotic anticancer therapeutic response when TRAIL signaling pathway was intact. To pursue a more general molecular imaging marker that can predict anticancer therapeutic response even when the signaling pathway is impaired, we explored a novel radiotracer for positron emission tomography (PET) imaging [(18)F]-3\u27-fluoro-3\u27-deoxy-L-thymidine ([(18)F]-FLT), an analogue of thymidine and a specific marker of DNA replication and cellular proliferation. Our results suggested that early changes in [(18)F]-PET may not only predict the tumor histological response to anticancer therapeutics but also determine superiority of one treatment regimen over another. In summary our proof-of-concept studies show that multimodality molecular imaging will greatly aid in accelerating anticancer drug approval process and improving survival and response rates in hard-to-treat cancer

Improving Clinical Diagnosis of Melanocytic Skin Lesions by Raman Spectroscopy

High-quality Raman signals from melanocytic lesions compatible with a possible clinical application have not been demonstrated yet. The objectives of the work described in this thesis were: I: The development of a Raman spectroscopic prototype for objective and fast assessment of melanocytic skin lesions clinically suspicious for melanoma; II: Identification of the main spectroscopic features of melanoma and benign melanocytic lesions suspicious for melanoma; III: Assessment of the feasibility of Raman spectroscopy as an adjunct technique to improve clinical diagnosis of melanocytic skin lesions

Optical spectroscopy and imaging systems for gynecological cancers: from Ultraviolet-C (UVC) to the Mid-infrared

Optical spectroscopy and imaging has proving to be of diagnostic relevance in many organ sites. We use fluorescence and FTIR spectroscopy to study gynecological organ sites and develop classification algorithms for cancer diagnosis. Ovarian cancer is the deadliest gynecological cancer. The American Cancer Society reports that for the year 20 I 0, there would be 21,880 new cases of ovarian cancer and 13,850 fatalities. This is partly due to the fact that current diagnostic and screening methods for the disease are not very accurate. In this study, we analyze the fluorescence spectra of excised normal and cancerous ovarian tissues at multiple excitation wavelengths. The data includes spectra obtained at the UVC wavelength 270nm and UVB wavelength 300nm. Excitation in the UVC has been especially understudied in spectroscopy for tissue diagnosis. We introduce the application of a novel SVM algorithm for the classification of fluorescence data. This SVM is trained subject to the Neyman Pearson (NP) criterion which allows for a decision rule that maximizes the detection specificity whilst constraining the sensitivity to a high value. This technique allows us to develop a binary classification algorithm that is not biased towards the larger group and this in tum leads to robust classifiers that are more suitable for clinical applications. We obtained sensitivities and specificities greater than 90% for ovarian cancer diagnosis using this algorithm. Also, FTIR is used to analyze cervical tissues. Absorption of light in the mid-IR region by biomolecules show up as peaks in the FTIR spectra, and there is differential absorption in tissue depending on the histopathology. The spectroscopic analysis informed our choosing of a wavelength for the illumination source ofa mid-IR microscope. We then present the design of an imaging system that employs the use ofa mid-IR quantum cascade laser(QCL) which can potentially have clinical use in the future. Finally a reflectance based fiber endoscope imaging system is presented. Cellular imaging is demonstrated with this system that has the potential for use in optical biopsy

Assessment and Diagnosis of Human Colorectal and Ovarian Cancer using Optical Imaging and Computer-aided Diagnosis

Tissue optical scattering has recently emerged as an important diagnosis parameter associated with early tumor development and progression. To characterize the differences between benign and malignant colorectal tissues, we have created an automated optical scattering coefficient mapping algorithm using an optical coherence tomography (OCT) system. A novel feature called the angular spectrum index quantifies the scattering coefficient distribution. In addition to scattering, subsurface morphological changes are also associated with the development of colorectal cancer. We have observed a specific mucosa structure indicating normal human colorectal tissue, and have developed a real-time pattern recognition neural network to localize this specific structure in OCT images, enabling classification of the morphological changes associated with the progression of human colon cancer. Differentiating normal from malignant tissues is critically important, however, identifying different subtypes of abnormalities is also useful in clinical diagnosis. We have designed a feature extraction method using texture features and computer-vision related features to characterize different types of colorectal tissues. We first ranked these features according to their importance, then trained two classifiers: one for normal vs. abnormal, and the other one for cancer vs. polyp, where polyp is a pre-cancer marker. In assessing tissue abnormalities, optical absorption reveals contrast related to tumor microvasculature and tumor angiogenesis. Spatial frequency domain imaging (SFDI), a powerful wide field, label-free imaging modality, is sensitive to both absorption and scattering. We designed a computer-aided diagnostic algorithm, AdaBoost, to use multispectral SFDI imaging for ex vivo assessment of different types of colorectal tissues, including normal and cancerous tissue and adenomatous polyps. For diagnosis of human ovarian cancer, we first designed a histogram-based feature extraction algorithm. Then we trained and tested traditional machine learning methods utilizing these histogram features for ovarian cancer diagnosis. We also explored the use of these features in characterizing human fallopian tubes, which are believed to be the origin of the most lethal subtype of human ovarian cancers