An Investigation of the Diagnostic Potential of Autofluorescence Lifetime Spectroscopy and Imaging for Label-Free Contrast of Disease

The work presented in this thesis aimed to study the application of fluorescence lifetime spectroscopy (FLS) and fluorescence lifetime imaging microscopy (FLIM) to investigate their potential for diagnostic contrast of diseased tissue with a particular emphasis on autofluorescence (AF) measurements of gastrointestinal (GI) disease. Initially, an ex vivo study utilising confocal FLIM was undertaken with 420 nm excitation to characterise the fluorescence lifetime (FL) images obtained from 71 GI samples from 35 patients. A significant decrease in FL was observed between normal colon and polyps (p = 0.024), and normal colon and inflammatory bowel disease (IBD) (p = 0.015). Confocal FLIM was also performed on 23 bladder samples. A longer, although not significant, FL for cancer was observed, in paired specimens (n = 5) instilled with a photosensitizer. The first in vivo study was a clinical investigation of skin cancer using a fibre-optic FL spectrofluorometer and involved the interrogation of 27 lesions from 25 patients. A significant decrease in the FL of basal cell carcinomas compared to healthy tissue was observed (p = 0.002) with 445 nm excitation. A novel clinically viable FLS fibre-optic probe was then applied ex vivo to measure 60 samples collected from 23 patients. In a paired analysis of neoplastic polyps and normal colon obtained from the same region of the colon in the same patient (n = 12), a significant decrease in FL was observed (p = 0.021) with 435 nm excitation. In contrast, with 375 nm excitation, the mean FL of IBD specimens (n = 4) was found to be longer than that of normal tissue, although not statistically significant. Finally, the FLS system was applied in vivo in 17 patients, with initial data indicating that 435 nm excitation results in AF lifetimes that are broadly consistent with ex vivo studies, although no diagnostically significant differences were observed in the signals obtained in vivo.Open Acces

Visual search for musical performances and endoscopic videos

[ANGLÈS] This project explores the potential of LIRE, an en existing Content-Based Image Retrieval (CBIR) system, when used to retrieve medical videos. These videos are recording of the live streams used by surgeons during the endoscopic procedures, captured from inside of the subject. The growth of such video content stored in servers requires search engines capable to assist surgeons in their management and retrieval. In our tool, queries are formulated by visual examples and those allow surgeons to re-find shots taken during the procedure. This thesis presents an extension and adaptation of Lire for video retrieval based on visual features and late fusion. The results are assessed from two perspectives: a quantitative and qualitative one. While the quantitative one follows the standard practices and metrics for video retrieval, the qualitative assessment has been based on an empirical social study using a semi-interactive web-interface. In particular, a thinking aloud test was applied to analyze if the user expectations and requirements were fulfilled. Due to the scarcity of surgeons available for the qualitative tests, a second domain was also addressed: videos captured at musical performances. These type of videos has also experienced an exponential growth with the advent of affordable multimedia smart phones, available to a large audience. Analogously to the endoscopic videos, searching in a large data set of such videos is a challenging topic.[CASTELLÀ] Este proyecto investiga el potencial de Lire, un sistema existente de recuperación basado en contenido de imagen (CBIR) utilizado en el dominio médico. Estos vídeos son grabaciones a tiempo real del interior de los pacientes y son utilizados por cirujanos durante las operaciones de endoscopia. La creciente demanda de este conjunto de vídeos que son almacenados en diferentes servidores, requiere nuevos motores de búsqueda capaces de dar soporte al trabajo de los médicos con su gestión y posterior recuperación cuando se necesite. En nuestra herramienta, las consultas son formuladas mediante ejemplos visuales. Esto permite a los cirujanos volver a encontrar los diferentes instantes capturados durante las intervenciones. En esta tesis se presenta una extensión y adaptación de Lire para la recuperación de vídeo basado en las características visuales y métodos de late fusion. Los resultados son evaluados desde dos perspectivas: una cuantitativa y una cualitativa. Mientras que la parte cuantitativa sigue el estándar de las prácticas y métricas empleadas en vídeo retrieval, la evaluación cualitativa ha sido basada en un estudio social empírico mediante una interfaz web semi-interactiva. Particularmente, se ha emprendido el método "thinking aloud test" para analizar si nuestra herramienta cumple con las expectativas y necesidades de los usuarios a la hora de utilizar la aplicación. Debido a la escasez de médicos disponibles para llevar a cabo las pruebas cualitativas, el trabajo se ha dirigido también a un segundo dominio: conjunto de vídeos de acontecimientos musicales. Este tipo de vídeos también ha experimentado un crecimiento exponencial con la llegada de los smart phones y se encuentran al alcance de un público muy amplio. Análogamente a los vídeos endoscópicos, hacer una busca en una gran base de datos de este tipo también es un tema difícil y motivo de estudio.[CATALÀ] Aquest projecte investiga el potencial de Lire, un sistema existent de recuperació basat en contingut d'imatge (CBIR) utilitzat en el domini mèdic. Aquests vídeos són enregistraments a temps real de l'interior dels pacients i són utilitzats per cirurgians durant les operacions d'endoscòpia. La creixent demanda d'aquest conjunt de vídeos que són emmagatzemats a diferents servidors, requereix nous motors de cerca capaços de donar suport a la feina dels metges amb la seva gestió i posterior recuperació quan es necessiti. A la nostra eina, les consultes són formulades mitjançant exemples visuals. Això permet als cirurgians tornar a trobar els diferents instants capturats durant la intervenció. En aquesta tesi es presenta una extensió i adaptació del Lire per a la recuperació de vídeo basat en característiques visuals i late fusion. Els resultats són avaluats des de dues perspectives: una quantitativa i una qualitativa. Mentre que la part quantitativa segueix l'estàndard de les pràctiques i mètriques per vídeo retrieval, l'avaluació qualitativa ha estat basada en un estudi social empíric mitjançant una interfície web semiinteractiva. Particularment, s'ha emprès el mètode "thinking aloud test" per analitzar si la nostra eina compleix amb les expectatives i necessitats dels usuaris a l'hora d'utilitzar l'aplicació. A causa de l'escassetat de metges disponibles per dur a terme les proves qualitatives, el treball s'ha adreçat també a un segon domini: conjunt de vídeos d'esdeveniments musicals. Aquest tipus de vídeos també ha experimentat un creixement exponencial amb l'arribada dels smart phones i es troben a l'abast d'un públic molt ampli. Anàlogament als vídeos endoscòpics, fer una cerca en una gran base de dades d'aquest tipus també és un tema difícil i motiu d'estudi

The assessment of visual behaviour and depth perception in surgery

Imperial Users onl

Roadmap on signal processing for next generation measurement systems

Signal processing is a fundamental component of almost any sensor-enabled system, with a wide range of applications across different scientific disciplines. Time series data, images, and video sequences comprise representative forms of signals that can be enhanced and analysed for information extraction and quantification. The recent advances in artificial intelligence and machine learning are shifting the research attention towards intelligent, data-driven, signal processing. This roadmap presents a critical overview of the state-of-the-art methods and applications aiming to highlight future challenges and research opportunities towards next generation measurement systems. It covers a broad spectrum of topics ranging from basic to industrial research, organized in concise thematic sections that reflect the trends and the impacts of current and future developments per research field. Furthermore, it offers guidance to researchers and funding agencies in identifying new prospects.AerodynamicsMicrowave Sensing, Signals & System

Multimodal Multispectral Optical Endoscopic Imaging for Biomedical Applications

Optical imaging is an emerging field of clinical diagnostics that can address the growing medical need for early cancer detection and diagnosis. Various human cancers are amenable to better prognosis and patient survival if found and treated during early disease onset. Besides providing wide-field, macroscopic diagnostic information similar to existing clinical imaging techniques, optical imaging modalities have the added advantage of microscopic, high resolution cellular-level imaging from in vivo tissues in real time. This comprehensive imaging approach to cancer detection and the possibility of performing an ‘optical biopsy’ without tissue removal has led to growing interest in the field with numerous techniques under investigation. Three optical techniques are discussed in this thesis, namely multispectral fluorescence imaging (MFI), hyperspectral reflectance imaging (HRI) and fluorescence confocal endomicroscopy (FCE). MFI and HRI are novel endoscopic imaging-based extensions of single point detection techniques, such as laser induced fluorescence spectroscopy and diffuse reflectance spectroscopy. This results in the acquisition of spectral data in an intuitive imaging format that allows for quantitative evaluation of tissue disease states. We demonstrate MFI and HRI on fluorophores, tissue phantoms and ex vivo tissues and present the results as an RGB colour image for more intuitive assessment. This follows dimensionality reduction of the acquired spectral data with a fixed-reference isomap diagnostic algorithm to extract only the most meaningful data parameters. FCE is a probe-based point imaging technique offering confocal detection in vivo with almost histology-grade images. We perform FCE imaging on chemotherapy-treated in vitro human ovarian cancer cells, ex vivo human cancer tissues and photosensitiser-treated in vivo murine tumours to show the enhanced detection capabilities of the technique. Finally, the three modalities are applied in combination to demonstrate an optical viewfinder approach as a possible minimally-invasive imaging method for early cancer detection and diagnosis

Developing endoscopic instrumentation and techniques for in vivo fluorescence lifetime imaging and spectroscopy

Confocal fluorescence endomicroscopes employ fibre optics along with miniaturised scanning and focussing mechanisms to allow microscopic investigation of remote tissue samples with sub-cellular resolution. For this reason they are widely used in biomedical research, both in clinical studies and in small animal imaging experiments. Fluorescence lifetime imaging microscopy (FLIM) has been shown to provide contrast between normal and unhealthy tissue in several diseases including gastro-intestinal (GI) cancer. As such, there is significant interest in developing instrumentation that will allow endoscopic confocal FLIM as this would permit the in vivo investigation of human GI tissue. This thesis describes the development and use of several instruments and techniques aimed at clinically viable in vivo fluorescence lifetime spectroscopy and confocal endomicroscopy. This research has consisted of two broad branches: the study of the fluorescence signature of healthy and diseased tissue both ex vivo and in vivo; and the development of a novel method for achieving beam scanning in confocal endomicroscopy. Firstly the tissue studies are discussed. This begins with the application of a compact steady-state diffuse reflectance/fluorescence spectrometer and a fibre-optic-coupled time-resolved spectrofluorometer to an in vivo investigation of the spectral signatures of skin cancer. This study – which involved the interrogation of 27 clinically diagnosed lesions – was carried out in collaboration with researchers at Lund University in Sweden and revealed significant differences between healthy and diseased tissue both in terms of fluorescence lifetime and steady state reflectance and fluorescence spectra. Further to this study, work is presented charting the development of a clinically viable spectrometer, which measures time-resolved fluorescence spectra with two excitation wavelengths (375 nm and 435 nm) as well as diffuse reflectance spectra. The entire system is contained within a compact trolley (120 x 70 x 55 cm) for easy transportation and safe use in a clinic. It utilises a fibre optic probe to deliver/collect light that can be inserted into the working channel of a medical endoscope meaning that the system can be used to measure diffuse reflectance and time-resolved fluorescence spectra in the GI tract in vivo. The development and testing of this system are discussed and data are presented from both ex vivo and in vivo studies of GI cancer. The second broad section of this thesis focuses more closely on confocal endomicroscopy. Firstly current methods used in this field are discussed and the sources of several drawbacks are explained. A novel approach to laser scanning endomicroscopy is then presented, which requires no moving parts and can be implemented without the need for any distal scanners or optics. This technique is similar in concept to the use of adaptive optics to focus through turbid media: it utilises a proximal spatial light modulator to correct for phase variations across a fibre imaging bundle and then to encode for arbitrary wavefronts at the distal end of that fibre bundle. Thus, it is possible to realise both focussing and beam scanning at the output of the fibre bundle with no distal components, permitting extremely compact endoscopic probes to be developed. Proof-of-principle results are presented illustrating the imaging capabilities of this novel system as well as simulations showing the achievable resolution and field of view in several feasible endoscopic configurations. Overall, this thesis contains work from two quite different projects both aimed at developing novel optical techniques for clinical diagnostic use in endoscopic procedures. The first is aimed at investigating the temporal and spectral properties of the fluorescence and reflectance signatures of cancer, while the goal of the second is to develop improved confocal endomicroscopes

Morphological Features of Dysplastic Progression in Epithelium: Quantification of Cytological, Microendoscopic, and Second Harmonic Generation Images

Advances in imaging technology have led to a variety of available clinical and investigational systems. In this collection of studies, we tested the relevance of morphological image feature quantification on several imaging systems and epithelial tissues. Quantification carries the benefit of creating numerical baselines and thresholds of healthy and abnormal tissues, to potentially aid clinicians in determining a diagnosis, as well as providing researchers with standardized, unbiased results for future dissemination and comparison. Morphological image features in proflavine stained oral cells were compared qualitatively to traditional Giemsa stained cells, and then we quantified the nuclear to cytoplasm ratio. We determined that quantification of proflavine stained cells matched our hypothesis, as the nuclei in oral carcinoma cells were significantly larger than healthy oral cells. Proflavine has been used in conjunction with translational fluorescence microendoscopy of the gastrointestinal tract, and we demonstrated the ability of our custom algorithm to accurately (up to 85% sensitivity) extract colorectal crypt area and circularity data, which could minimize the burden of training on clinicians. In addition, we proposed fluorescein as an alternative fluorescent dye, providing comparable crypt area and circularity information. In order to investigate the morphological changes of crypts via the supporting collagen structures, we adapted our quantification algorithm to analyze crypt area, circularity, and an additional shape parameter in second harmonic generation images of label-free freshly resected murine epithelium. Murine models of colorectal cancer (CRC) were imaged at early and late stages of tumor progression, and we noted significant differences between the Control groups and the late cancer stages, with some differences between early and late stages of CRC progression

Multispectral image analysis in laparoscopy – A machine learning approach to live perfusion monitoring

Modern visceral surgery is often performed through small incisions. Compared to open surgery, these minimally invasive interventions result in smaller scars, fewer complications and a quicker recovery. While to the patients benefit, it has the drawback of limiting the physician’s perception largely to that of visual feedback through a camera mounted on a rod lens: the laparoscope. Conventional laparoscopes are limited by “imitating” the human eye. Multispectral cameras remove this arbitrary restriction of recording only red, green and blue colors. Instead, they capture many specific bands of light. Although these could help characterize important indications such as ischemia and early stage adenoma, the lack of powerful digital image processing prevents realizing the technique’s full potential. The primary objective of this thesis was to pioneer fluent functional multispectral imaging (MSI) in laparoscopy. The main technical obstacles were: (1) The lack of image analysis concepts that provide both high accuracy and speed. (2) Multispectral image recording is slow, typically ranging from seconds to minutes. (3) Obtaining a quantitative ground truth for the measurements is hard or even impossible. To overcome these hurdles and enable functional laparoscopy, for the first time in this field physical models are combined with powerful machine learning techniques. The physical model is employed to create highly accurate simulations, which in turn teach the algorithm to rapidly relate multispectral pixels to underlying functional changes. To reduce the domain shift introduced by learning from simulations, a novel transfer learning approach automatically adapts generic simulations to match almost arbitrary recordings of visceral tissue. In combination with the only available video-rate capable multispectral sensor, the method pioneers fluent perfusion monitoring with MSI. This system was carefully tested in a multistage process, involving in silico quantitative evaluations, tissue phantoms and a porcine study. Clinical applicability was ensured through in-patient recordings in the context of partial nephrectomy; in these, the novel system characterized ischemia live during the intervention. Verified against a fluorescence reference, the results indicate that fluent, non-invasive ischemia detection and monitoring is now possible. In conclusion, this thesis presents the first multispectral laparoscope capable of videorate functional analysis. The system was successfully evaluated in in-patient trials, and future work should be directed towards evaluation of the system in a larger study. Due to the broad applicability and the large potential clinical benefit of the presented functional estimation approach, I am confident the descendants of this system are an integral part of the next generation OR

Action representation in the mouse parieto-frontal network

The posterior parietal cortex (PPC) and frontal motor areas comprise a cortical network supporting goal-directed behaviour, with functions including sensorimotor transformations and decision making. In primates, this network links performed and observed actions via mirror neurons, which fire both when individuals perform an action and when they observe the same action performed by a conspecific. Mirror neurons are believed to be important for social learning, but it is not known whether mirror-like neurons occur in similar networks in other social species, such as rodents, or if they can be measured in such models using paradigms where observers passively view a demonstrator. Therefore, we imaged Ca2+ responses in PPC and secondary motor cortex (M2) while mice performed and observed pellet-reaching and wheel-running tasks, and found that cell populations in both areas robustly encoded several naturalistic behaviours. However, neural responses to the same set of observed actions were absent, although we verified that observer mice were attentive to performers and that PPC neurons responded reliably to visual cues. Statistical modelling also indicated that executed actions outperformed observed actions in predicting neural responses. These results raise the possibility that sensorimotor action recognition in rodents could take place outside of the parieto-frontal circuit, and underscore that detecting socially-driven neural coding depends critically on the species and behavioural paradigm used