Information Extraction Techniques in Hyperspectral Imaging Biomedical Applications

Hyperspectral imaging (HSI) is a technology able to measure information about the spectral reflectance or transmission of light from the surface. The spectral data, usually within the ultraviolet and infrared regions of the electromagnetic spectrum, provide information about the interaction between light and different materials within the image. This fact enables the identification of different materials based on such spectral information. In recent years, this technology is being actively explored for clinical applications. One of the most relevant challenges in medical HSI is the information extraction, where image processing methods are used to extract useful information for disease detection and diagnosis. In this chapter, we provide an overview of the information extraction techniques for HSI. First, we introduce the background of HSI, and the main motivations of its usage for medical applications. Second, we present information extraction techniques based on both light propagation models within tissue and machine learning approaches. Then, we survey the usage of such information extraction techniques in HSI biomedical research applications. Finally, we discuss the main advantages and disadvantages of the most commonly used image processing approaches and the current challenges in HSI information extraction techniques in clinical applications

Intraoperative tissue classification methods in orthopedic and neurological surgeries: A systematic review

Accurate tissue differentiation during orthopedic and neurological surgeries is critical, given that such surgeries involve operations on or in the vicinity of vital neurovascular structures and erroneous surgical maneuvers can lead to surgical complications. By now, the number of emerging technologies tackling the problem of intraoperative tissue classification methods is increasing. Therefore, this systematic review paper intends to give a general overview of existing technologies. The review was done based on the PRISMA principle and two databases: PubMed and IEEE Xplore. The screening process resulted in 60 full-text papers. The general characteristics of the methodology from extracted papers included data processing pipeline, machine learning methods if applicable, types of tissues that can be identified with them, phantom used to conduct the experiment, and evaluation results. This paper can be useful in identifying the problems in the current status of the state-of-the-art intraoperative tissue classification methods and designing new enhanced techniques

Hyperspectral imaging benchmark based on machine learning for intraoperative brain tumour detection

publishedVersio

In-Vivo Hyperspectral Human Brain Image Database for Brain Cancer Detection

The use of hyperspectral imaging for medical applications is becoming more common in recent years. One of the main obstacles that researchers find when developing hyperspectral algorithms for medical applications is the lack of specific, publicly available, and hyperspectral medical data. The work described in this paper was developed within the framework of the European project HELICoiD (HypErspectraL Imaging Cancer Detection), which had as a main goal the application of hyperspectral imaging to the delineation of brain tumors in real-time during neurosurgical operations. In this paper, the methodology followed to generate the first hyperspectral database of in-vivo human brain tissues is presented. Data was acquired employing a customized hyperspectral acquisition system capable of capturing information in the Visual and Near InfraRed (VNIR) range from 400 to 1000 nm. Repeatability was assessed for the cases where two images of the same scene were captured consecutively. The analysis reveals that the system works more efficiently in the spectral range between 450 and 900 nm. A total of 36 hyperspectral images from 22 different patients were obtained. From these data, more than 300 000 spectral signatures were labeled employing a semi-automatic methodology based on the spectral angle mapper algorithm. Four different classes were defined: normal tissue, tumor tissue, blood vessel, and background elements. All the hyperspectral data has been made available in a public repository

Artificial Intelligence in Brain Tumour Surgery—An Emerging Paradigm

Artificial intelligence (AI) platforms have the potential to cause a paradigm shift in brain tumour surgery. Brain tumour surgery augmented with AI can result in safer and more effective treatment. In this review article, we explore the current and future role of AI in patients undergoing brain tumour surgery, including aiding diagnosis, optimising the surgical plan, providing support during the operation, and better predicting the prognosis. Finally, we discuss barriers to the successful clinical implementation, the ethical concerns, and we provide our perspective on how the field could be advanced

Development of deep learning methods for head and neck cancer detection in hyperspectral imaging and digital pathology for surgical guidance

Surgeons performing routine cancer resections utilize palpation and visual inspection, along with time-consuming microscopic tissue analysis, to ensure removal of cancer. Despite this, inadequate surgical cancer margins are reported for up to 10-20% of head and neck squamous cell carcinoma (SCC) operations. There exists a need for surgical guidance with optical imaging to ensure complete cancer resection in the operating room. The objective of this dissertation is to evaluate hyperspectral imaging (HSI) as a non-contact, label-free optical imaging modality to provide intraoperative diagnostic information. For comparison of different optical methods, autofluorescence, RGB composite images synthesized from HSI, and two fluorescent dyes are also acquired and investigated for head and neck cancer detection. A novel and comprehensive dataset was obtained of 585 excised tissue specimens from 204 patients undergoing routine head and neck cancer surgeries. The first aim was to use SCC tissue specimens to determine the potential of HSI for surgical guidance in the challenging task of head and neck SCC detection. It is hypothesized that HSI could reduce time and provide quantitative cancer predictions. State-of-the-art deep learning algorithms were developed for SCC detection in 102 patients and compared to other optical methods. HSI detected SCC with a median AUC score of 85%, and several anatomical locations demonstrated good SCC detection, such as the larynx, oropharynx, hypopharynx, and nasal cavity. To understand the ability of HSI for SCC detection, the most important spectral features were calculated and correlated with known cancer physiology signals, notably oxygenated and deoxygenated hemoglobin. The second aim was to evaluate HSI for tumor detection in thyroid and salivary glands, and RGB images were synthesized using the spectral response curves of the human eye for comparison. Using deep learning, HSI detected thyroid tumors with 86% average AUC score, which outperformed fluorescent dyes and autofluorescence, but HSI-synthesized RGB imagery performed with 90% AUC score. The last aim was to develop deep learning algorithms for head and neck cancer detection in hundreds of digitized histology slides. Slides containing SCC or thyroid carcinoma can be distinguished from normal slides with 94% and 99% AUC scores, respectively, and SCC and thyroid carcinoma can be localized within whole-slide images with 92% and 95% AUC scores, respectively. In conclusion, the outcomes of this thesis work demonstrate that HSI and deep learning methods could aid surgeons and pathologists in detecting head and neck cancers.Ph.D

Lightfield hyperspectral imaging in neuro-oncology surgery: an IDEAL 0 and 1 study

IntroductionHyperspectral imaging (HSI) has shown promise in the field of intra-operative imaging and tissue differentiation as it carries the capability to provide real-time information invisible to the naked eye whilst remaining label free. Previous iterations of intra-operative HSI systems have shown limitations, either due to carrying a large footprint limiting ease of use within the confines of a neurosurgical theater environment, having a slow image acquisition time, or by compromising spatial/spectral resolution in favor of improvements to the surgical workflow. Lightfield hyperspectral imaging is a novel technique that has the potential to facilitate video rate image acquisition whilst maintaining a high spectral resolution. Our pre-clinical and first-in-human studies (IDEAL 0 and 1, respectively) demonstrate the necessary steps leading to the first in-vivo use of a real-time lightfield hyperspectral system in neuro-oncology surgery.MethodsA lightfield hyperspectral camera (Cubert Ultris ×50) was integrated in a bespoke imaging system setup so that it could be safely adopted into the open neurosurgical workflow whilst maintaining sterility. Our system allowed the surgeon to capture in-vivo hyperspectral data (155 bands, 350–1,000 nm) at 1.5 Hz. Following successful implementation in a pre-clinical setup (IDEAL 0), our system was evaluated during brain tumor surgery in a single patient to remove a posterior fossa meningioma (IDEAL 1). Feedback from the theater team was analyzed and incorporated in a follow-up design aimed at implementing an IDEAL 2a study.ResultsFocusing on our IDEAL 1 study results, hyperspectral information was acquired from the cerebellum and associated meningioma with minimal disruption to the neurosurgical workflow. To the best of our knowledge, this is the first demonstration of HSI acquisition with 100+ spectral bands at a frame rate over 1Hz in surgery.DiscussionThis work demonstrated that a lightfield hyperspectral imaging system not only meets the design criteria and specifications outlined in an IDEAL-0 (pre-clinical) study, but also that it can translate into clinical practice as illustrated by a successful first in human study (IDEAL 1). This opens doors for further development and optimisation, given the increasing evidence that hyperspectral imaging can provide live, wide-field, and label-free intra-operative imaging and tissue differentiation

A New Representation for Spectral Data Applied to Raman Spectroscopy of Brain Cancer

Par sa nature infiltrative et son confinement derrière la barrière hémo-encéphalique, le cancer primaire du cerveau est l’une des néoplasies les plus difficiles à diagnostiquer et traiter. Son traitement repose sur la résection chirurgicale maximale. La spectroscopie Raman, capable d’identifier en temps réel des régions cancéreuses qui apparaîtraient normales à l’œil nu, promet d’améliorer considérablement le guidage neurochirurgical et maximiser la résection de la masse tumorale. Cependant, le signal Raman est très complexe à interpréter : les systèmes Raman peuvent maintenant capter des signaux de grande qualité que les méthodes analytiques actuelles ne parviennent pas à interpréter de manière reproductible. Ceci constitue une barrière importante à l’acceptation de la spectroscopie Raman par les médecins et les chercheurs œuvrant sur le cancer du cerveau. L’objectif de ce travail est de développer une méthode robuste d’ingénierie des variables (« Feature engineering ») qui permettrait d’identifier les processus moléculaires exploités par les systèmes Raman pour différentier les régions cancéreuses des régions saines lors de chirurgies cérébrales. Tout d’abord, nous avons identifié les régions Raman ayant une haute spécificité à notre problématique clinique par une revue systématique de la littérature. Un algorithme d’ajustement de courbe a été développé afin d’extraire la forme des pics Raman dans les régions sélectionnées. Puis, nous avons élaboré un modèle mathématique qui tient compte de l’interactivité entre les molécules de l’échantillon interrogé, ainsi qu’entre le signal Raman et l’âge du patient opéré. Pour valider le modèle, nous avons comparé sa capacité à compresser le signal avec celle de l’analyse en composante principale (ACP), le standard en spectroscopie Raman. Finalement, nous avons appliqué la méthode d’ingénierie des variables à des spectres Raman acquis en salle d’opération afin d’identifier quels processus moléculaires indiquaient la présence de cancer. Notre méthode a démontré une meilleure rétention d’information que l’ACP. En l’appliquant aux spectres Raman in vivo, les zones denses en cellules malignes démontrent une expression augmentée d’acides nucléiques ainsi que de certaines protéines, notamment le collagène, le tryptophan et la phénylalanine. De plus, l’âge des patients semble affecter l’impact qu’ont certaines protéines, lipides et acides nucléiques sur le spectre Raman. Nos travaux révèlent l’importance d’une modélisation statistique appropriée pour l’implémentation clinique de systèmes Raman chirurgicaux.----------ABSTRACT Because of its infiltrative nature and concealment behind the blood-brain barrier, primary brain cancer remains one of the most challenging oncological condition to diagnose and treat. The mainstay of treatment is maximal surgical resection. Raman spectroscopy has shown great promise to guide surgeons intraoperatively by identifying, in real-time, dense cancer regions that appear normal to the naked eye. The Raman signal of living tissue is, however, very challenging to interpret, and while most advances in Raman systems targeted the hardware, appropriate statistical modeling techniques are lacking. As a result, there is conflicting evidence as to which molecular processes are captured by Raman probes. This limitation hinders clinical translation and usage of the technology by the cancer-research community. This work focuses on the analytical aspect of Raman-based surgical systems. Its objective is to develop a robust data processing pipeline to confidently identify which molecular phenomena allow Raman systems to differentiate healthy brain and cancer during neurosurgeries. We first selected high-yield Raman regions based on previous literature on the subject, resulting in a list of reproducible Raman bands with high likelihood of brain-specific Raman signal. We then developed a peak-fitting algorithm to extract the shape (height and width) of the Raman signal at those specific bands. We described a mathematical model that accounted for all possible interactions between the selected Raman peaks, and the interaction between the peaks’ shape and the patient’s age. To validate the model, we compared its capacity to compress the signal while maintaining high information content against a Principal Component Analysis (PCA) of the Raman spectra, the fields’ standard. As a final step, we applied the feature engineering model to a dataset of intraoperative human Raman spectra to identify which molecular processes were indicative of brain cancer. Our method showed better information retention than PCA. Our analysis of in vivo Raman measurement showed that areas with high-density of malignant cells had increased expression of nucleic acids and protein compounds, notably collagen, tryptophan and phenylalanine. Patient age seemed to affect the impact of nucleic acids, proteins and lipids on the Raman spectra. Our work demonstrates the importance of appropriate statistical modeling in the implementation of Raman-based surgical devices