Tongue Tumor Detection in Medical Hyperspectral Images

A hyperspectral imaging system to measure and analyze the reflectance spectra of the human tongue with high spatial resolution is proposed for tongue tumor detection. To achieve fast and accurate performance for detecting tongue tumors, reflectance data were collected using spectral acousto-optic tunable filters and a spectral adapter, and sparse representation was used for the data analysis algorithm. Based on the tumor image database, a recognition rate of 96.5% was achieved. The experimental results show that hyperspectral imaging for tongue tumor diagnosis, together with the spectroscopic classification method provide a new approach for the noninvasive computer-aided diagnosis of tongue tumors

A Multi-layer Perceptron Approach to Automatically Detect Tissue via NIR Multispectral Imaging

We present a novel pixel-level spectra based multi-layer perceptron(MLP) to discriminate regions of biomedical multispectral imagingdata into two categories: tissue and non-tissue. The spectra usedfor this study are 740nm, 780nm, 850nm, and 945nm as thesewavelengths are on either side of the isosbestic point for oxyhemoglobinand deoxyhemoglobin; absorbers that are common in allhealthy tissues. An MLP is trained using multispectral data from12 human subjects and 12 non-tissue objects. The MLP is testedon three multispectral challenge image sets, from which the accuracy,sensitivity, and specificity of the model yield results of 91.3%(+/-0.2%), 98.1% (+/-0.3%), and 88.5% (+/- 0.3%) respectively

Leveraging Computer Vision for Applications in Biomedicine and Geoscience

Skin cancer is one of the most common types of cancer and is usually classified as either non-melanoma and melanoma skin cancer. Melanoma skin cancer accounts for about half of all skin cancer-related deaths. The 5-year survival rate is 99% when the cancer is detected early but drops to 25% once it becomes metastatic. In other words, the key to preventing death is early detection. Foraminifera are microscopic single-celled organisms that exist in marine environments and are classified as living a benthic or planktic lifestyle. In total, roughly 50,000 species are known to have existed, of which about 9,000 are still living today. Foraminifera are important proxies for reconstructing past ocean and climate conditions and as bio-indicators of anthropogenic pollution. Since the 1800s, the identification and counting of foraminifera have been performed manually. The process is resource-intensive. In this dissertation, we leverage recent advances in computer vision, driven by breakthroughs in deep learning methodologies and scale-space theory, to make progress towards both early detection of melanoma skin cancer and automation of the identification and counting of microscopic foraminifera. First, we investigate the use of hyperspectral images in skin cancer detection by performing a critical review of relevant, peer-reviewed research. Second, we present a novel scale-space methodology for detecting changes in hyperspectral images. Third, we develop a deep learning model for classifying microscopic foraminifera. Finally, we present a deep learning model for instance segmentation of microscopic foraminifera. The works presented in this dissertation are valuable contributions in the fields of biomedicine and geoscience, more specifically, towards the challenges of early detection of melanoma skin cancer and automation of the identification, counting, and picking of microscopic foraminifera

Low cost hyperspectral imaging using deep learning based spectral reconstruction

The increasing number of applications of hyperspectral imaging results in a high demand for low cost, mobile devices. We propose a multispectral imaging (MSI) system based on time-multiplexed lighting using RGB Light Emitting Diodes (LED). We train a deep neural network that maps low dimensional multispectral input onto high dimensional hyperspectral (HSI) output that is collected with a HSI camera covering the range of 400 – 950 nm. Results on the 24 colour patches of the Macbeth colour checker chart show that with only five multispectral bands, a very accurate reconstruction of HSI data can be achieved

Bibliometric analysis of the current status and trends on medical hyperspectral imaging

Hyperspectral imaging (HSI) is a promising technology that can provide valuable support for the advancement of the medical field. Bibliometrics can analyze a vast number of publications on both macroscopic and microscopic levels, providing scholars with essential foundations to shape future directions. The purpose of this study is to comprehensively review the existing literature on medical hyperspectral imaging (MHSI). Based on the Web of Science (WOS) database, this study systematically combs through literature using bibliometric methods and visualization software such as VOSviewer and CiteSpace to draw scientific conclusions. The analysis yielded 2,274 articles from 73 countries/regions, involving 7,401 authors, 2,037 institutions, 1,038 journals/conferences, and a total of 7,522 keywords. The field of MHSI is currently in a positive stage of development and has conducted extensive research worldwide. This research encompasses not only HSI technology but also its application to diverse medical research subjects, such as skin, cancer, tumors, etc., covering a wide range of hardware constructions and software algorithms. In addition to advancements in hardware, the future should focus on the development of algorithm standards for specific medical research targets and cultivate medical professionals of managing vast amounts of technical information

Multispectral image analysis in laparoscopy – A machine learning approach to live perfusion monitoring

Modern visceral surgery is often performed through small incisions. Compared to open surgery, these minimally invasive interventions result in smaller scars, fewer complications and a quicker recovery. While to the patients benefit, it has the drawback of limiting the physician’s perception largely to that of visual feedback through a camera mounted on a rod lens: the laparoscope. Conventional laparoscopes are limited by “imitating” the human eye. Multispectral cameras remove this arbitrary restriction of recording only red, green and blue colors. Instead, they capture many specific bands of light. Although these could help characterize important indications such as ischemia and early stage adenoma, the lack of powerful digital image processing prevents realizing the technique’s full potential. The primary objective of this thesis was to pioneer fluent functional multispectral imaging (MSI) in laparoscopy. The main technical obstacles were: (1) The lack of image analysis concepts that provide both high accuracy and speed. (2) Multispectral image recording is slow, typically ranging from seconds to minutes. (3) Obtaining a quantitative ground truth for the measurements is hard or even impossible. To overcome these hurdles and enable functional laparoscopy, for the first time in this field physical models are combined with powerful machine learning techniques. The physical model is employed to create highly accurate simulations, which in turn teach the algorithm to rapidly relate multispectral pixels to underlying functional changes. To reduce the domain shift introduced by learning from simulations, a novel transfer learning approach automatically adapts generic simulations to match almost arbitrary recordings of visceral tissue. In combination with the only available video-rate capable multispectral sensor, the method pioneers fluent perfusion monitoring with MSI. This system was carefully tested in a multistage process, involving in silico quantitative evaluations, tissue phantoms and a porcine study. Clinical applicability was ensured through in-patient recordings in the context of partial nephrectomy; in these, the novel system characterized ischemia live during the intervention. Verified against a fluorescence reference, the results indicate that fluent, non-invasive ischemia detection and monitoring is now possible. In conclusion, this thesis presents the first multispectral laparoscope capable of videorate functional analysis. The system was successfully evaluated in in-patient trials, and future work should be directed towards evaluation of the system in a larger study. Due to the broad applicability and the large potential clinical benefit of the presented functional estimation approach, I am confident the descendants of this system are an integral part of the next generation OR

PET and PVC separation with hyperspectral Imagery

Traditional plants for plastic separation in homogeneous products employ material physical properties (for instance density). Due to the small intervals of variability of different polymer properties, the output quality may not be adequate. Sensing technologies based on hyperspectral imaging have been introduced in order to classify materials and to increase the quality of recycled products, which have to comply with specific standards determined by industrial applications. This paper presents the results of the characterization of two different plastic polymers—polyethylene terephthalate (PET) and polyvinyl chloride (PVC)—in different phases of their life cycle (primary raw materials, urban and urban-assimilated waste and secondary raw materials) to show the contribution of hyperspectral sensors in the field of material recycling. This is accomplished via near-infrared (900–1700 nm) reflectance spectra extracted from hyperspectral images acquired with a two-linear-spectrometer apparatus. Results have shown that a rapid and reliable identification of PET and PVC can be achieved by using a simple two near-infrared wavelength operator coupled to an analysis of reflectance spectra. This resulted in 100% classification accuracy. A sensor based on this identification method appears suitable and inexpensive to build and provides the necessary speed and performance required by the recycling industry

Neural Network Classifiers for Human Tissue Classification in NIR Biomedical Multispectral Imaging

Near infrared imaging (NIR) is an imaging modality that has gained traction for solving biomedical problems in recent years. By leveraging the NIR spectrum, multiple spectra from the NIR range can be used to extract meaningful data from a variety of targets including human tissue; this technique is known as multispectral imaging (MSI) analysis. A generalized tissue classification method that identifies human tissue in an NIR multispectral imaging field is explored. NIR images are captured from four different wavelengths, and features are extracted from the individual images. The features are then manually labeled and used to train machine learning models to identify tissue/non-tissue areas within a multispectral image set. Although the application in this thesis is used to classify tissue/non-tissue, the techniques presented can be generalized to solve many other MSI classification problems in a variety of fields. In particular, two machine learning models are explored in this thesis; a multi-layer perceptron (MLP) and a convolutional neural network (CNN) approach. For each approach, feature selection and hyper-parameter tuning were used to design the machine learning architectures. After the design process, quantitative and qualitative tests were conducted to evaluate the merits of each algorithm design. Analysis found that the CNN approach yields excellent reliability and accuracy compared to the MLP. The accuracy, sensitivity, and specificity of the CNN is 95.2, 94.4, and 95.7% as calculated on a test set of MSI data. The MLP results on the same data set yield accuracy, sensitivity, and specificity values of 83.9, 85.4, and 83.1% respectively. It is also demonstrated that the CNN design maintains excellent accuracy even when challenged with varying tissue types and body compositions. The impact of this research will be most applicable to biomedical imaging modalities that utilize multispectral data. The techniques presented can be used to classify different types of tissues and their pathologies. Furthermore, the techniques can be generalized to other fields where multispectral data is used for inferencing, such as remote sensing applications