Bio-Inspired Multi-Spectral and Polarization Imaging Sensors for Image-Guided Surgery

Image-guided surgery (IGS) can enhance cancer treatment by decreasing, and ideally eliminating, positive tumor margins and iatrogenic damage to healthy tissue. Current state-of-the-art near-infrared fluorescence imaging systems are bulky, costly, lack sensitivity under surgical illumination, and lack co-registration accuracy between multimodal images. As a result, an overwhelming majority of physicians still rely on their unaided eyes and palpation as the primary sensing modalities to distinguish cancerous from healthy tissue. In my thesis, I have addressed these challenges in IGC by mimicking the visual systems of several animals to construct low power, compact and highly sensitive multi-spectral and color-polarization sensors. I have realized single-chip multi-spectral imagers with 1000-fold higher sensitivity and 7-fold better spatial co-registration accuracy compared to clinical imaging systems in current use by monolithically integrating spectral tapetal and polarization filters with an array of vertically stacked photodetectors. These imaging sensors yield the unique capabilities of imaging simultaneously color, polarization, and multiple fluorophores for near-infrared fluorescence imaging. Preclinical and clinical data demonstrate seamless integration of this technologies in the surgical work flow while providing surgeons with real-time information on the location of cancerous tissue and sentinel lymph nodes, respectively. Due to its low cost, the bio-inspired sensors will provide resource-limited hospitals with much-needed technology to enable more accurate value-based health care

Unsupervised hyperspectral image segmentation of films: a hierarchical clustering-based approach

Hyperspectral imaging (HSI) has been drastically applied in recent years to cultural heritage (CH) analysis, conservation, and also digital restoration. However, the efficient processing of the large datasets registered remains challenging and still in development. In this paper, we propose to use the hierarchical clustering algorithm (HCA) as an alternative machine learning approach to the most common practices, such as principal component analysis(PCA). HCA has shown its potential in the past decades for spectral data classification and segmentation in many other fields, maximizing the information to be extracted from the high-dimensional spectral dataset via the formation of the agglomerative hierarchical tree. However, to date, there has been very limited implementation of HCA in the field of cultural heritage. Data used in this experiment were acquired on real historic film samples with various degradation degrees, using a custom-made push-broom VNIR hyperspectral camera (380–780nm). With the proposed HCA workflow, multiple samples in the entire dataset were processed simultaneously and the degradation areas with distinctive characteristics were successfully segmented into clusters with various hierarchies. A range of algorithmic parameters was tested, including the grid sizes, metrics, and agglomeration methods, and the best combinations were proposed at the end. This novel application of the semi-automating and unsupervised HCA could provide a basis for future digital unfading, and show the potential to solve other CH problems such as pigment mapping

Hyperspectral benthic mapping from underwater robotic platforms

We live on a planet of vast oceans; 70% of the Earth's surface is covered in water. They are integral to supporting life, providing 99% of the inhabitable space on Earth. Our oceans and the habitats within them are under threat due to a variety of factors. To understand the impacts and possible solutions, the monitoring of marine habitats is critically important. Optical imaging as a method for monitoring can provide a vast array of information however imaging through water is complex. To compensate for the selective attenuation of light in water, this thesis presents a novel light propagation model and illustrates how it can improve optical imaging performance. An in-situ hyperspectral system is designed which comprised of two upward looking spectrometers at different positions in the water column. The downwelling light in the water column is continuously sampled by the system which allows for the generation of a dynamic water model. In addition to the two upward looking spectrometers the in-situ system contains an imaging module which can be used for imaging of the seafloor. It consists of a hyperspectral sensor and a trichromatic stereo camera. New calibration methods are presented for the spatial and spectral co-registration of the two optical sensors. The water model is used to create image data which is invariant to the changing optical properties of the water and changing environmental conditions. In this thesis the in-situ optical system is mounted onboard an Autonomous Underwater Vehicle. Data from the imaging module is also used to classify seafloor materials. The classified seafloor patches are integrated into a high resolution 3D benthic map of the surveyed site. Given the limited imaging resolution of the hyperspectral sensor used in this work, a new method is also presented that uses information from the co-registered colour images to inform a new spectral unmixing method to resolve subpixel materials

ON THE LOGIC, METHOD AND SCIENTIFIC DIVERSITY OF TECHNICAL SYSTEMS: AN INQUIRY INTO THE DIAGNOSTIC MEASUREMENT OF HUMAN SKIN

This dissertation explores some of the scientific, technical and cultural history of human skin measurement and diagnostics. Through a significant collection of primary texts and case studies, I track the changing technologies and methods used to measure skin, as well as the scientific and sociotechnical applications. I then map these histories onto some of the diverse understandings of the human body, physics, biology, natural philosophy and language that underpinned the scientific enterprise of skin measurement. The main argument of my thesis demonstrates how these diverse histories of science historically and theoretically inform the succeeding methods and applications for skin measurement from early Greek medicine, to beginnings of Anthropology as scientific discipline, to the emergence of scientific racism, to the age of digital imaging analysis, remote sensing, algorithms, massive databases and biometric technologies; further, these new digital applications go beyond just health diagnostics and are creating new technical categorizations of human skin divorced from the established ethical mechanisms of modern science. Based on this research, I inquire how communication practices within the scientific enterprise address the ethical and historical implications for a growing set of digital biometric applications with industrial, military, sociopolitical and public functions

Tapestries revealed : novel methods of characterisation, conservation and presentation

The digital conservation of cultural heritage has received significant attention in recent years. This active area of research endeavors to digitally conserve culturally significant items. The digital archives produced serve as an important resource for conservators. These records allow the accurate tracking of the degradation of the materials used in the construction of these artefacts.This project outlines the digital conservation and subsequent presentation of a historically significant tapestry held by the Royal Collection at Hampton Court Palace. The tapestry is one of The Story of Abraham set constructed by Willem de Kempeneer in Brussels in the 1540s. These tapestries were commissioned by King Henry VIII and were displayed as a reflection of his wealth and power. The materials used in their construction included wool, silk, silver and gold threads. The objectives of the Thesis are as follows:1) To digitally conserve the tapestry, the Oath and Departure of Eliezer.2) To produce a colorimetrically accurate projection system. This system will be used to project an accurate representation of the original tapestry colours onto the current photofaded version.3) To investigate the photo-fading properties of the natural dyes used to produce the Oath and Departure of Eliezer and their interactions with the metallic threads woven within the tapestry.The work presented in this Thesis contributed to a visitor exhibition called "Henry VIII's Tapestries Revealed" held at Hampton Court Palace between April 2009-October 2010 as part of Historic Royal Palaces' celebrations of the 500th anniversary of Henry VIII's accession to the throne.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Retrieval of Lake Erie Water Quality Parameters from Satellite Remote Sensing and Impact on Simulations with a 1-D Lake Model

Lake Erie is a freshwater lake, and the most southern of the Laurentian Great Lakes in North America. It is the smallest by volume, the fourth largest in surface area (25,700 km2), and the shallowest of the Laurentian Great Lakes. The lake’s high productivity and warm weather in its watershed has attracted one-third of the total human population of the Great Lake’s basin. The industrial and agricultural activities of this huge population has caused serious environmental problems for Lake Erie namely harmful algal blooms, dissolved organic/inorganic matters from river inputs, and sediment loadings. If these sorts of water contaminations exceed a certain level, it can seriously influence the lake ecosystem. Hence, an effective and continuous water quality monitoring program is of outmost importance for Lake Erie. The use of Earth observation satellites to improve monitoring of environmental changes in water bodies has been receiving increased attention in recent years. Satellite observations can provide long term spatial and temporal trends of water quality indicators which cannot be achieved through discontinuous conventional point-wise in situ sampling. Different regression-based empirical models have been developed in the literature to derive the water optical properties from a single (or band ratio of) remote sensing reflectance (radiance). In situ measurements are used to build these regressions. The repeated in situ measurements in space and/or time causes clustered and correlated data that violates the assumption of regression models. Considering this correlation in developing regression models was one of the topics examined in this thesis. More complicated semi-analytical models are applied in Case II waters, aiming to distinguish several constituents confounding water-leaving signals more effectively. The MERIS neural network (NN) algorithms are the most widely used among semi-analytical models. The applicability of these algorithms to derive chl-a concentration and Secchi Disk Depth (SDD) in Lake Erie was assessed for the first time in this thesis. Satellite-observations of water turbidity were then coupled with a 1-D lake model to improve its performance on Lake Erie, where the common practice is to use a constant value for water turbidity in the model due to insufficient in situ measurements of water turbidity for lakes globally. In the first chapter, four well-established MERIS NN algorithms to derive chl-a concentration as well as two band-ratio chl-a related indices were evaluated against in situ measurements. The investigated products are those produced by NN algorithms, including Case 2 Regional (C2R), Eutrophic (EU), Free University of Berlin WeW WATER processor (FUB/WeW), and CoastColour (CC) processors, as well as from band-ratio algorithms of fluorescence line height (FLH) and maximum chlorophyll index (MCI). Two approaches were taken to compare and evaluate the performance of these algorithms to predict chl-a concentration after lake-specific calibration of the algorithms. First, all available chl-a matchups, which were collected from different locations on the lake, were evaluated at once. In the second approach, a classification of three optical water types was applied, and the algorithms’ performance was assessed for each type, individually. The results of this chapter show that the FUB/WeW processor outperforms other algorithms when the full matchup data of the lake was used (root mean square error (RMSE) = 1.99 mg m-3, index-of-agreement (I_a) = 0.67). However, the best performing algorithm was different when each water optical type was investigated individually. The findings of this study provide practical and valuable information on the effectiveness of the already existing MERIS-based algorithms to derive the trophic state of Lake Erie, an optically complex lake. Unlike the first chapter, where physically-based and already trained algorithms were implemented to evaluate satellite derived chl-a concentration, in the next chapter, two lake-specific, robust semi-empirical algorithms were developed to derive chl-a and SDD using Linear Mixed Effect (LME) models. LME considers the correlation that exists in the field measurements which have been repeatedly performed in space and time. Each developed algorithm was then employed to investigate the monthly-averaged spatial and temporal trends of chl-a concentration and water turbidity during the period of 2005-2011. SDD was used as the indicator of water turbidity. LME models were developed between the logarithmic scale of the parameters and the band ratio of B7:665 nm to B9:708.75 nm for log10chl-a, and the band ratio of B6:620 nm to B4:510 nm for log10SDD. The models resulted in RMSE of 0.30 for log10chl-a and 0.19 for log10SDD. Maps produced with the two LME models revealed distinct monthly patterns for different regions of the lake that are in agreement with the biogeochemical properties of Lake Erie. Lastly the water turbidity (extinction coefficient; Kd) of Lake Erie was estimated using the globally available satellite-based CC product. The CC-derived Kd product was in a good agreement with the SDD field observations (RMSE=0.74 m-1, mean bias error (MBE)=0.53 m-1, I_a=0.53). CC-derived Kd was then used as input for simulations with the 1-D Freshwater Lake (FLake) model. An annual average constant Kd value calculated from the CC product improved simulation results of lake surface water temperature (LSWT) compared to a “generic” constant value (0.2 m-1) used in previous studies (CC lake-specific yearly average Kd value: RMSE=1.54 ºC, MBE= -0.08 ºC; generic constant Kd value: RMSE=1.76 ºC, MBE= -1.26 ºC). Results suggest that a time-independent, lake-specific, and constant Kd value from CC can improve FLake LSWT simulations with sufficient accuracy. A sensitivity analysis was also conducted to assess the performance of FLake to simulate LSWT, mean water column temperature (MWCT) and mixed layer depth (MLD) using different values of Kd. Results showed that the model is very sensitive to the variations of Kd, particularly when Kd value is below 0.5 m-1. The sensitivity of FLake to Kd variations was more pronounced in simulations of MWCT and MLD. This study shows that a global mapping of the extinction coefficient can be created using satellite-based observations of lakes optical properties to improve the 1-D FLake model. Overall, results from this thesis clearly demonstrate the benefits of remote sensing measurements of water quality parameters (such as chl-a concentration and water turbidity) for lake monitoring. Also, this research shows that the integration of space-borne water clarity (extinction coefficient) measurements into the 1-D FLake model improves simulations of LSWT

Chromatic processing in the zebrafish (Danio rerio) inner retina: bipolar cell physiology and open hardware designs for spectrally accurate stimulation under two-photon

Colour vision describes the ability of animals to differentiate objects based on their spectral reflectance properties independent of light intensity. It is an essential evolutionary trait that allows species to efficiently forage for food, avoid predation, break camouflage, communicate with conspecifics, or to find mates. Zebrafish is a powerful model for studying colour vision as it possesses four cone-photoreceptor types which can be categorised as Red-, Green-, Blue- and UV-sensitive. From first principles, its retina therefore holds the potential to process diverse chromatic computations. In the presented work, the focus was on retinal bipolar cells (BC). These are the retina’s first projection neurons. They receive inputs from the photoreceptors in the outer retina, and send their axon terminals to the inner retina, the inner plexiform layer (IPL). Diverse types within this class of interneuron shape light responses collected by the photoreceptor array into parallel channels with diverse spectral properties. BCs also make connections with all other neuron types within the retina, including horizontal cells in the outer retina, and amacrine as well as retinal ganglion cells in the inner retina. This makes them a central hub for spectral processing within the retina. By combining genetically encoded calcium indicator and two-photon microscopy, light-driven activity from larval zebrafish BC synaptic terminals was systematically recorded in vivo. Synaptic responses to tetrachromatic light stimulation unveiled an unprecedented degree of visual specialisation, including retinal regions dedicated to distinct light-guided behaviours. These regional characteristics were further correlated to functional BC types which were strongly associated with specific retinal positions and axonal stratification depths. Overall, BC projections to the inner plexiform layer displayed a sophisticated level of organisation, structured into chromatic and achromatic functional layers which systematically adjusted their response profiles across the eye to match natural spectral input statistics. Together, these findings bolster our understanding of “colour-processing” in this animal’s inner retina and suggest that unlike in mammals, teleost fish BCs already encode complex chromatic responses in the inner plexiform layer before driving retinal ganglion cells. Additionally, the study of colour vision from an organism requires precise control over the light stimuli’s temporal, spatial and spectral features. Therefore, chromatic stimulators, designed to be combined with two-photon microscopy, were developed throughout this work. These devices allowed circumventing experimental limitations, such as spectral crosstalk between the microscope and the stimulus light. Furthermore, they were conceived as open source projects to be easily replicated and adapted to any organism’s retina with different spectral sensitivities through the free control over the number and spectra of stimulation light sources. These open source projects originated from the desire to set up a stimulation standard for the field of visual neuroscience