Bio-Inspired Multi-Spectral Image Sensor and Augmented Reality Display for Near-Infrared Fluorescence Image-Guided Surgery

Background: Cancer remains a major public health problem worldwide and poses a huge economic burden. Near-infrared (NIR) fluorescence image-guided surgery (IGS) utilizes molecular markers and imaging instruments to identify and locate tumors during surgical resection. Unfortunately, current state-of-the-art NIR fluorescence imaging systems are bulky, costly, and lack both fluorescence sensitivity under surgical illumination and co-registration accuracy between multimodal images. Additionally, the monitor-based display units are disruptive to the surgical workflow and are suboptimal at indicating the 3-dimensional position of labeled tumors. These major obstacles have prevented the wide acceptance of NIR fluorescence imaging as the standard of care for cancer surgery. The goal of this dissertation is to enhance cancer treatment by developing novel image sensors and presenting the information using holographic augmented reality (AR) display to the physician in intraoperative settings. Method: By mimicking the visual system of the Morpho butterfly, several single-chip, color-NIR fluorescence image sensors and systems were developed with CMOS technologies and pixelated interference filters. Using a holographic AR goggle platform, an NIR fluorescence IGS display system was developed. Optoelectronic evaluation was performed on the prototypes to evaluate the performance of each component, and small animal models and large animal models were used to verify the overall effectiveness of the integrated systems at cancer detection. Result: The single-chip bio-inspired multispectral logarithmic image sensor I developed has better main performance indicators than the state-of-the-art NIR fluorescence imaging instruments. The image sensors achieve up to 140 dB dynamic range. The sensitivity under surgical illumination achieves 6108 V/(mW/cm2), which is up to 25 times higher. The signal-to-noise ratio is up to 56 dB, which is 11 dB greater. These enable high sensitivity fluorescence imaging under surgical illumination. The pixelated interference filters enable temperature-independent co-registration accuracy between multimodal images. Pre-clinical trials with small animal model demonstrate that the sensor can achieve up to 95% sensitivity and 94% specificity with tumor-targeted NIR molecular probes. The holographic AR goggle provides the physician with a non-disruptive 3-dimensional display in the clinical setup. This is the first display system that co-registers a virtual image with human eyes and allows video rate image transmission. The imaging system is tested in the veterinary science operating room on canine patients with naturally occurring cancers. In addition, a time domain pulse-width-modulation address-event-representation multispectral image sensor and a handheld multispectral camera prototype are developed. Conclusion: The major problems of current state-of-the-art NIR fluorescence imaging systems are successfully solved. Due to enhanced performance and user experience, the bio-inspired sensors and augmented reality display system will give medical care providers much needed technology to enable more accurate value-based healthcare

Implantable CMOS Biomedical Devices

The results of recent research on our implantable CMOS biomedical devices are reviewed. Topics include retinal prosthesis devices and deep-brain implantation devices for small animals. Fundamental device structures and characteristics as well as in vivo experiments are presented

Beyond solid-state lighting: Miniaturization, hybrid integration, and applications og GaN nano- and micro-LEDs

Gallium Nitride (GaN) light-emitting-diode (LED) technology has been the revolution in modern lighting. In the last decade, a huge global market of efficient, long-lasting and ubiquitous white light sources has developed around the inception of the Nobel-price-winning blue GaN LEDs. Today GaN optoelectronics is developing beyond lighting, leading to new and innovative devices, e.g. for micro-displays, being the core technology for future augmented reality and visualization, as well as point light sources for optical excitation in communications, imaging, and sensing. This explosion of applications is driven by two main directions: the ability to produce very small GaN LEDs (microLEDs and nanoLEDs) with high efficiency and across large areas, in combination with the possibility to merge optoelectronic-grade GaN microLEDs with silicon microelectronics in a fully hybrid approach. GaN LED technology today is even spreading into the realm of display technology, which has been occupied by organic LED (OLED) and liquid crystal display (LCD) for decades. In this review, the technological transition towards GaN micro- and nanodevices beyond lighting is discussed including an up-to-date overview on the state of the art

Light-Addressing and Chemical Imaging Technologies for Electrochemical Sensing

Visualizing chemical components in a specimen is an essential technology in many branches of science and practical applications. This book deals with electrochemical imaging techniques based on semiconductor devices with capability of spatially resolved sensing. Two types of such sensing devices have been extensively studied and applied in various fields, i.e., arrayed sensors and light-addressed sensors. An ion-sensitive field-effect transistor (ISFET) array and a charge-coupled device (CCD) ion image sensor are examples of arrayed sensors. They take advantage of semiconductor microfabrication technology to integrate a large number of sensing elements on a single chip, each representing a pixel to form a chemical image. A light-addressable potentiometric sensor (LAPS), on the other hand, has no pixel structure. A chemical image is obtained by raster-scanning the sensor plate with a light beam, which can flexibly define the position and size of a pixel. This light-addressing approach is further applied in other LAPS-inspired methods. Scanning photo-induced impedance microscopy (SPIM) realized impedance mapping and light-addressable electrodes/light-activated electrochemistry (LAE) realized local activation of Faradaic processes. This book includes eight articles on state-of-the-art technologies of light-addressing/chemical imaging devices and their application to biology and materials science

A 72 × 60 Angle-Sensitive SPAD Imaging Array for Lens-less FLIM

We present a 72 × 60, angle-sensitive single photon avalanche diode (A-SPAD) array for lens-less 3D fluorescence lifetime imaging. An A-SPAD pixel consists of (1) a SPAD to provide precise photon arrival time where a time-resolved operation is utilized to avoid stimulus-induced saturation, and (2) integrated diffraction gratings on top of the SPAD to extract incident angles of the incoming light. The combination enables mapping of fluorescent sources with different lifetimes in 3D space down to micrometer scale. Futhermore, the chip presented herein integrates pixel-level counters to reduce output data-rate and to enable a precise timing control. The array is implemented in standard 180 nm complementary metal-oxide-semiconductor (CMOS) technology and characterized without any post-processing