Bio-Inspired Multi-Spectral Imaging Sensors and Algorithms for Image Guided Surgery

Image guided surgery (IGS) utilizes emerging imaging technologies to provide additional structural and functional information to the physician in clinical settings. This additional visual information can help physicians delineate cancerous tissue during resection as well as avoid damage to near-by healthy tissue. Near-infrared (NIR) fluorescence imaging (700 nm to 900 nm wavelengths) is a promising imaging modality for IGS, namely for the following reasons: First, tissue absorption and scattering in the NIR window is very low, which allows for deeper imaging and localization of tumor tissue in the range of several millimeters to a centimeter depending on the tissue surrounding the tumor. Second, spontaneous tissue fluorescence emission is minimal in the NIR region, allowing for high signal-to-background ratio imaging compared to visible spectrum fluorescence imaging. Third, decoupling the fluorescence signal from the visible spectrum allows for optimization of NIR fluorescence while attaining high quality color images. Fourth, there are two FDA approved fluorescent dyes in the NIR region—namely methylene blue (MB) and indocyanine green—which can help to identify tumor tissue due to passive accumulation in human subjects. The aforementioned advantages have led to the development of NIR fluorescence imaging systems for a variety of clinical applications, such as sentinel lymph node imaging, angiography, and tumor margin assessment. With these technological advances, secondary surgeries due to positive tumor margins or damage to healthy organs can be largely mitigated, reducing the emotional and financial toll on the patient. Currently, several NIR fluorescence imaging systems (NFIS) are available commercially or are undergoing clinical trials, such as FLARE, SPY, PDE, Fluobeam, and others. These systems capture multi-spectral images using complex optical equipment and are combined with real-time image processing to present an augmented view to the surgeon. The information is presented on a standard monitor above the operating bed, which requires the physician to stop the surgical procedure and look up at the monitor. The break in the surgical flow sometimes outweighs the benefits of fluorescence based IGS, especially in time-critical surgical situations. Furthermore, these instruments tend to be very bulky and have a large foot print, which significantly complicates their adoption in an already crowded operating room. In this document, I present the development of a compact and wearable goggle system capable of real-time sensing of both NIR fluorescence and color information. The imaging system is inspired by the ommatidia of the monarch butterfly, in which pixelated spectral filters are integrated with light sensitive elements. The pixelated spectral filters are fabricated via a carefully optimized nanofabrication procedure and integrated with a CMOS imaging array. The entire imaging system has been optimized for high signal-to-background fluorescence imaging using an analytical approach, and the efficacy of the system has been experimentally verified. The bio-inspired spectral imaging sensor is integrated with an FPGA for compact and real-time signal processing and a wearable goggle for easy integration in the operating room. The complete imaging system is undergoing clinical trials at Washington University in the St. Louis Medical School for imaging sentinel lymph nodes in both breast cancer patients and melanoma patients

Split-screen single-camera stereoscopic PIV application to a turbulent confined swirling layer with free surface

An annular liquid wall jet, or vortex tube, generated by helical injection inside a tube is studied experimentally as a possible means of fusion reactor shielding. The hollow confined vortex/swirling layer exhibits simultaneously all the complexities of swirling turbulence, free surface, droplet formation, bubble entrapment; all posing challenging diagnostic issues. The construction of flow apparatus and the choice of working liquid and seeding particles facilitate unimpeded optical access to the flow field. A split-screen, single-camera stereoscopic particle image velocimetry (SPIV) scheme is employed for flow field characterization. Image calibration and free surface identification issues are discussed. The interference in measurements of laser beam reflection at the interface are identified and discussed. Selected velocity measurements and turbulence statistics are presented at Re_λ = 70 (Re = 3500 based on mean layer thickness)

Digital elevation modelling by radargrammetry in real-beam ground mapping mode

In this thesis the problem of estimating terrain elevation using two-dimensional radar data from the multirole ghter aircraft JAS 39 Gripen is considered. Radar data contains information about range from the aircraft to the reflecting terrain, as well as horizontal angle. In general, radar data has high resolution in range and low resolution in angle, giving rise to interesting problems. A new radar with higher resolution is in development for the next-generation Gripen. This thesis aims at answering whether estimation of terrain height can be done using available radar data, in order to evaluate the plausibility of doing this with data from the new radar. The approach is to find matching terrain features in subsequent images of the ground, and use this information to calculate terrain elevation. Two approaches are implemented and studied, both on simulated radar data and on real datasets. One approach uses Harris corner detection and the other uses Speeded-Up Robust Features (SURF). Conclusions drawn are that the algorithms do not work for the available radar data, but that they possibly could work when higher resolution data from the new radar is available

Experimental and simulation study results for video landmark acquisition and tracking technology

A synopsis of related Earth observation technology is provided and includes surface-feature tracking, generic feature classification and landmark identification, and navigation by multicolor correlation. With the advent of the Space Shuttle era, the NASA role takes on new significance in that one can now conceive of dedicated Earth resources missions. Space Shuttle also provides a unique test bed for evaluating advanced sensor technology like that described in this report. As a result of this type of rationale, the FILE OSTA-1 Shuttle experiment, which grew out of the Video Landmark Acquisition and Tracking (VILAT) activity, was developed and is described in this report along with the relevant tradeoffs. In addition, a synopsis of FILE computer simulation activity is included. This synopsis relates to future required capabilities such as landmark registration, reacquisition, and tracking