Bio-Inspired Multi-Spectral and Polarization Imaging Sensors for Image-Guided Surgery

Image-guided surgery (IGS) can enhance cancer treatment by decreasing, and ideally eliminating, positive tumor margins and iatrogenic damage to healthy tissue. Current state-of-the-art near-infrared fluorescence imaging systems are bulky, costly, lack sensitivity under surgical illumination, and lack co-registration accuracy between multimodal images. As a result, an overwhelming majority of physicians still rely on their unaided eyes and palpation as the primary sensing modalities to distinguish cancerous from healthy tissue. In my thesis, I have addressed these challenges in IGC by mimicking the visual systems of several animals to construct low power, compact and highly sensitive multi-spectral and color-polarization sensors. I have realized single-chip multi-spectral imagers with 1000-fold higher sensitivity and 7-fold better spatial co-registration accuracy compared to clinical imaging systems in current use by monolithically integrating spectral tapetal and polarization filters with an array of vertically stacked photodetectors. These imaging sensors yield the unique capabilities of imaging simultaneously color, polarization, and multiple fluorophores for near-infrared fluorescence imaging. Preclinical and clinical data demonstrate seamless integration of this technologies in the surgical work flow while providing surgeons with real-time information on the location of cancerous tissue and sentinel lymph nodes, respectively. Due to its low cost, the bio-inspired sensors will provide resource-limited hospitals with much-needed technology to enable more accurate value-based health care

On-site surface reflectometry

The rapid development of Augmented Reality (AR) and Virtual Reality (VR) applications over the past years has created the need to quickly and accurately scan the real world to populate immersive, realistic virtual environments for the end user to enjoy. While geometry processing has already gone a long way towards that goal, with self-contained solutions commercially available for on-site acquisition of large scale 3D models, capturing the appearance of the materials that compose those models remains an open problem in general uncontrolled environments. The appearance of a material is indeed a complex function of its geometry, intrinsic physical properties and furthermore depends on the illumination conditions in which it is observed, thus traditionally limiting the scope of reflectometry to highly controlled lighting conditions in a laboratory setup. With the rapid development of digital photography, especially on mobile devices, a new trend in the appearance modelling community has emerged, that investigates novel acquisition methods and algorithms to relax the hard constraints imposed by laboratory-like setups, for easy use by digital artists. While arguably not as accurate, we demonstrate the ability of such self-contained methods to enable quick and easy solutions for on-site reflectometry, able to produce compelling, photo-realistic imagery. In particular, this dissertation investigates novel methods for on-site acquisition of surface reflectance based on off-the-shelf, commodity hardware. We successfully demonstrate how a mobile device can be utilised to capture high quality reflectance maps of spatially-varying planar surfaces in general indoor lighting conditions. We further present a novel methodology for the acquisition of highly detailed reflectance maps of permanent on-site, outdoor surfaces by exploiting polarisation from reflection under natural illumination. We demonstrate the versatility of the presented approaches by scanning various surfaces from the real world and show good qualitative and quantitative agreement with existing methods for appearance acquisition employing controlled or semi-controlled illumination setups.Open Acces

Polarization- and Specular-Reflection-Based, Non-contact Latent Fingerprint Imaging and Lifting

In forensic science the finger marks left unintentionally by people at a crime scene are referred to as latent fingerprints . Most existing techniques to detect and lift latent fingerprints require application of certain material directly onto the exhibit. The chemical and physical processing applied onto the fingerprint potentially degrades or prevents further forensic testing on the same evidence sample. Many existing methods also come with deleterious side effects. We introduce a method to detect and extract latent fingerprint images without applying any powder or chemicals on the object. Our method is based on the optical phenomena of polarization and specular reflection together with the physiology of fingerprint formation. The recovered image quality is comparable to existing methods. In some cases like the sticky side of a tape our method shows unique advantages

University of Nevada, Las Vegas Transmutation Research Program Annual Report 2002

The UNLV Transmutation Research Program consists of four components: Program Support, Research Infrastructure Augmentation, International Collaboration, and Student Research. In the first year of the program, the student research component was supported by the infrastructure augmentation and the program support components. In the second year of the program, the fourth leg of the support system, international collaborations, was added to the research support system

Invisibility and Cloaking: Origins, Present, and Future Perspectives

The development of metamaterials, i.e., artificially structured materials that interact with waves in unconventional ways, has revolutionized our ability to manipulate the propagation of electromagnetic waves and their interaction with matter. One of the most exciting applications of metamaterial science is related to the possibility of totally suppressing the scattering of an object using an invisibility cloak. Here, we review the available methods to make an object undetectable to electromagnetic waves, and we highlight the outstanding challenges that need to be addressed in order to obtain a fully functional coating capable of suppressing the total scattering of an object. Our outlook discusses how, while passive linear cloaks are fundamentally limited in terms of bandwidth of operation and overall scattering suppression, active and/or nonlinear cloaks hold the promise to overcome, at least partially, some of these limitations.AFOSR Award FA9550-13-1-0204NSF CAREER Award ECCS-0953311DTRA YIP Award HDTRA1-12-1-0022Electrical and Computer Engineerin

The Boston University Photonics Center annual report 2005-2006

This repository item contains an annual report that summarizes activities of the Boston University Photonics Center in the 2005-2006 academic year. The report provides quantitative and descriptive information regarding photonics programs in education, interdisciplinary research, business innovation, and technology development. The Boston University Photonics Center (BUPC) is an interdisciplinary hub for education, research, scholarship, innovation, and technology development associated with practical uses of light.This Annual Report is intended to serve as a synopsis of the Boston University Photonics Center’s wide-ranging activities for the period from July 2005 through June 2006, corresponding to the University’s fiscal year. It is my hope that the document is reflective of the Center’s core values in innovation, entrepreneurship, and education, and that it projects our shared vision, and our dedication to excellence in this exciting field. For further information, you may visit our new website at www.bu.edu/photonics. Though only recently appointed as Director, my involvement in Center activities dates back to the Center’s formation more than ten years ago. In the early years, I worked with a team of faculty and staff colleagues to design and construct the shared laboratories that now provide every Center member extraordinary capabilities for fabrication and testing of advanced photonic devices and systems. I helped launch the business incubator by forming a company around an idea that emerged from my research laboratory. While that company failed to realize its vision of transforming the compact disc industry, it did help us form a unique vision for our program of academically engaged business acceleration. I co-developed a course in optical microsystems for telecommunications that I taught to advanced undergraduates and graduate students in the new M.S. degree program in Photonics offered through the Electrical and Computer Engineering Department. And since the Center’s inception, I have contributed to its scholarly mission through my work in optical microsystem design and precision manufacturing at the Center’s core Precision Engineering Research Laboratory. Recently, I had the opportunity to lead the Provost’s Faculty Advisory Committee on Photonics, charged with broadening the Center’s mission to better integrate academic and educational programs with its more established programs for business incubation and prototype development. [TRUNCATED

University of Nevada, Las Vegas Transmutation Research Program Annual Report Academic Year 2003-2004

It is my pleasure to present the UNLV Transmutation Research Program’s third annual report that highlights the academic year 2003 – 2004. Supporting this document are the many technical reports and scientific papers that have been generated over the past three years. In the third year of our program, we experienced infrastructure growth despite a decreasing budget. This past year we continued into the final phases of the initial 16 independent student research tasks started in 2001 and 2002, supporting 45 graduate students and 11 undergraduates in 6 academic departments across the UNLV scientific and engineering communities during the academic year 2003 – 2004

Polarimetric Multi-View Inverse Rendering

A polarization camera has great potential for 3D reconstruction since the angle of polarization (AoP) and the degree of polarization (DoP) of reflected light are related to an object's surface normal. In this paper, we propose a novel 3D reconstruction method called Polarimetric Multi-View Inverse Rendering (Polarimetric MVIR) that effectively exploits geometric, photometric, and polarimetric cues extracted from input multi-view color-polarization images. We first estimate camera poses and an initial 3D model by geometric reconstruction with a standard structure-from-motion and multi-view stereo pipeline. We then refine the initial model by optimizing photometric rendering errors and polarimetric errors using multi-view RGB, AoP, and DoP images, where we propose a novel polarimetric cost function that enables an effective constraint on the estimated surface normal of each vertex, while considering four possible ambiguous azimuth angles revealed from the AoP measurement. The weight for the polarimetric cost is effectively determined based on the DoP measurement, which is regarded as the reliability of polarimetric information. Experimental results using both synthetic and real data demonstrate that our Polarimetric MVIR can reconstruct a detailed 3D shape without assuming a specific surface material and lighting condition.Comment: Paper accepted in IEEE Transactions on Pattern Analysis and Machine Intelligence (2022). arXiv admin note: substantial text overlap with arXiv:2007.0883

Systems evaluation for computer graphics rendering of the total appearance of paintings

One of the challenges when imaging paintings is recording total appearance, that is, the object\u27s color, surface microstructure (gloss), and surface macrostructure (topography). In this thesis, various systems were used to achieve this task, and a psychophysical paired comparison experiment was conducted to evaluate their performance. A pair of strobe lights arranged at 60° from the normal on either side of the painting captured color information where the strobes produced either directional or diffuse illumination geometry. By adding a third strobe, arranging them 120° apart annularly, and cross polarizing, diffuse color and surface normal maps were measured. A fourth strobe was added and the four lights were rearranged 90° apart annularly, capturing similar data. This system was augmented by two scanning linear light sources arranged perpendicularly, facilitating the measurement of spatially varying BRDF and specular maps. A laser scanner was used to capture surface macrostructure and was combined with the diffuse color maps from the four-light configuration. Finally, a dome illumination system was used with software developed by Conservation Heritage Imaging to produce color maps. In all, eight different configurations were achieved and used to image three small paintings with a range of appearance attributes. Twenty-five naive observers compared computer-graphic renderings to the actual painting and judged similarity in terms of total appearance, gloss/shininess, texture, and color. Although the rankings varied with painting, two general trends emerged. First, the four-light configuration with or without the independent laser scanning produced images visually equivalent to conventional strobe illumination. Second, diffuse illumination was always ranked lowest