Event-based Vision: A Survey

Event cameras are bio-inspired sensors that differ from conventional frame cameras: Instead of capturing images at a fixed rate, they asynchronously measure per-pixel brightness changes, and output a stream of events that encode the time, location and sign of the brightness changes. Event cameras offer attractive properties compared to traditional cameras: high temporal resolution (in the order of microseconds), very high dynamic range (140 dB vs. 60 dB), low power consumption, and high pixel bandwidth (on the order of kHz) resulting in reduced motion blur. Hence, event cameras have a large potential for robotics and computer vision in challenging scenarios for traditional cameras, such as low-latency, high speed, and high dynamic range. However, novel methods are required to process the unconventional output of these sensors in order to unlock their potential. This paper provides a comprehensive overview of the emerging field of event-based vision, with a focus on the applications and the algorithms developed to unlock the outstanding properties of event cameras. We present event cameras from their working principle, the actual sensors that are available and the tasks that they have been used for, from low-level vision (feature detection and tracking, optic flow, etc.) to high-level vision (reconstruction, segmentation, recognition). We also discuss the techniques developed to process events, including learning-based techniques, as well as specialized processors for these novel sensors, such as spiking neural networks. Additionally, we highlight the challenges that remain to be tackled and the opportunities that lie ahead in the search for a more efficient, bio-inspired way for machines to perceive and interact with the world

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

Light-sheet microscopy: a tutorial

This paper is intended to give a comprehensive review of light-sheet (LS) microscopy from an optics perspective. As such, emphasis is placed on the advantages that LS microscope configurations present, given the degree of freedom gained by uncoupling the excitation and detection arms. The new imaging properties are first highlighted in terms of optical parameters and how these have enabled several biomedical applications. Then, the basics are presented for understanding how a LS microscope works. This is followed by a presentation of a tutorial for LS microscope designs, each working at different resolutions and for different applications. Then, based on a numerical Fourier analysis and given the multiple possibilities for generating the LS in the microscope (using Gaussian, Bessel, and Airy beams in the linear and nonlinear regimes), a systematic comparison of their optical performance is presented. Finally, based on advances in optics and photonics, the novel optical implementations possible in a LS microscope are highlighted.Peer ReviewedPostprint (published version

Plasmonic Optical Sensors: Performance Analysis and Engineering Towards Biosensing

Surface plasmon resonance (SPR) sensing for quantitative analysis of chemical reactions and biological interactions has become one of the most promising applications of plasmonics. This thesis focuses on performance analysis for plasmonic sensors and implementation of plamonic optical sensors with novel nanofabrication techniques. A universal performance analysis model is established for general two-dimensional plasmonic sensors. This model is based on the fundamental facts of surface plasmon theory. The sensitivity only depends on excitation light wavelength as well as dielectric properties of metal and dielectrics. The expression involves no structure-specified parameters, which validates this formula in broad cases of periodic, quasiperiodic and aperiodic nanostructures. Further analysis reveals the intrinsic relationship between plamonic sensor performance and essential physics of surface plasmon. The analytical results are compared to the sensitivities of previously reported plasmonic sensors in the field. This universal model is a promising qualification criterion for plasmonic sensors. Plasmonic optical sensors are engineered into high-performance on-chip sensors, plasmonic optical fibers and freestanding nanomembranes. (1) Periodic nanohole arrays are patterned on chip by a simple and robust template-transfer approach. A spectral analysis approach is also developed for improving the sensor performance. This sensor is applied to demonstrate the on-chip detection of cardiac troponin-I. (2) Plasmonic optical fibers are constructed by transferring periodic metal nanostructures from patterned templates onto endfaces of optical fibers using an epoxy adhesive. Patterned metal structures are generally extended from nanohole arrays to nanoslit arrays. A special plasmonic fiber is designed to simultaneously implement multimode refractive index sensing with remarkably narrow linewidth and high figure of merit. A real-time immunoassay relying on plasmonic fiber is demonstrated. Plasmonic optical fibers also take advantages of consistent optical responses, excellent stability during fiber bending and capability of spectrum filtering. (3) Large-area freestanding metal nanomembranes are implemented using a novel fabrication approach. The formed transferrable membranes feature high-quality and uniform periodic nanohole arrays. The freestanding nanomembranes exhibit remarkably higher transmission intensity in comparison to the nanohole arrays with same features on the substrate. These three modalities of plasmonic sensors possess different applicability to fulfill various plasmonic sensing tasks in respective scenarios

Remote Sensing

This dual conception of remote sensing brought us to the idea of preparing two different books; in addition to the first book which displays recent advances in remote sensing applications, this book is devoted to new techniques for data processing, sensors and platforms. We do not intend this book to cover all aspects of remote sensing techniques and platforms, since it would be an impossible task for a single volume. Instead, we have collected a number of high-quality, original and representative contributions in those areas