Roadmap on digital holography [Invited]

This Roadmap article on digital holography provides an overview of a vast array of research activities in the field of digital holography. The paper consists of a series of 25 sections from the prominent experts in digital holography presenting various aspects of the field on sensing, 3D imaging and displays, virtual and augmented reality, microscopy, cell identification, tomography, label-free live cell imaging, and other applications. Each section represents the vision of its author to describe the significant progress, potential impact, important developments, and challenging issues in the field of digital holography

Automated Three-Dimensional Microbial Sensing and Recognition Using Digital Holography and Statistical Sampling

We overview an approach to providing automated three-dimensional (3D) sensing and recognition of biological micro/nanoorganisms integrating Gabor digital holographic microscopy and statistical sampling methods. For 3D data acquisition of biological specimens, a coherent beam propagates through the specimen and its transversely and longitudinally magnified diffraction pattern observed by the microscope objective is optically recorded with an image sensor array interfaced with a computer. 3D visualization of the biological specimen from the magnified diffraction pattern is accomplished by using the computational Fresnel propagation algorithm. For 3D recognition of the biological specimen, a watershed image segmentation algorithm is applied to automatically remove the unnecessary background parts in the reconstructed holographic image. Statistical estimation and inference algorithms are developed to the automatically segmented holographic image. Overviews of preliminary experimental results illustrate how the holographic image reconstructed from the Gabor digital hologram of biological specimen contains important information for microbial recognition

Digital Holography and Cell Studies

Digital holography microscopy (DHM) has developed into a broad field, and one of all the interesting applications is to study cells without staining, labeling or in any other way affecting them. Both fixed and living, dying or dead cells can be studied. The first DHM images showing living cells were published in 2004 and 2005 (Carl et al. 2004, Marquet et al. 2005), making this field of research rather new. Digital holography makes it possible to easily measure cell properties that previously have been very difficult to study, such as cell thickness and volume (Marquet et al. 2005, Mölder et al. 2008). Two of the major advantages of DHM is the 3-D imaging possibility and measurements over time. Digital holography has ben used to study several types of cells, such as nerve cells, red blood cells, stem cells and cancer cells (Emery et al. 2007, Kemper et al. 2006, Langehanenberg et al. 2009) . It has also been applied for studies of cell proliferation, cell movement, sub-cellular structures and cell morphology (Kemper et al. 2009, Yu et al. 2009). Both 2-D and 3-D cell movement can be determined ( Langehanenberg et al. 2009). Even cell viability status can be determined using DHM. Interestingly, it is possible to study both single cells and entire populations simultaneously, allowing for very nuanced studies. Older, well known techniques often require some degree of cell disturbance such as the fluorescent antibody labeling required for fluorescense or confocal microscopy studies. In this paper we will present some of the studies made possible by DHM. We will compare DHM with previously used techniques and discuss the benefits and drawbacks of digital holography cell measurements

Subsampled phase retrieval for temporal resolution enhancement in lensless on-chip holographic video

On-chip holographic video is a convenient way to monitor biological samples simultaneously at high spatial resolution and over a wide field-of-view. However, due to the limited readout rate of digital detector arrays, one often faces a tradeoff between the per-frame pixel count and frame rate of the captured video. In this report, we propose a subsampled phase retrieval (SPR) algorithm to overcome the spatial-temporal trade-off in holographic video. Compared to traditional phase retrieval approaches, our SPR algorithm uses over an order of magnitude less pixel measurements while maintaining suitable reconstruction quality. We use an on-chip holographic video setup with pixel sub-sampling to experimentally demonstrate a factor of 5.5 increase in sensor frame rate while monitoring the in vivo movement of Peranema microorganisms

Automated red blood cells extraction from holographic images using fully convolutional neural networks

In this paper, we present two models for automatically extracting red blood cells (RBCs) from RBCs holographic images based on a deep learning fully convolutional neural network (FCN) algorithm. The first model, called FCN-1, only uses the FCN algorithm to carry out RBCs prediction, whereas the second model, called FCN-2, combines the FCN approach with the marker-controlled watershed transform segmentation scheme to achieve RBCs extraction. Both models achieve good segmentation accuracy. In addition, the second model has much better performance in terms of cell separation than traditional segmentation methods. In the proposed methods, the RBCs phase images are first numerically reconstructed from RBCs holograms recorded with off-axis digital holographic microscopy. Then, some RBCs phase images are manually segmented and used as training data to fine-tune the FCN. Finally, each pixel in new input RBCs phase images is predicted into either foreground or background using the trained FCN models. The RBCs prediction result from the first model is the final segmentation result, whereas the result from the second model is used as the internal markers of the marker-controlled transform algorithm for further segmentation. Experimental results show that the given schemes can automatically extract RBCs from RBCs phase images and much better RBCs separation results are obtained when the FCN technique is combined with the marker-controlled watershed segmentation algorithm. © 2017 Optical Society of America.1

Multidimensional Optical Sensing and Imaging Systems (MOSIS): From Macro to Micro Scales

Multidimensional optical imaging systems for information processing and visualization technologies have numerous applications in fields such as manufacturing, medical sciences, entertainment, robotics, surveillance, and defense. Among different three-dimensional (3-D) imaging methods, integral imaging is a promising multiperspective sensing and display technique. Compared with other 3-D imaging techniques, integral imaging can capture a scene using an incoherent light source and generate real 3-D images for observation without any special viewing devices. This review paper describes passive multidimensional imaging systems combined with different integral imaging configurations. One example is the integral-imaging-based multidimensional optical sensing and imaging systems (MOSIS), which can be used for 3-D visualization, seeing through obscurations, material inspection, and object recognition from microscales to long range imaging. This system utilizes many degrees of freedom such as time and space multiplexing, depth information, polarimetric, temporal, photon flux and multispectral information based on integral imaging to record and reconstruct the multidimensionally integrated scene. Image fusion may be used to integrate the multidimensional images obtained by polarimetric sensors, multispectral cameras, and various multiplexing techniques. The multidimensional images contain substantially more information compared with two-dimensional (2-D) images or conventional 3-D images. In addition, we present recent progress and applications of 3-D integral imaging including human gesture recognition in the time domain, depth estimation, mid-wave-infrared photon counting, 3-D polarimetric imaging for object shape and material identification, dynamic integral imaging implemented with liquid-crystal devices, and 3-D endoscopy for healthcare applications.B. Javidi wishes to acknowledge support by the National Science Foundation (NSF) under Grant NSF/IIS-1422179, and DARPA and US Army under contract number W911NF-13-1-0485. The work of P. Latorre Carmona, A. Martínez-Uso, J. M. Sotoca and F. Pla was supported by the Spanish Ministry of Economy under the project ESP2013-48458-C4-3-P, and by MICINN under the project MTM2013-48371-C2-2-PDGI, by Generalitat Valenciana under the project PROMETEO-II/2014/062, and by Universitat Jaume I through project P11B2014-09. The work of M. Martínez-Corral and G. Saavedra was supported by the Spanish Ministry of Economy and Competitiveness under the grant DPI2015-66458-C2-1R, and by the Generalitat Valenciana, Spain under the project PROMETEOII/2014/072

Spatially multiplexed interferometric microscopy: from basic principles to advanced arrangements

La posibilidad de visualizar y analizar objetos microscópicos transparentes de una manera no invasiva ha sido uno de los principales retos de la microscopía óptica a lo largo del siglo XX. Para ello, se desarrollaron diversas técnicas de microscopía que convertían las variaciones en el índice de refracción de los objetos en variaciones de intensidad, haciendo estos objetos visibles a simple vista, entre las que destacan la microscopía de contraste de fase de Zernike o de contraste diferencial de Nomarski. Sin embargo, estas técnicas solamente proporcionan información cualitativa del objeto, por lo que su análisis se limita a la simple visualización. Por otro lado, existen otras técnicas de microscopía basadas en la interferometría, que proporcionan información cuantitativa de fase de un modo sencillo y directo. A partir de esta información de fase es posible obtener, de una manera precisa, información sobre la morfología y el índice de refracción del objeto bajo análisis. Este hecho hace que este tipo de técnicas sean muy interesantes en diversas áreas de conocimiento como la medicina, la biofotónica, o la biología, entre otras. Quizás la técnica interferométrica por excelencia para la obtención de imágenes cuantitativas de fase sea la microscopía holográfica digital. La microscopía holográfica digital surge de la combinación de holografía digital y la microscopía óptica. En los últimos años, se han llevado a cabo numerosos avances en el campo de la microscopía holográfica digital con el fin de introducir mejoras en términos de robustez, simplicidad, precisión y coste. En la misma línea de estos avances, esta tesis está centrada en el desarrollo y la mejora de una técnica llamada “microscopía interferométrica por multiplexado espacial”. Esta técnica se basa en la introducción de una serie de modificaciones sencillas en el cuerpo de un microscopio estándar de campo claro, con el objetivo de convertirlo en uno holográfico de una manera muy robusta, sencilla y económica. Todas las modificaciones realizadas están encauzadas a la implementación de un interferómetro de camino común empleando estrategias de multiplexado espacial en el microscopio. Estas modificaciones son principalmente tres: 1) la sustitución de la fuente de iluminación de banda ancha del propio microscopio por una fuente luminosa coherente que permita interferencias; 2) el multiplexado espacial del campo de visión mediante su división en dos o tres regiones para la transmisión de un haz de referencia; y 3) la inserción de un elemento interferométrico, tal como una red de difracción o un cubo divisor de haz, que produzca el patrón interferencial a registrar. Así pues, todas las técnicas desarrolladas en esta tesis están encaminados a la mejora de esta técnica en términos de: 1) ruido coherente, 2) diseño del campo de visión, 3) resolución espacial, 4) capacidad de análisis de objetos no transparentes, 5) caracterización del índice de refracción, y 6) capacidad de análisis a tiempo real. Todas las validaciones experimentales realizadas durante esta tesis demuestran que la técnica de microscopía interferométrica por multiplexado espacial es una técnica muy versátil, potente y económica que permite la obtención de imágenes cuantitativas de fase a partir de un microscopio de campo claro convencional.The possibility of visualizing and analysing transparent microscopic objects in a non-invasively manner was one of the addressed challenges in the microscopy field during 20th century. Several microscopy techniques were created for that purpose, including quantitative phase imaging. Quantitative phase imaging provides numerical information about the morphology and the refractive index of such objects, so that it can be very appealing in diverse fields of knowledge such as medicine, biophotonics or life science, just to cite a few. One of the easiest ways of achieving quantitative phase imaging is employing digital holographic microscopy techniques. Digital holographic microscopy arises from the combination of digital holography and optical microscopy. In recent years, many novel digital holographic microscopy approaches have been successfully developed in order to improve their capabilities in terms of robustness, simplicity, usability, accuracy, and price. In line with that, this thesis is focused on the development and improvement of the technique named "Spatially Multiplexed Interferometric Microscopy". This technique introduces minimal modifications in the embodiment of a conventional bright field microscope in order to convert it into a holographic one in an extremely simple, low-cost and highly-stable way. The modifications are aimed to implement a common-path interferometer by a spatially multiplexed approach in the embodiment of the microscope and are mainly three: 1) the replacement of the broadband illumination source of the microscope by a coherent one; 2) the spatial multiplexed of the input plane by dividing it into two or three regions; 3) and the insertion of an interferometric component such as a diffraction grating or a beam splitter cube. All performed arrangements and phase retrieval procedures are focused on the enhancement of such a technique regarding: 1) coherent noise; 2) spatial multiplexed input plane; 3) spatial resolution; 4) ability for reflective samples analysis; 5) refractive index characterization; and 6) real-time analysis. Experimental validations carried out during the thesis demonstrate that spatially multiplexed interferometric microscopy is a powerful, versatile, and low-cost technique for achieving quantitative phase images from a commercially available standard microscope

Quantitative analysis of three-dimensional morphology and membrane dynamics of red blood cells during temperature elevation

The optimal functionality of red blood cells is closely associated with the surrounding environment. This study was undertaken to analyze the changes in membrane profile, mean corpuscular hemoglobin (MCH), and cell membrane fluctuations (CMF) of healthy red blood cells (RBC) at varying temperatures. The temperature was elevated from 17 °C to 41 °C within a duration of less than one hour, and the holograms were recorded by an off-axis configuration. After hologram reconstruction, we extracted single RBCs and evaluated their morphologically related features (projected surface area and sphericity coefficient), MCH, and CMF. We observed that elevating the temperature results in changes in the three-dimensional (3D) profile. Since CMF amplitude is highly correlated to the bending curvature of RBC membrane, temperature-induced shape changes can alter CMF’s map and amplitude; mainly larger fluctuations appear on dimple area at a higher temperature. Regardless of the shape changes, no alterations in MCH were seen with temperature variation. © 2019, The Author(s).1

3D imaging lipidometry in single cell by in-flow holographic tomography

The most recent discoveries in the biochemical field are highlighting the increasingly important role of lipid droplets (LDs) in several regulatory mechanisms in living cells. LDs are dynamic organelles and therefore their complete characteriza- tion in terms of number, size, spatial positioning and relative distribution in the cell volume can shed light on the roles played by LDs. Until now, fluorescence microscopy and transmission electron microscopy are assessed as the gold standard methods for identifying LDs due to their high sensitivity and specificity. However, such methods generally only provide 2D assays and partial measurements. Furthermore, both can be destructive and with low productivity, thus limit- ing analysis of large cell numbers in a sample. Here we demonstrate for the first time the capability of 3D visualization and the full LD characterization in high-throughput with a tomographic phase-contrast flow-cytometer, by using ovarian cancer cells and monocyte cell lines as models. A strategy for retrieving significant parameters on spatial correlations and LD 3D positioning inside each cell volume is reported. The information gathered by this new method could allow more in depth understanding and lead to new discoveries on how LDs are correlated to cellular functions