Deep convolutional neural networks and digital holographic microscopy for in-focus depth estimation of microscopic objects

Deep artificial neural network learning is an emerging tool in image analysis. We demonstrate its poten- tial in the field of digital holographic microscopy by addressing the challenging problem of determining the in-focus reconstruction depth of an arbitrary epithelial cell cluster encoded in a digital hologram. A deep convolutional neural network learns the in-focus depths from half a million hologram amplitude images. The trained network correctly determines the in-focus depth of new holograms with high probability, with- out performing numerical propagation. To our knowledge, this is the first application of deep learning in the field of digital holographic microscopy

Development of Microscopy Systems for Super-Resolution, Whole-Slide, Hyperspectral, and Confocal Imaging

Optical microscope is an important tool for researchers to study small objects. In this thesis, we will focus on the improvement of traditional microscope systems from several aspects including resolution, field of view, speed, cost, compactness, multimodality. In particular, we will investigate computational imaging methods that bypass the limitations with traditional microscope systems by combining the optical hardware design and image processing algorithm. Examples will include optimizing illumination strategy for the Fourier ptychography (FP), developing field-portable high-resolution microscope using a cellphone lens, investigating pattern-illuminated FP for fluorescence microscopy, demonstrating multimodal microscopic imaging with the use of liquid crystal display, achieving fast and accurate autofocusing for whole slide imaging system

Fast Autofocusing using Tiny Transformer Networks for Digital Holographic Microscopy

The numerical wavefront backpropagation principle of digital holography confers unique extended focus capabilities, without mechanical displacements along z-axis. However, the determination of the correct focusing distance is a non-trivial and time consuming issue. A deep learning (DL) solution is proposed to cast the autofocusing as a regression problem and tested over both experimental and simulated holograms. Single wavelength digital holograms were recorded by a Digital Holographic Microscope (DHM) with a 10$\mathrm{x}$ microscope objective from a patterned target moving in 3D over an axial range of 92 $\mu$m. Tiny DL models are proposed and compared such as a tiny Vision Transformer (TViT), tiny VGG16 (TVGG) and a tiny Swin-Transfomer (TSwinT). The experiments show that the predicted focusing distance $Z_R^{\mathrm{Pred}}$ is accurately inferred with an accuracy of 1.2 $\mu$m in average in comparison with the DHM depth of field of 15 $\mu$m. Numerical simulations show that all tiny models give the $Z_R^{\mathrm{Pred}}$ with an error below 0.3 $\mu$m. Such a prospect would significantly improve the current capabilities of computer vision position sensing in applications such as 3D microscopy for life sciences or micro-robotics. Moreover, all models reach state of the art inference time on CPU, less than 25 ms per inference

Modeling and applications of the focus cue in conventional digital cameras

El enfoque en cámaras digitales juega un papel fundamental tanto en la calidad de la imagen como en la percepción del entorno. Esta tesis estudia el enfoque en cámaras digitales convencionales, tales como cámaras de móviles, fotográficas, webcams y similares. Una revisión rigurosa de los conceptos teóricos detras del enfoque en cámaras convencionales muestra que, a pasar de su utilidad, el modelo clásico del thin lens presenta muchas limitaciones para aplicación en diferentes problemas relacionados con el foco. En esta tesis, el focus profile es propuesto como una alternativa a conceptos clásicos como la profundidad de campo. Los nuevos conceptos introducidos en esta tesis son aplicados a diferentes problemas relacionados con el foco, tales como la adquisición eficiente de imágenes, estimación de profundidad, integración de elementos perceptuales y fusión de imágenes. Los resultados experimentales muestran la aplicación exitosa de los modelos propuestos.The focus of digital cameras plays a fundamental role in both the quality of the acquired images and the perception of the imaged scene. This thesis studies the focus cue in conventional cameras with focus control, such as cellphone cameras, photography cameras, webcams and the like. A deep review of the theoretical concepts behind focus in conventional cameras reveals that, despite its usefulness, the widely known thin lens model has several limitations for solving different focus-related problems in computer vision. In order to overcome these limitations, the focus profile model is introduced as an alternative to classic concepts, such as the near and far limits of the depth-of-field. The new concepts introduced in this dissertation are exploited for solving diverse focus-related problems, such as efficient image capture, depth estimation, visual cue integration and image fusion. The results obtained through an exhaustive experimental validation demonstrate the applicability of the proposed models

All-optical microscope autofocus based on an electrically tunable lens and a totally internally reflected IR laser

Funding Information: Deutsche Forschungsgemeinschaft (SFB 688); NIH (Prime Grant No. 5 R01 DA038882-02); University of Wuerzburg.Microscopic imaging at high spatial-temporal resolution over long time scales (minutes to hours) requires rapid and precise stabilization of the microscope focus. Conventional and commercial autofocus systems are largely based on piezoelectric stages or mechanical objective actuators. Objective to sample distance is either measured by image analysis approaches or by hardware modules measuring the intensity of reflected infrared light. We propose here a truly all-optical microscope autofocus taking advantage of an electrically tunable lens and a totally internally reflected infrared probe beam. We implement a feedback-loop based on the lateral position of a totally internally reflected infrared laser on a quadrant photodetector, as an indicator of the relative defocus. We show here how to treat the combined contributions due to mechanical defocus and deformation of the tunable lens. As a result, the sample can be kept in focus without any mechanical movement, at rates up to hundreds of Hertz. The device requires only reflective optics and can be implemented at a fraction of the cost required for a comparable piezo-based actuator.Publisher PDFPeer reviewe