Medical Imaging of Microrobots: Toward In Vivo Applications

Medical microrobots (MRs) have been demonstrated for a variety of non-invasive biomedical applications, such as tissue engineering, drug delivery, and assisted fertilization, among others. However, most of these demonstrations have been carried out in in vitro settings and under optical microscopy, being significantly different from the clinical practice. Thus, medical imaging techniques are required for localizing and tracking such tiny therapeutic machines when used in medical-relevant applications. This review aims at analyzing the state of the art of microrobots imaging by critically discussing the potentialities and limitations of the techniques employed in this field. Moreover, the physics and the working principle behind each analyzed imaging strategy, the spatiotemporal resolution, and the penetration depth are thoroughly discussed. The paper deals with the suitability of each imaging technique for tracking single or swarms of MRs and discusses the scenarios where contrast or imaging agent's inclusion is required, either to absorb, emit, or reflect a determined physical signal detected by an external system. Finally, the review highlights the existing challenges and perspective solutions which could be promising for future in vivo applications

Volumetric HiLo microscopy employing an electrically tunable lens

Electrically tunable lenses exhibit strong potential for fast motion-free axial scanning in a variety of microscopes. However, they also lead to a degradation of the achievable resolution because of aberrations and misalignment between illumination and detection optics that are induced by the scan itself. Additionally, the typically nonlinear relation between actuation voltage and axial displacement leads to over- or under-sampled frame acquisition in most microscopic techniques because of their static depth-of-field. To overcome these limitations, we present an Adaptive-Lens-High-and-Low-frequency (AL-HiLo) microscope that enables volumetric measurements employing an electrically tunable lens. By using speckle-patterned illumination, we ensure stability against aberrations of the electrically tunable lens. Its depth-of-field can be adjusted a-posteriori and hence enables to create flexible scans, which compensates for irregular axial measurement positions. The adaptive HiLo microscope provides an axial scanning range of 1 mm with an axial resolution of about 4 μm and sub-micron lateral resolution over the full scanning range. Proof of concept measurements at home-built specimens as well as zebrafish embryos with reporter gene-driven fluorescence in the thyroid gland are shown

Structured illumination 3D microscopy using adaptive lenses and multimode fibers

Microscopic techniques with high spatial and temporal resolution are required for in vivo studying biological cells and tissues. Adaptive lenses exhibit strong potential for fast motion-free axial scanning. However, they also lead to a degradation of the achievable resolution because of aberrations. This hurdle can be overcome by digital optical technologies. We present a novel High-and-Low-frequency (HiLo) 3D-microscope using structured illumination and an adaptive lens. Uniform illumination is used to obtain optical sectioning for the high-frequency (Hi) components of the image, and nonuniform illumination is needed to obtain optical sectioning for the low-frequency (Lo) components of the image. Nonuniform illumination is provided by a multimode fiber. It ensures robustness against optical aberrations of the adaptive lens. The depth-of-field of our microscope can be adjusted a-posteriori by computational optics. It enables to create flexible scans, which compensate for irregular axial measurement positions. The adaptive HiLo 3D-microscope provides an axial scanning range of 1 mm with an axial resolution of about 4 microns and sub-micron lateral resolution over the full scanning range. In result, volumetric measurements with high temporal and spatial resolution are provided. Demonstration measurements of zebrafish embryos with reporter gene-driven fluorescence in the thyroid gland are presented

Complex Wavefront Shaping through a Multi-Core Fiber

Wavefront shaping through a multi-core fiber (MCF) is turning into an attractive method for endoscopic imaging and optical cell-manipulation on a chip. However, the discrete distribution and the low number of cores induce pixelated phase modulation, becoming an obstacle for delivering complex light field distributions through MCFs. We demonstrate a novel phase retrieval algorithm named Core–Gerchberg–Saxton (Core-GS) employing the captured core distribution map to retrieve tailored modulation hologram for the targeted intensity distribution at the distal far-field. Complex light fields are reconstructed through MCFs with high fidelity up to 96.2%. Closed-loop control with experimental feedback denotes the capability of the Core-GS algorithm for precise intensity manipulation of the reconstructed light field. Core-GS provides a robust way for wavefront shaping through MCFs; it facilitates the MCF becoming a vital waveguide in endoscopic and lab-on-a-chip applications

Assignment of Focus Position with Convolutional Neural Networks in Adaptive Lens Based Axial Scanning for Confocal Microscopy

Adaptive lenses offer axial scanning without mechanical translation and thus are promising to replace mechanical-movement-based axial scanning in microscopy. The scan is accomplished by sweeping the applied voltage. However, the relation between the applied voltage and the resulting axial focus position is not unambiguous. Adaptive lenses suffer from hysteresis effects, and their behaviour depends on environmental conditions. This is especially a hurdle when complex adaptive lenses are used that offer additional functionalities and are controlled with more degrees of freedom. In such case, a common approach is to iterate the voltage and monitor the adaptive lens. Here, we introduce an alternative approach which provides a single shot estimation of the current axial focus position by a convolutional neural network. We use the experimental data of our custom confocal microscope for training and validation. This leads to fast scanning without photo bleaching of the sample and opens the door to automatized and aberration-free smart microscopy. Applications in different types of laser-scanning microscopes are possible. However, maybe the training procedure of the neural network must be adapted for some use cases

Transmission Matrix Measurement of Multimode Optical Fibers by Mode-Selective Excitation Using One Spatial Light Modulator

Multimode fibers (MMF) are promising candidates to increase the data rate while reducing the space required for optical fiber networks. However, their use is hampered by mode mixing and other effects, leading to speckled output patterns. This can be overcome by measuring the transmission matrix (TM) of a multimode fiber. In this contribution, a mode-selective excitation of complex amplitudes is performed with only one phase-only spatial light modulator. The light field propagating through the fiber is measured holographically and is analyzed by a rapid decomposition method. This technique requires a small amount of measurements N, which corresponds to the degree of freedom of the fiber. The TM determines the amplitude and phase relationships of the modes, which allows us to understand the mode scrambling processes in the MMF and can be used for mode division multiplexing