Adaptive optics with reflected light and deep neural networks

Light scattering and aberrations limit optical microscopy in biological tissue, which motivates the development of adaptive optics techniques. Here, we develop a method for adaptive optics with reflected light and deep neural networks compatible with an epi-detection configuration. Large datasets of sample aberrations which consist of excitation and detection path aberrations as well as the corresponding reflected focus images are generated. These datasets are used for training deep neural networks. After training, these networks can disentangle and independently correct excitation and detection aberrations based on reflected light images recorded from scattering samples. A similar deep learning approach is also demonstrated with scattering guide stars. The predicted aberration corrections are validated using two photon imaging

COMPENSATION THROUGH PREDICTION FOR ATMOSPHERIC TURBULENCE EFFECTS ON TARGET IMAGING AND HIGH ENERGY LASER BEAM

Atmospheric turbulence significantly degrades the performance of High Energy Laser (HEL) beams. The three key undesirable effects are: (1) degraded target images used for target tracking; (2) inaccurate HEL pointing; and (3) reduction in HEL power during propagation to the target. The current approach for compensating for these turbulence effects uses adaptive optics to measure atmospheric turbulence and compensate the aberration in the optical beam. However, an adaptive optics system has limited performance in strong turbulence and an optical system makes the HEL system more complex. With improvements in Deep Learning algorithms and further development in Artificial Intelligence, we used Deep Learning and Convolutional Neural Networks to predict the atmospheric turbulence and compensate for its negative effects on laser beams. The predicted turbulence can be used for image correction and HEL beam correction using a deformable mirror to reduce turbulence effects during propagation.Military Expert 5, Republic of Singapore NavyApproved for public release. Distribution is unlimited

Practical sensorless aberration estimation for 3D microscopy with deep learning

Estimation of optical aberrations from volumetric intensity images is a key step in sensorless adaptive optics for 3D microscopy. Recent approaches based on deep learning promise accurate results at fast processing speeds. However, collecting ground truth microscopy data for training the network is typically very difficult or even impossible thereby limiting this approach in practice. Here, we demonstrate that neural networks trained only on simulated data yield accurate predictions for real experimental images. We validate our approach on simulated and experimental datasets acquired with two different microscopy modalities, and also compare the results to non-learned methods. Additionally, we study the predictability of individual aberrations with respect to their data requirements and find that the symmetry of the wavefront plays a crucial role. Finally, we make our implementation freely available as open source software in Python

Various Applications of Methods and Elements of Adaptive Optics

This volume is focused on a wide range of topics, including adaptive optic components and tools, wavefront sensing, different control algorithms, astronomy, and propagation through turbulent and turbid media

Alignment of electron optical beam shaping elements using a convolutional neural network

A convolutional neural network is used to align an orbital angular momentum sorter in a transmission electron microscope. The method is demonstrated using simulations and experiments. As a result of its accuracy and speed, it offers the possibility of real-time tuning of other electron optical devices and electron beam shaping configurations

On the use of deep learning for phase recovery

Phase recovery (PR) refers to calculating the phase of the light field from its intensity measurements. As exemplified from quantitative phase imaging and coherent diffraction imaging to adaptive optics, PR is essential for reconstructing the refractive index distribution or topography of an object and correcting the aberration of an imaging system. In recent years, deep learning (DL), often implemented through deep neural networks, has provided unprecedented support for computational imaging, leading to more efficient solutions for various PR problems. In this review, we first briefly introduce conventional methods for PR. Then, we review how DL provides support for PR from the following three stages, namely, pre-processing, in-processing, and post-processing. We also review how DL is used in phase image processing. Finally, we summarize the work in DL for PR and outlook on how to better use DL to improve the reliability and efficiency in PR. Furthermore, we present a live-updating resource (https://github.com/kqwang/phase-recovery) for readers to learn more about PR.Comment: 82 pages, 32 figure

Development of advanced control strategies for Adaptive Optics systems

Atmospheric turbulence is a fast disturbance that requires high control frequency. At the same time, celestial objects are faint sources of light and thus WFSs often work in a low photon count regime. These two conditions require a trade-off between high closed-loop control frequency to improve the disturbance rejection performance, and large WFS exposure time to gather enough photons for the integrated signal to increase the Signal-to-Noise ratio (SNR), making the control a delicate yet fundamental aspect for AO systems. The AO plant and atmospheric turbulence were formalized as state-space linear time-invariant systems. The full AO system model is the ground upon which a model-based control can be designed. A Shack-Hartmann wavefront sensor was used to measure the horizontal atmospheric turbulence. The experimental measurements yielded to the Cn2 atmospheric structure parameter, which is key to describe the turbulence statistics, and the Zernike terms time-series. Experimental validation shows that the centroid extraction algorithm implemented on the Jetson GPU outperforms (i.e. is faster) than the CPU implementation on the same hardware. In fact, due to the construction of the Shack-Hartmann wavefront sensor, the intensity image captured from its camera is partitioned into several sub-images, each related to a point of the incoming wavefront. Such sub-images are independent each-other and can be computed concurrently. The AO model is exploited to automatically design an advanced linear-quadratic Gaussian controller with integral action. Experimental evidence shows that the system augmentation approach outperforms the simple integrator and the integrator filtered with the Kalman predictor, and that it requires less parameters to tune

Design of a portable observatory control system

In this thesis, we synthesize the development of a new concept of operation of small robotic telescopes operated over the Internet. Our design includes a set of improvements in control algorithmic and hardware of several critical points of the list of subsystems necessary to obtain suitable data from a telescope. We can synthesize the principal contributions of this thesis into five independent innovations: - An advanced drive closed-loop control: We designed an innovative hardware and software solution for controlling a telescope position at high precision and high robustness. - A complete Telescope Control System (TCS): We implemented a light and portable software using advanced astronomical algorithms libraries for optimally compute in real-time the telescope positioning. This software also provides a new multiple simultaneous pointing models system using state machines which allows reaching higher pointing precision and longer exposure times with external guiding telescopes. - A distributed software architecture (CoolObs): CoolObs is the implementation of a ZeroC-ICE framework allowing the control, interaction, and communication of all the peripherals present in an astronomical observatory. - A patented system for dynamic collimation of optics: SAPACAN is a mechanical parallel arrangement and its associated software used for active compensation of low-frequency aberration variations in small telescopes. - Collimation estimation algorithms: A sensor-less AO algorithm have been applied by the analysis of images obtained with the field camera. This algorithm can detect effects of lousy collimation. The measured misalignments can later feed corrections to a device like SAPACAN. Due to the constant presence of new technologies in the field of astronomy, it had been one of the first fields to introduce material which was not democratized at this time such as Coupled Charged Devices, internet, adaptive optics, remote and robotic control of devices. However, every time one of these new technologies was included in the field it was necessary to design software protocol according to the epoch’s state of the art software. Then with the democratization of the same devices, years after the definition of their protocols, the same communication rules tend to be used to keep backward compatibility with old - and progressively unused- devices. When using lots of cumulated software knowledge such as with robotic observing, we can dig in several nonsenses in the commonly used architectures due to the previously explained reasons. The described situation is the reason why we will propose as follows a new concept of considering an observatory as an entity and not a separated list of independent peripherals. We will describe the application of this concept in the field or robotic telescopes and implement it in various completely different examples to show its versatility and robustness. First of all, we will give a short introduction of the astronomical concepts which will be used all along the document, in a second part, we will expose a state of the art of the current solutions used in the different subsystems of an observing facility and explain why they fail in being used in small telescopes. The principal section will be dedicated to detail and explain each of the five innovations enumerated previously, and finally, we will present the fabrication and integration of these solutions. We will show here how the joint use of all of them allowed obtaining satisfactory outstanding results in the robotic use of a new prototype and on the adaptation on several existing refurbished telescopes. Finally, we dedicate the last chapter of this thesis to resuming the conclusions of our work.En esta tesis, presentamos el desarrollo de un nuevo concepto de operación de telescopio robótica operados a través de Internet. Nuestro diseño incluye un conjunto de mejoras de los algoritmos de control y hardware de varios puntos críticos de la lista de subsistemas necesarios para obtener datos de calidad científica con un telescopio. Podemos sintetizar las principales contribuciones de esta tesis en cinco innovaciones independientes: - Un control de motor avanzado en bucle cerrado: Diseñamos un hardware y software innovadores para controlar la posición y movimiento fino de un telescopio con alta precisión y alta robustez. - Un software de control de telescopio (TCS) integrado: Implementamos un software ligero y portátil que ocupa bibliotecas de algoritmos astronómicos avanzados para calcular de manera óptima y en tiempo real la posición teórica del telescopio. Este software también proporciona un software innovador de modelo de pointing múltiples simultáneos. Esto permite alcanzar una mayor precisión de seguimiento y así ocupar tiempos de integración más importante ocupando un telescopio de guía mecánicamente apartado al telescopio principal. - Una arquitectura de software distribuido (CoolObs): CoolObs es una implementación de software ocupando la plataforma de desarrollo ZeroC-ICE la cual permite el control, la interacción y la comunicación de todos los periféricos presentes en un observatorio astronómico. - Un sistema patentado para la colimación dinámica de la óptica: SAPACAN es un sistema mecánico de movimiento paralelo y su software asociado. Se puede ocupar para compensar activamente las aberraciones ópticas de bajo orden en pequeños telescopios. - Algoritmos de estimación de colimación: Se desarrolló un algoritmo de óptica adaptiva sin sensor en base al análisis de imágenes obtenidas con una cámara cerca del plano focal del telescopio. Este algoritmo puede detectar efectos de mala colimación de las ópticas. Los desajustes, una vez medidos, pueden posteriormente ser aplicados como correcciones a un dispositivo como SAPACAN. Astronomía es un terreno propicio al desarrollo de nuevas tecnologías y, debido a esto, los protocolos de comunicación entre periféricos pueden ser obsoletos porque se han escritos en etapas tempranas de existencia de estas nuevas tecnologías. Las mejoras se han hecho de a poco para mantener la compatibilidad de los sistemas ya existentes, ocupando un planteamiento general de la problemática de control de telescopios robóticos, proponemos un nuevo concepto de observatorio robótico visto como una entidad y no una lista de periféricos independientes. A lo largo de esta tesis, describiremos la aplicación de este concepto en el campo de telescopios robóticos e implementarlo en varios sistemas independientes y variados para mostrar la versatilidad y robustez de la propuesta.Postprint (published version

Roadmap on digital holography [Invited]

This Roadmap article on digital holography provides an overview of a vast array of research activities in the field of digital holography. The paper consists of a series of 25 sections from the prominent experts in digital holography presenting various aspects of the field on sensing, 3D imaging and displays, virtual and augmented reality, microscopy, cell identification, tomography, label-free live cell imaging, and other applications. Each section represents the vision of its author to describe the significant progress, potential impact, important developments, and challenging issues in the field of digital holography