Computational methods for 3D imaging of neural activity in light-field microscopy

Light Field Microscopy (LFM) is a 3D imaging technique that captures spatial and angular information from light in a single snapshot. LFM is an appealing technique for applications in biological imaging due to its relatively simple implementation and fast 3D imaging speed. For instance, LFM can help to understand how neurons process information, as shown for functional neuronal calcium imaging. However, traditional volume reconstruction approaches for LFM suffer from low lateral resolution, high computational cost, and reconstruction artifacts near the native object plane. Therefore, in this thesis, we propose computational methods to improve the reconstruction performance of 3D imaging for LFM with applications to imaging neural activity. First, we study the image formation process and propose methods for discretization and simplification of the LF system. Typical approaches for discretization are performed by computing the discrete impulse response at different input locations defined by a sampling grid. Unlike conventional methods, we propose an approach that uses shift-invariant subspaces to generalize the discretization framework used in LFM. Our approach allows the selection of diverse sampling kernels and sampling intervals. Furthermore, the typical discretization method is a particular case of our formulation. Moreover, we propose a description of the system based on filter banks that fit the physics of the system. The periodic-shift invariant property per depth guarantees that the system can be accurately described by using filter banks. This description leads to a novel method to reduce the computational time using singular value decomposition (SVD). Our simplification method capitalizes on the inherent low-rank behaviour of the system. Furthermore, we propose rearranging our filter-bank model into a linear convolution neural network (CNN) that allows more convenient implementation using existing deep-learning software. Then, we study the problem of 3D reconstruction from single light-field images. We propose the shift-invariant-subspace assumption as a prior for volume reconstruction under ideal conditions. We experimentally show that artifact-free reconstruction (aliasing-free) is achievable under these settings. Furthermore, the tools developed to study the forward model are exploited to design a reconstruction algorithm based on ADMM that allows artifact-free 3D reconstruction for real data. Contrary to traditional approaches, our method includes additional priors for reconstruction without dramatically increasing the computational complexity. We extensively evaluate our approach on synthetic and real data and show that our approach performs better than conventional model-based strategies in computational time, image quality, and artifact reduction. Finally, we exploit deep-learning techniques for reconstruction. Specifically, we propose to use two-photon imaging to enhance the performance of LFM when imaging neurons in brain tissues. The architecture of our network is derived from a sparsity-based algorithm for reconstruction named Iterative Shrinkage and Thresholding Algorithm (ISTA). Furthermore, we propose a semi-supervised training based on Generative Adversarial Neural Networks (GANs) that exploits the knowledge of the forward model to achieve remarkable reconstruction quality. We propose efficient architectures to compute the forward model using linear CNNs. This description allows fast computation of the forward model and complements our reconstruction approach. Our method is tested under adverse conditions: lack of training data, background noise, and non-transparent samples. We experimentally show that our method performs better than model-based reconstruction strategies and typical neural networks for imaging neuronal activity in mammalian brain tissue. Our approach enjoys both the robustness of the model-based methods and the reconstruction speed of deep learning.Open Acces

Proceedings of the 2018 Joint Workshop of Fraunhofer IOSB and Institute for Anthropomatics, Vision and Fusion Laboratory

The Proceeding of the annual joint workshop of the Fraunhofer IOSB and the Vision and Fusion Laboratory (IES) 2018 of the KIT contain technical reports of the PhD-stundents on the status of their research. The discussed topics ranging from computer vision and optical metrology to network security and machine learning. This volume provides a comprehensive and up-to-date overview of the research program of the IES Laboratory and the Fraunhofer IOSB

Extended Field of View using Multi Conjugated Deformable Lenses

Aberration correction in an optical system is a critical step in optical design. Nowadays optical design can be consideredd optically perfect and most of the cases where we face aberration or imagedegradation this is due to external causes such as refractive index inhomogeneities of the sample in the case of microscopy, thermal aberrations in the case oh high power beam lasers or air turbulence in the case of astronomy or optical communications. One of the main tool for aberrations correction is Adaptive Optics (AO) that by using a multi actuator deformable mirror in its standard configuration can correct for pupil aberrations. There are some cases however where the aberrations are field dependent. This led to the Multi Conjugated Adaptive Optics (MCAO) implementation where multiple deformable mirrors are used. This configuration tough highly increase system complexity, so in my thesis work I studied for the first time the use of deformable lenses as a replacement for deformable mirrors in the case of a MCAO setup in microscopy or atmospheric turbulence correction.Aberration correction in an optical system is a critical step in optical design. Nowadays optical design can be consideredd optically perfect and most of the cases where we face aberration or imagedegradation this is due to external causes such as refractive index inhomogeneities of the sample in the case of microscopy, thermal aberrations in the case oh high power beam lasers or air turbulence in the case of astronomy or optical communications. One of the main tool for aberrations correction is Adaptive Optics (AO) that by using a multi actuator deformable mirror in its standard configuration can correct for pupil aberrations. There are some cases however where the aberrations are field dependent. This led to the Multi Conjugated Adaptive Optics (MCAO) implementation where multiple deformable mirrors are used. This configuration tough highly increase system complexity, so in my thesis work I studied for the first time the use of deformable lenses as a replacement for deformable mirrors in the case of a MCAO setup in microscopy or atmospheric turbulence correction

Foundations of Software Science and Computation Structures

This open access book constitutes the proceedings of the 23rd International Conference on Foundations of Software Science and Computational Structures, FOSSACS 2020, which took place in Dublin, Ireland, in April 2020, and was held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2020. The 31 regular papers presented in this volume were carefully reviewed and selected from 98 submissions. The papers cover topics such as categorical models and logics; language theory, automata, and games; modal, spatial, and temporal logics; type theory and proof theory; concurrency theory and process calculi; rewriting theory; semantics of programming languages; program analysis, correctness, transformation, and verification; logics of programming; software specification and refinement; models of concurrent, reactive, stochastic, distributed, hybrid, and mobile systems; emerging models of computation; logical aspects of computational complexity; models of software security; and logical foundations of data bases.