High-speed in vitro intensity diffraction tomography

We demonstrate a label-free, scan-free intensity diffraction tomography technique utilizing annular illumination (aIDT) to rapidly characterize large-volume three-dimensional (3-D) refractive index distributions in vitro. By optimally matching the illumination geometry to the microscope pupil, our technique reduces the data requirement by 60 times to achieve high-speed 10-Hz volume rates. Using eight intensity images, we recover volumes of ∼350 μm  ×  100 μm  ×  20  μm, with near diffraction-limited lateral resolution of   ∼  487  nm and axial resolution of   ∼  3.4  μm. The attained large volume rate and high-resolution enable 3-D quantitative phase imaging of complex living biological samples across multiple length scales. We demonstrate aIDT’s capabilities on unicellular diatom microalgae, epithelial buccal cell clusters with native bacteria, and live Caenorhabditis elegans specimens. Within these samples, we recover macroscale cellular structures, subcellular organelles, and dynamic micro-organism tissues with minimal motion artifacts. Quantifying such features has significant utility in oncology, immunology, and cellular pathophysiology, where these morphological features are evaluated for changes in the presence of disease, parasites, and new drug treatments. Finally, we simulate the aIDT system to highlight the accuracy and sensitivity of the proposed technique. aIDT shows promise as a powerful high-speed, label-free computational microscopy approach for applications where natural imaging is required to evaluate environmental effects on a sample in real time.https://arxiv.org/abs/1904.06004Accepted manuscrip

Superresolution fluorescence microscopy with structured illumination

Cotutela Universitat Politècnica de Catalunya i Aix-Marseille UniversitéThe resolution of a conventional fluorescence microscope image is diffraction limited which achieves a spatial resolution of 200nm lateral and 500nm axial. Recently, many superresolution fluorescence microscopy techniques have been developed which allow the observation of many biological structures beyond the diffraction limit. Structured illumination microscopy (SIM) is one of them. The principle of SIM is based on using a harmonic light grid which down modulates the high spatial frequencies of the sample into the observable region of the microscope. The resolution enhancement is highly dependent on the reconstruction technique, which restores the high spatial frequencies of the sample to their original position. Common SIM reconstructions require the perfect knowledge of the illumination pattern. However, to perfectly control the harmonic illumination patterns on the sample plane is not easy in experimental implementations and this makes the experimental setup very technical. Reconstructing SIM images assuming the perfect knowledge of the illumination intensity patterns may, therefore, introduce artifacts on the estimated sample due to the misalignment of the grid that can occur during experimental acquisitions. To tackle this drawback of SIM, in this these, we have developed blind-SIM reconstruction strategies which are independent of the illumination patterns. Using the 3D blind-SIM reconstruction strategies we extended the harmonic SIM to speckle illumination microscopy which uses random unknown speckle patterns that need no control, unlike the harmonic grid patterns. For harmonic-SIM images, since incorporating some information about illumination patterns is valuable, we have developed a 3D positive filtered blind-SIM reconstruction which confines the iterative estimation of the illuminations in the vicinity of the Fourier peaks (using carefully designed Fourier filter masks) in the Fourier space. Using blind-SIM reconstruction techniques a lateral resolution of about 100nm and axial resolution of about 200nm is obtained in both speckle and harmonic SIM. In addition, to reduce the out-of-focus problem in widefield images, a simple computational technique which is based on reconstructing 2D data with 3D PSF is developed based on blind-SIM reconstruction. Moreover, to combine the functionalities of SIM and light sheet microscopy, as a proof of concept, we have developed a simple microscope setup which produces a structured light sheet illumination pattern.La microscopie de fluorescence optique est l’un des outils les plus puissants pour étudier les structures cellulaires et moléculaires au niveau subcellulaire. La résolution d’une image de microscope conventionnel à fluorescence est limitée par la diffraction, ce qui permet d’obtenir une résolution spatiale latérale de 200nm et axiale de 500nm. Récemment, de nombreuses techniques de microscopie de fluorescence de super-résolution ont été développées pour permettre d’observer de nombreuses structures biologiques au-delà de la limite de diffraction. La microscopie d’illumination structurée (SIM) est l’une de ces technologies. Le principe de la SIM est basé sur l’utilisation d’une grille de lumière harmonique qui permet de translater les hautes fréquences spatiales de l’échantillon vers la région d’observation du microscope. L’amélioration de la résolution de cette technologie de microscopie dépend fortement de la technique de reconstruction, qui rétablit les hautes fréquences spatiales de l’échantillon dans leur position d’origine. Les méthodes classiques de reconstruction SIM nécessitent une connaissance parfaite de l’illumination de l’échantillon. Cependant, l’implémentation d’un contrôle parfait de l’illumination harmonique sur le plan de l’échantillon n’est pas facile expérimentalement et il présente un grand défi. L’hypothèse de la connaissance parfaite de l’intensité de la lumière illuminant l’échantillon en SIM peut donc introduire des artefacts sur l’image reconstruite de l’échantillon, à cause des erreurs d’alignement de la grille qui peuvent se présenter lors de l’acquisition expérimentale. Afin de surmonter ce défi, nous avons développé dans cette thèse des stratégies de reconstruction «aveugle» qui sont indépendantes de d’illumination. À l’aide de ces stratégies de reconstruction dites «blind-SIM», nous avons étendu la SIM harmonique pour l’appliquer aux cas de «SIM-speckle» qui utilisent des illuminations aléatoires et inconnues qui contrairement à l’illumination harmonique, ne nécessitent pas de controle. Comme il est utile de récupérer des informations sur l’illumination en SIM harmonique, nous avons développé une reconstruction blind-SIM tridimensionnel et filtrée qui confine l’estimation itérative des illuminations au voisinage des pics dans l’espace de Fourier, en utilisant des masques de filtre de Fourier soigneusement conçus. En utilisant des techniques de reconstruction blind-SIM, une résolution latérale d’environ 100 nm et une résolution axiale d’environ 200 nm sont obtenues, à la fois en SIM harmonique et en SIM speckle. En outre, pour réduire le problème de focalisation dans les images de champ large, une technique de calcul simple qui repose sur la reconstruction bidimensionnel de données à partir de PSF tridimensionnel est développée. En outre, afin de combiner à la fois les fonctionnalités de la SIM et de la microscopie á nappe de lumière, en tant que preuve de concept, nous avons développé une configuration de microscope simple qui produit une nappe de lumière structuréePostprint (published version

Superresolution fluorescence microscopy with structured illumination

The resolution of a conventional fluorescence microscope image is diffraction limited which achieves a spatial resolution of 200nm lateral and 500nm axial. Recently, many superresolution fluorescence microscopy techniques have been developed which allow the observation of many biological structures beyond the diffraction limit. Structured illumination microscopy (SIM) is one of them. The principle of SIM is based on using a harmonic light grid which down modulates the high spatial frequencies of the sample into the observable region of the microscope. The resolution enhancement is highly dependent on the reconstruction technique, which restores the high spatial frequencies of the sample to their original position. Common SIM reconstructions require the perfect knowledge of the illumination pattern. However, to perfectly control the harmonic illumination patterns on the sample plane is not easy in experimental implementations and this makes the experimental setup very technical. Reconstructing SIM images assuming the perfect knowledge of the illumination intensity patterns may, therefore, introduce artifacts on the estimated sample due to the misalignment of the grid that can occur during experimental acquisitions. To tackle this drawback of SIM, in this these, we have developed blind-SIM reconstruction strategies which are independent of the illumination patterns. Using the 3D blind-SIM reconstruction strategies we extended the harmonic SIM to speckle illumination microscopy which uses random unknown speckle patterns that need no control, unlike the harmonic grid patterns. For harmonic-SIM images, since incorporating some information about illumination patterns is valuable, we have developed a 3D positive filtered blind-SIM reconstruction which confines the iterative estimation of the illuminations in the vicinity of the Fourier peaks (using carefully designed Fourier filter masks) in the Fourier space. Using blind-SIM reconstruction techniques a lateral resolution of about 100nm and axial resolution of about 200nm is obtained in both speckle and harmonic SIM. In addition, to reduce the out-of-focus problem in widefield images, a simple computational technique which is based on reconstructing 2D data with 3D PSF is developed based on blind-SIM reconstruction. Moreover, to combine the functionalities of SIM and light sheet microscopy, as a proof of concept, we have developed a simple microscope setup which produces a structured light sheet illumination pattern.La microscopie de fluorescence optique est l’un des outils les plus puissants pour étudier les structures cellulaires et moléculaires au niveau subcellulaire. La résolution d’une image de microscope conventionnel à fluorescence est limitée par la diffraction, ce qui permet d’obtenir une résolution spatiale latérale de 200nm et axiale de 500nm. Récemment, de nombreuses techniques de microscopie de fluorescence de super-résolution ont été développées pour permettre d’observer de nombreuses structures biologiques au-delà de la limite de diffraction. La microscopie d’illumination structurée (SIM) est l’une de ces technologies. Le principe de la SIM est basé sur l’utilisation d’une grille de lumière harmonique qui permet de translater les hautes fréquences spatiales de l’échantillon vers la région d’observation du microscope. L’amélioration de la résolution de cette technologie de microscopie dépend fortement de la technique de reconstruction, qui rétablit les hautes fréquences spatiales de l’échantillon dans leur position d’origine. Les méthodes classiques de reconstruction SIM nécessitent une connaissance parfaite de l’illumination de l’échantillon. Cependant, l’implémentation d’un contrôle parfait de l’illumination harmonique sur le plan de l’échantillon n’est pas facile expérimentalement et il présente un grand défi. L’hypothèse de la connaissance parfaite de l’intensité de la lumière illuminant l’échantillon en SIM peut donc introduire des artefacts sur l’image reconstruite de l’échantillon, à cause des erreurs d’alignement de la grille qui peuvent se présenter lors de l’acquisition expérimentale. Afin de surmonter ce défi, nous avons développé dans cette thèse des stratégies de reconstruction «aveugle» qui sont indépendantes de d’illumination. À l’aide de ces stratégies de reconstruction dites «blind-SIM», nous avons étendu la SIM harmonique pour l’appliquer aux cas de «SIM-speckle» qui utilisent des illuminations aléatoires et inconnues qui contrairement à l’illumination harmonique, ne nécessitent pas de controle. Comme il est utile de récupérer des informations sur l’illumination en SIM harmonique, nous avons développé une reconstruction blind-SIM tridimensionnel et filtrée qui confine l’estimation itérative des illuminations au voisinage des pics dans l’espace de Fourier, en utilisant des masques de filtre de Fourier soigneusement conçus. En utilisant des techniques de reconstruction blind-SIM, une résolution latérale d’environ 100 nm et une résolution axiale d’environ 200 nm sont obtenues, à la fois en SIM harmonique et en SIM speckle. En outre, pour réduire le problème de focalisation dans les images de champ large, une technique de calcul simple qui repose sur la reconstruction bidimensionnel de données à partir de PSF tridimensionnel est développée. En outre, afin de combiner à la fois les fonctionnalités de la SIM et de la microscopie á nappe de lumière, en tant que preuve de concept, nous avons développé une configuration de microscope simple qui produit une nappe de lumière structuré

Semi-Blind Spatially-Variant Deconvolution in Optical Microscopy with Local Point Spread Function Estimation By Use Of Convolutional Neural Networks

We present a semi-blind, spatially-variant deconvolution technique aimed at optical microscopy that combines a local estimation step of the point spread function (PSF) and deconvolution using a spatially variant, regularized Richardson-Lucy algorithm. To find the local PSF map in a computationally tractable way, we train a convolutional neural network to perform regression of an optical parametric model on synthetically blurred image patches. We deconvolved both synthetic and experimentally-acquired data, and achieved an improvement of image SNR of 1.00 dB on average, compared to other deconvolution algorithms.Comment: 2018/02/11: submitted to IEEE ICIP 2018 - 2018/05/04: accepted to IEEE ICIP 201

Phase control and measurement in digital microscopy

The ongoing merger of the digital and optical components of the modern microscope is creating opportunities for new measurement techniques, along with new challenges for optical modelling. This thesis investigates several such opportunities and challenges which are particularly relevant to biomedical imaging. Fourier optics is used throughout the thesis as the underlying conceptual model, with a particular emphasis on three--dimensional Fourier optics. A new challenge for optical modelling provided by digital microscopy is the relaxation of traditional symmetry constraints on optical design. An extension of optical transfer function theory to deal with arbitrary lens pupil functions is presented in this thesis. This is used to chart the 3D vectorial structure of the spatial frequency spectrum of the intensity in the focal region of a high aperture lens when illuminated by linearly polarised beam. Wavefront coding has been used successfully in paraxial imaging systems to extend the depth of field. This is achieved by controlling the pupil phase with a cubic phase mask, and thereby balancing optical behaviour with digital processing. In this thesis I present a high aperture vectorial model for focusing with a cubic phase mask, and compare it with results calculated using the paraxial approximation. The effect of a refractive index change is also explored. High aperture measurements of the point spread function are reported, along with experimental confirmation of high aperture extended depth of field imaging of a biological specimen. Differential interference contrast is a popular method for imaging phase changes in otherwise transparent biological specimens. In this thesis I report on a new isotropic algorithm for retrieving the phase from differential interference contrast images of the phase gradient, using phase shifting, two directions of shear, and non--iterative Fourier phase integration incorporating a modified spiral phase transform. This method does not assume that the specimen has a constant amplitude. A simulation is presented which demonstrates good agreement between the retrieved phase and the phase of the simulated object, with excellent immunity to imaging noise

Phase control and measurement in digital microscopy

The ongoing merger of the digital and optical components of the modern microscope is creating opportunities for new measurement techniques, along with new challenges for optical modelling. This thesis investigates several such opportunities and challenges which are particularly relevant to biomedical imaging. Fourier optics is used throughout the thesis as the underlying conceptual model, with a particular emphasis on three--dimensional Fourier optics. A new challenge for optical modelling provided by digital microscopy is the relaxation of traditional symmetry constraints on optical design. An extension of optical transfer function theory to deal with arbitrary lens pupil functions is presented in this thesis. This is used to chart the 3D vectorial structure of the spatial frequency spectrum of the intensity in the focal region of a high aperture lens when illuminated by linearly polarised beam. Wavefront coding has been used successfully in paraxial imaging systems to extend the depth of field. This is achieved by controlling the pupil phase with a cubic phase mask, and thereby balancing optical behaviour with digital processing. In this thesis I present a high aperture vectorial model for focusing with a cubic phase mask, and compare it with results calculated using the paraxial approximation. The effect of a refractive index change is also explored. High aperture measurements of the point spread function are reported, along with experimental confirmation of high aperture extended depth of field imaging of a biological specimen. Differential interference contrast is a popular method for imaging phase changes in otherwise transparent biological specimens. In this thesis I report on a new isotropic algorithm for retrieving the phase from differential interference contrast images of the phase gradient, using phase shifting, two directions of shear, and non--iterative Fourier phase integration incorporating a modified spiral phase transform. This method does not assume that the specimen has a constant amplitude. A simulation is presented which demonstrates good agreement between the retrieved phase and the phase of the simulated object, with excellent immunity to imaging noise

Extended Depth of Field for High Resolution Scanning Transmission Electron Microscopy

Aberration-corrected scanning transmission electron microscopes (STEM) provide sub-angstrom lateral resolution; however, the large convergence angle greatly reduces the depth of field. For microscopes with a small depth of field, information outside of the focal plane quickly becomes blurred and less defined. It may not be possible to image some samples entirely in focus. Extended depth-of-field techniques, however, allow a single image, with all areas in-focus, to be extracted from a series of images focused at a range of depths. In recent years, a variety of algorithmic approaches have been employed for bright field optical microscopy. Here, we demonstrate that some established optical microscopy methods can also be applied to extend the ~6 nm depth of focus of a 100 kV 5th-order aberration-corrected STEM (alpha_max = 33 mrad) to image Pt-Co nanoparticles on a thick vulcanized carbon support. These techniques allow us to automatically obtain a single image with all the particles in focus as well as a complimentary topography map.Comment: Accepted, Microscopy and Microanalysi

Light sheet adaptive optics microscope for 3D live imaging

Optical microscopy is still the main research tool for many biological studies. Indeed with the advent of genetic manipulation and specifically, the use of fluorescent protein expressing in animals and plants it has actually seen a renaissance in the past ten years, in particular with the development of novel techniques such as CARS, PALM, STORM, STED and SPIM. In all of microscopy methods one has to look through the sample at some point. The sample thus adds an additional and uncontrolled optical path, which leads to aberrations in the final image. Adaptive optics (AO) is a way of removing these unwanted aberrations which can cause image degradation and even potentially artifacts within the image. This thesis is concerned with the implementation of AO in non scanning microscopes and presents some novel methods both in wavefront sensored and sensorless configurations. A first implementation of AO on the emission path of a light sheet microscope is also presented

Light sheet adaptive optics microscope for 3D live imaging

We report on the incorporation of adaptive optics (AO) into the imaging arm of a selective plane illumination microscope (SPIM). SPIM has recently emerged as an important tool for life science research due to its ability to deliver high-speed, optically sectioned, time-lapse microscope images from deep within in vivo selected samples. SPIM provides a very interesting system for the incorporation of AO as the illumination and imaging paths are decoupled and AO may be useful in both paths. In this paper, we will report the use of AO applied to the imaging path of a SPIM, demonstrating significant improvement in image quality of a live GFP-labeled transgenic zebrafish embryo heart using a modal, wavefront sensorless approach and a heart synchronization method. These experimental results are linked to a computational model showing that significant aberrations are produced by the tube holding the sample in addition to the aberration from the biological sample itself