Video-rate multi-color structured illumination microscopy with simultaneous real-time reconstruction

Super-resolved structured illumination microscopy (SR-SIM) is among the fastest fluorescence microscopy techniques capable of surpassing the optical diffraction limit. Current custom-build instruments are able to deliver two-fold resolution enhancement with high acquisition speed. SR-SIM is usually a two-step process, with raw-data acquisition and subsequent, time-consuming post-processing for image reconstruction. In contrast, wide-field and (multi-spot) confocal techniques produce high-resolution images instantly. Such immediacy is also possible with SR-SIM, by tight integration of a video-rate capable SIM with fast reconstruction software. Here we present instant SR-SIM by VIGOR (Video-rate Immediate GPU-accelerated Open-Source Reconstruction). We demonstrate multi-color SR-SIM at video frame-rates, with less than 250 ms delay between measurement and reconstructed image display. This is achieved by modifying and extending high-speed SR-SIM image acquisition with a new, GPU-enhanced, network-enabled image-reconstruction software. We demonstrate high-speed surveying of biological samples in multiple colors and live imaging of moving mitochondria as an example of intracellular dynamics

Video-rate multi-color structured illumination microscopy with simultaneous real-time reconstruction

Markwirth A, Lachetta M, Mönkemöller V, et al. Video-rate multi-color structured illumination microscopy with simultaneous real-time reconstruction. Nature Communications. 2019;10(1): 4315.Super-resolved structured illumination microscopy (SR-SIM) is among the fastest fluorescence microscopy techniques capable of surpassing the optical diffraction limit. Current custom-build instruments are able to deliver two-fold resolution enhancement with high acquisition speed. SR-SIM is usually a two-step process, with raw-data acquisition and subsequent, time-consuming post-processing for image reconstruction. In contrast, wide-field and (multi-spot) confocal techniques produce high-resolution images instantly. Such immediacy is also possible with SR-SIM, by tight integration of a video-rate capable SIM with fast reconstruction software. Here we present instant SR-SIM by VIGOR (Video-rate Immediate GPU-accelerated Open-Source Reconstruction). We demonstrate multi-color SR-SIM at video frame-rates, with less than 250ms delay between measurement and reconstructed image display. This is achieved by modifying and extending high-speed SR-SIM image acquisition with a new, GPU-enhanced, network-enabled image-reconstruction software. We demonstrate high-speed surveying of biological samples in multiple colors and live imaging of moving mitochondria as an example of intracellular dynamics

Development and cost-effective implementation of a structured illumination microscope and an optical projection tomography setup

Lachetta M. Development and cost-effective implementation of a structured illumination microscope and an optical projection tomography setup. Bielefeld: Universität Bielefeld; 2021

DMD-based super-resolution structured illumination microscopy visualizes live cell dynamics at high speed and low cost

Sandmeyer A, Lachetta M, Sandmeyer H, Hübner W, Huser T, Müller M. DMD-based super-resolution structured illumination microscopy visualizes live cell dynamics at high speed and low cost. bioRxiv. 2019.Structured illumination microscopy (SIM) is among the most widely used super-resolution fluorescence microscopy techniques for visualizing the dynamics of cellular organelles, such as mitochondria, the endoplasmic reticulum, or the cytoskeleton. In its most wide-spread implementation, SIM relies on the creation of an interference pattern at the diffraction limit using the coherent addition of laser beams created by a diffraction pattern. Spatial light modulators based on liquid crystal displays allow SIM micro-scopes to run at image rates of up to hundreds of super-resolved images per second. Digital micromirror devices are another natural choice for creating interference-based SIM patterns, but are not used to their fullest potential because of the blazed grating effect. This effect arises due to the fixed angles between which the mirrors can be switched, creating a sawtooth arrangement of mirrors and thus leading to a change in the intensity distribution of the diffracted beams. This results in SIM patterns with varying modulation contrast which are prone to reconstruction artifacts. We have carefully studied the blazed grating effect of DMDs by simulations, varying a range of parameters and compared the simulation results with experiments. This allowed us to identify settings which result in very high modulation contrast across all angles and phases required to generate 2-beam SIM pattern. The use of inexpensive industry-grade CMOS cameras as well as low-cost lasers enabled us to construct a cost-effective, high-speed SIM system. Reconstruction of the super-resolved SIM images is achieved on a recently demonstrated parallel-computing platform, which allowed us to visualize living cells with super-resolution at multiple reconstructed frames per second in real time. We demonstrate the versatility of this new platform by imaging cellular organelle dynamics based on live-cell fluorescent stains as well as with fluorescent protein stained samples

Cost-Effective Live Cell Structured Illumination Microscopy with Video-Rate Imaging

Sandmeyer A, Lachetta M, Sandmeyer H, Hübner W, Huser T, Müller M. Cost-Effective Live Cell Structured Illumination Microscopy with Video-Rate Imaging. ACS Photonics. 2021;8(6):1639-1648.Optical nanoscopy is rapidly gaining momentum in the life sciences. Current instruments are, however, often large and expensive, and there is a substantial delay between raw data collection and super-resolved image display. Here, we describe the implementation of a compact, cost-effective, high-speed, structured illumination microscope (SIM), which allows for video-rate super-resolved image reconstructions at imaging rates up to 60 Hz. The instrument is based on a digital micromirror device (DMD) and a global-shutter camera, which enables faster pattern cycles and higher duty cycles than commonly used liquid crystal-based spatial light modulators. In order to utilize a DMD for creating illumination patterns by the coherent superposition of laser beams, we carefully studied its blazed grating effect Through both simulation and experimental determination of system parameters, we identified and optimized its alignment for optimal SIM pattern contrast. Raw image data are collected using inexpensive industry-grade CMOS cameras, while a parallel-computing platform allowed us to reconstruct and visualize living cells in real time. We demonstrate the performance of this system by imaging submicron-sized fluorescent beads diffusing in an aqueous solution, resolving bead-bead interactions in real time. We show that the system is sensitive enough to image intracellular vesicles labeled with fluorescent proteins in fixed cells. We also image dynamic fluctuations of the endoplasmic reticulum (ER), as well as the movement of mitochondria in living osteosarcoma cells, where the cellular organelles are labeled with live cell fluorescent stains

Simulating digital micromirror devices for patterning coherent excitation light in structured illumination microscopy

Lachetta M, Sandmeyer H, Sandmeyer A, Schulte Am Esch J, Huser T, Müller M. Simulating digital micromirror devices for patterning coherent excitation light in structured illumination microscopy. Philosophical transactions. Series A, Mathematical, physical, and engineering sciences. 2021;379(2199).Digital micromirror devices (DMDs) are spatial light modulators that employ the electro-mechanical movement of miniaturized mirrors to steer and thus modulate the light reflected off a mirror array. Their wide availability, low cost and high speed make them a popular choice both in consumer electronics such as video projectors, and scientific applications such as microscopy. High-end fluorescence microscopy systems typically employ laser light sources, which by their nature provide coherent excitation light. In super-resolution microscopy applications that use light modulation, most notably structured illumination microscopy (SIM), the coherent nature of the excitation light becomes a requirement to achieve optimal interference pattern contrast. The universal combination of DMDs and coherent light sources, especially when working with multiple different wavelengths, is unfortunately not straight forward. The substructure of the tilted micromirror array gives rise to a blazed grating, which has to be understood and which must be taken into account when designing a DMD-based illumination system. Here, we present a set of simulation frameworks that explore the use of DMDs in conjunction with coherent light sources, motivated by their application in SIM, but which are generalizable to other light patterning applications. This framework provides all the tools to explore and compute DMD-based diffraction effects and to simulate possible system alignment configurations computationally, which simplifies the system design process and provides guidance for setting up DMD-based microscopes. This article is part of the Theo Murphy meeting 'Super-resolution structured illumination microscopy (part 1)'

Dual color DMD-SIM by temperature-controlled laser wavelength matching

Lachetta M, Wiebusch G, Hübner W, Schulte am Esch J, Huser T, Müller M. Dual color DMD-SIM by temperature-controlled laser wavelength matching. Optics Express. 2021;29(24):39696-39708.Structured illumination microscopy (SIM) is a fast and gentle super-resolution fluorescence imaging technique, featuring live-cell compatible excitation light levels and high imaging speeds. To achieve SIM, spatial modulation of the fluorescence excitation light is employed. This is typically achieved by interfering coherent laser beams in the sample plane, which are often created by spatial light modulators (SLMs). Digital micromirror devices (DMDs) are a form of SLMs with certain advantages, such as high speed, low cost and wide availability, which present certain hurdles in their implementation, mainly the blazed grating effect caused by the jagged surface structure of the tilted mirrors. Recent works have studied this effect through modelling, simulations and experiments, and laid out possible implementations of multi-color SIM imaging based on DMDs. Here, we present an implementation of a dual-color DMD based SIM microscope using temperature-controlled wavelength matching. By carefully controlling the output wavelength of a diode laser by temperature, we can tune two laser wavelengths in such a way that no opto-mechanical realignment of the SIM setup is necessary when switching between both wavelengths. This reduces system complexity and increases imaging speed. With measurements on nano-bead reference samples, as well as the actin skeleton and membrane of fixed U2OS cells, we demonstrate the capabilities of the setup. (C) 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreemen

Multiscale and Multimodal Optical Imaging of the Ultrastructure of Human Liver Biopsies

The liver as the largest organ in the human body is composed of a complex macroscopic and microscopic architecture that supports its indispensable function to maintain physiological homeostasis. Optical imaging of the human liver is particularly challenging because of the need to cover length scales across 7 orders of magnitude (from the centimeter scale to the nanometer scale) in order to fully assess the ultrastructure of the entire organ down to the subcellular scale and probe its physiological function. This task becomes even more challenging the deeper within the organ one hopes to image, because of the strong absorption and scattering of visible light by the liver. Here, we demonstrate how optical imaging methods utilizing highly specific fluorescent labels, as well as label-free optical methods can seamlessly cover this entire size range in excised, fixed human liver tissue and we exemplify this by reconstructing the biliary tree in three-dimensional space. Imaging of tissue beyond approximately 0.5 mm length requires optical clearing of the human liver. We present the successful use of optical projection tomography and light-sheet fluorescence microscopy to derive information about the liver architecture on the millimeter scale. The intermediate size range is covered using label-free structural and chemically sensitive methods, such as second harmonic generation and coherent anti-Stokes Raman scattering microscopy. Laser-scanning confocal microscopy extends the resolution to the nanoscale, allowing us to ultimately image individual liver sinusoidal endothelial cells and their fenestrations by super-resolution structured illumination microscopy. This allowed us to visualize the human hepatobiliary system in 3D down to the cellular level, which indicates that reticular biliary networks communicate with portal bile ducts via single or a few ductuli. Non-linear optical microscopy enabled us to identify fibrotic regions extending from the portal field to the parenchyma, along with microvesicular steatosis in liver biopsies from an older patient. Lastly, super-resolution microscopy allowed us to visualize and determine the size distribution of fenestrations in human liver sinusoidal endothelial cells for the first time under aqueous conditions. Thus, this proof-of-concept study allows us to demonstrate, how, in combination, these techniques open up a new chapter in liver biopsy analysis

Multiscale and Multimodal Optical Imaging of the Ultrastructure of Human Liver Biopsies

Kong C, Bobe S, Pilger C, et al. Multiscale and Multimodal Optical Imaging of the Ultrastructure of Human Liver Biopsies. Frontiers in Physiology. 2021;12: 637136.The liver as the largest organ in the human body is composed of a complex macroscopic and microscopic architecture that supports its indispensable function to maintain physiological homeostasis. Optical imaging of the human liver is particularly challenging because of the need to cover length scales across 7 orders of magnitude (from the centimeter scale to the nanometer scale) in order to fully assess the ultrastructure of the entire organ down to the subcellular scale and probe its physiological function. This task becomes even more challenging the deeper within the organ one hopes to image, because of the strong absorption and scattering of visible light by the liver. Here, we demonstrate how optical imaging methods utilizing highly specific fluorescent labels, as well as label-free optical methods can seamlessly cover this entire size range in excised, fixed human liver tissue and we exemplify this by reconstructing the biliary tree in three-dimensional space. Imaging of tissue beyond approximately 0.5 mm length requires optical clearing of the human liver. We present the successful use of optical projection tomography and light-sheet fluorescence microscopy to derive information about the liver architecture on the millimeter scale. The intermediate size range is covered using label-free structural and chemically sensitive methods, such as second harmonic generation and coherent anti-Stokes Raman scattering microscopy. Laser-scanning confocal microscopy extends the resolution to the nanoscale, allowing us to ultimately image individual liver sinusoidal endothelial cells and their fenestrations by super-resolution structured illumination microscopy. This allowed us to visualize the human hepatobiliary system in 3D down to the cellular level, which indicates that reticular biliary networks communicate with portal bile ducts via single or a few ductuli. Non-linear optical microscopy enabled us to identify fibrotic regions extending from the portal field to the parenchyma, along with microvesicular steatosis in liver biopsies from an older patient. Lastly, super-resolution microscopy allowed us to visualize and determine the size distribution of fenestrations in human liver sinusoidal endothelial cells for the first time under aqueous conditions. Thus, this proof-of-concept study allows us to demonstrate, how, in combination, these techniques open up a new chapter in liver biopsy analysis