Untrained, physics-informed neural networks for structured illumination microscopy

In recent years there has been great interest in using deep neural networks (DNN) for super-resolution image reconstruction including for structured illumination microscopy (SIM). While these methods have shown very promising results, they all rely on data-driven, supervised training strategies that need a large number of ground truth images, which is experimentally difficult to realize. For SIM imaging, there exists a need for a flexible, general, and open-source reconstruction method that can be readily adapted to different forms of structured illumination. We demonstrate that we can combine a deep neural network with the forward model of the structured illumination process to reconstruct sub-diffraction images without training data. The resulting physics-informed neural network (PINN) can be optimized on a single set of diffraction limited sub-images and thus doesn't require any training set. We show with simulated and experimental data that this PINN can be applied to a wide variety of SIM methods by simply changing the known illumination patterns used in the loss function and can achieve resolution improvements that match well with theoretical expectations

Supplementary document for Untrained, physics-informed neural networks for structured illumination microscopy - 6092239.pdf

supplemental information documen

Iterative Machine Learning for Classification and Discovery of Single-Molecule Unfolding Trajectories from Force Spectroscopy Data

We report the application of machine learning techniques to expedite classification and analysis of protein unfolding trajectories from force spectroscopy data. Using kernel methods, logistic regression, and triplet loss, we developed a workflow called Forced Unfolding and Supervised Iterative Online (FUSION) learning where a user classifies a small number of repeatable unfolding patterns encoded as images, and a machine is tasked with identifying similar images to classify the remaining data. We tested the workflow using two case studies on a multidomain XMod-Dockerin/Cohesin complex, validating the approach first using synthetic data generated with a Monte Carlo algorithm and then deploying the method on experimental atomic force spectroscopy data. FUSION efficiently separated traces that passed quality filters from unusable ones, classified curves with high accuracy, and identified unfolding pathways that were undetected by the user. This study demonstrates the potential of machine learning to accelerate data analysis and generate new insights in protein biophysics

Super-Resolution Imaging by Random Adsorbed Molecule Probes

Single molecule localization (SML) is a powerful tool to measure the position and trajectory of molecules in numerous systems, with nanometer accuracy. This technique has been recently utilized to overcome the diffraction limit in optical imaging. So far, super-resolution imaging by SML was demonstrated using photoactivable or photoswitchable fluorophores, as well as diffusive fluorophore probes in solution. All these methods, however, rely on special fluorophore or object properties. In this Letter, we propose and demonstrate a new super-resolution technique attainable for a bio/dielectric structure on a metal substrate. A sub-diffraction-limited image is obtained by randomly adsorbed fluorescent probe molecules on a liquid–solid interface, while the metal substrate, quenching the unwanted fluorescent signal, provides a significantly enhanced imaging contrast. As this approach does not use specific stain techniques, it can be readily applied to general dielectric objects, such as nanopatterned photoresist, inorganic nanowires, subcellular structures, etc

Supplementary document for Inverse design of metasurface based off-axis image relay - 6831828.pdf

Supplementary informatio

Coherent Four-Fold Super-Resolution Imaging with Composite Photonic–Plasmonic Structured Illumination

We present a far-field super-resolution imaging scheme based on coherent scattering under a composite photonic–plasmonic structured illumination. The super-resolved image retrieval method, which involves the combination of 13 different diffraction-limited images of the specimen, is first developed within a Fourier optics framework. A feasible implementation of this optical microscopy technique working at 465 nm is proposed and its point spread function is investigated using full electromagnetics calculations. The 4-fold super-resolution power of the scheme, able to resolve 60 nm feature sizes at the operating wavelength, is demonstrated against both Abbe’s (imaging a single object) and Rayleigh’s (imaging two closely spaced objects) criteria

High Spatiotemporal Resolution Imaging with Localized Plasmonic Structured Illumination Microscopy

Localized plasmonic structured illumination microscopy (LPSIM) provides multicolor wide-field super-resolution imaging with low phototoxicity and high-speed capability. LPSIM utilizes a nanoscale plasmonic antenna array to provide a series of tunable illumination patterns beyond the traditional diffraction limit, allowing for enhanced resolving powers down to a few tens of nanometers. Here, we demonstrate wide-field LPSIM with 50 nm spatial resolution at video rate speed by imaging microtubule dynamics with low illumination power intensity. The design of the LPSIM system makes it suitable for imaging surface effects of cells and tissues with regular sample preparation protocols. LPSIM can be extended to much higher resolution, representing an excellent technology for live-cell imaging of protein dynamics and interactions

Supplementary document for Visible and near-infrared dual-band switchable metasurface-enabled edge imaging - 5953079.pdf

Supplementary infomatio

High Spatiotemporal Resolution Imaging with Localized Plasmonic Structured Illumination Microscopy

Localized plasmonic structured illumination microscopy (LPSIM) provides multicolor wide-field super-resolution imaging with low phototoxicity and high-speed capability. LPSIM utilizes a nanoscale plasmonic antenna array to provide a series of tunable illumination patterns beyond the traditional diffraction limit, allowing for enhanced resolving powers down to a few tens of nanometers. Here, we demonstrate wide-field LPSIM with 50 nm spatial resolution at video rate speed by imaging microtubule dynamics with low illumination power intensity. The design of the LPSIM system makes it suitable for imaging surface effects of cells and tissues with regular sample preparation protocols. LPSIM can be extended to much higher resolution, representing an excellent technology for live-cell imaging of protein dynamics and interactions

Infrared Color-Sorting Metasurface

The process of sorting light based on colors (photon energy) is a prerequisite in broadband optical systems, typically achieved in the form of guiding incoming signals through a sequence of spectral filters. The assembly of filters often leads to lengthy optical trains and consequently, large system footprints. In this work, we address this issue by proposing a flat color-sorting device comprised of a diffraction grating and a dielectric Huygens’ metasurface. Upon the incidence of a broadband beam, the grating disperses the light and redirects wavelengths to a continuous range of angles in accordance with the law of diffraction. Subsequently, the metasurface with multiple paired Huygens’ resonances corrects the dispersion and binds wavelengths to the corresponding waveband with a designated output angle. We demonstrate the efficacy by designing a color-sorting metasurface with two discrete dispersion-compensated outputs (10.8±0.3 μm and 11.9±0.3 μm), based on the proposed approach. The optimized metasurface possesses an overall transmittance exceeding 57% and reduces lateral dispersion by 90% at the output. The proposed color-sorting mechanism provides a solution that benefits the designing of metasurface for miniature multi-band systems