GedankenNet: Self-supervised learning of hologram reconstruction using physics consistency

The past decade has witnessed transformative applications of deep learning in various computational imaging, sensing and microscopy tasks. Due to the supervised learning schemes employed, most of these methods depend on large-scale, diverse, and labeled training data. The acquisition and preparation of such training image datasets are often laborious and costly, also leading to biased estimation and limited generalization to new types of samples. Here, we report a self-supervised learning model, termed GedankenNet, that eliminates the need for labeled or experimental training data, and demonstrate its effectiveness and superior generalization on hologram reconstruction tasks. Without prior knowledge about the sample types to be imaged, the self-supervised learning model was trained using a physics-consistency loss and artificial random images that are synthetically generated without any experiments or resemblance to real-world samples. After its self-supervised training, GedankenNet successfully generalized to experimental holograms of various unseen biological samples, reconstructing the phase and amplitude images of different types of objects using experimentally acquired test holograms. Without access to experimental data or the knowledge of real samples of interest or their spatial features, GedankenNet's self-supervised learning achieved complex-valued image reconstructions that are consistent with the Maxwell's equations, meaning that its output inference and object solutions accurately represent the wave propagation in free-space. This self-supervised learning of image reconstruction tasks opens up new opportunities for various inverse problems in holography, microscopy and computational imaging fields.Comment: 30 pages, 6 Figure

Virtual histological staining of unlabeled autopsy tissue

Histological examination is a crucial step in an autopsy; however, the traditional histochemical staining of post-mortem samples faces multiple challenges, including the inferior staining quality due to autolysis caused by delayed fixation of cadaver tissue, as well as the resource-intensive nature of chemical staining procedures covering large tissue areas, which demand substantial labor, cost, and time. These challenges can become more pronounced during global health crises when the availability of histopathology services is limited, resulting in further delays in tissue fixation and more severe staining artifacts. Here, we report the first demonstration of virtual staining of autopsy tissue and show that a trained neural network can rapidly transform autofluorescence images of label-free autopsy tissue sections into brightfield equivalent images that match hematoxylin and eosin (H&E) stained versions of the same samples, eliminating autolysis-induced severe staining artifacts inherent in traditional histochemical staining of autopsied tissue. Our virtual H&E model was trained using >0.7 TB of image data and a data-efficient collaboration scheme that integrates the virtual staining network with an image registration network. The trained model effectively accentuated nuclear, cytoplasmic and extracellular features in new autopsy tissue samples that experienced severe autolysis, such as COVID-19 samples never seen before, where the traditional histochemical staining failed to provide consistent staining quality. This virtual autopsy staining technique can also be extended to necrotic tissue, and can rapidly and cost-effectively generate artifact-free H&E stains despite severe autolysis and cell death, also reducing labor, cost and infrastructure requirements associated with the standard histochemical staining.Comment: 24 Pages, 7 Figure

Roadmap on label-free super-resolution imaging

Label-free super-resolution (LFSR) imaging relies on light-scattering processes in nanoscale objects without a need for fluorescent (FL) staining required in super-resolved FL microscopy. The objectives of this Roadmap are to present a comprehensive vision of the developments, the state-of-the-art in this field, and to discuss the resolution boundaries and hurdles that need to be overcome to break the classical diffraction limit of the label-free imaging. The scope of this Roadmap spans from the advanced interference detection techniques, where the diffraction-limited lateral resolution is combined with unsurpassed axial and temporal resolution, to techniques with true lateral super-resolution capability that are based on understanding resolution as an information science problem, on using novel structured illumination, near-field scanning, and nonlinear optics approaches, and on designing superlenses based on nanoplasmonics, metamaterials, transformation optics, and microsphere-assisted approaches. To this end, this Roadmap brings under the same umbrella researchers from the physics and biomedical optics communities in which such studies have often been developing separately. The ultimate intent of this paper is to create a vision for the current and future developments of LFSR imaging based on its physical mechanisms and to create a great opening for the series of articles in this field

Roadmap on Label-Free Super-resolution Imaging

Label-free super-resolution (LFSR) imaging relies on light-scattering processes in nanoscale objects without a need for fluorescent (FL) staining required in super-resolved FL microscopy. The objectives of this Roadmap are to present a comprehensive vision of the developments, the state-of-the-art in this field, and to discuss the resolution boundaries and hurdles that need to be overcome to break the classical diffraction limit of the label-free imaging. The scope of this Roadmap spans from the advanced interference detection techniques, where the diffraction-limited lateral resolution is combined with unsurpassed axial and temporal resolution, to techniques with true lateral super-resolution capability that are based on understanding resolution as an information science problem, on using novel structured illumination, near-field scanning, and nonlinear optics approaches, and on designing superlenses based on nanoplasmonics, metamaterials, transformation optics, and microsphere-assisted approaches. To this end, this Roadmap brings under the same umbrella researchers from the physics and biomedical optics communities in which such studies have often been developing separately. The ultimate intent of this paper is to create a vision for the current and future developments of LFSR imaging based on its physical mechanisms and to create a great opening for the series of articles in this field.Peer reviewe

Few-shot Transfer Learning for Holographic Image Reconstruction using a Recurrent Neural Network

Deep learning-based methods in computational microscopy have been shown to be powerful but in general face some challenges due to limited generalization to new types of samples and requirements for large and diverse training data. Here, we demonstrate a few-shot transfer learning method that helps a holographic image reconstruction deep neural network rapidly generalize to new types of samples using small datasets. We pre-trained a convolutional recurrent neural network on a large dataset with diverse types of samples, which serves as the backbone model. By fixing the recurrent blocks and transferring the rest of the convolutional blocks of the pre-trained model, we reduced the number of trainable parameters by ~90% compared with standard transfer learning, while achieving equivalent generalization. We validated the effectiveness of this approach by successfully generalizing to new types of samples using small holographic datasets for training, and achieved (i) ~2.5-fold convergence speed acceleration, (ii) ~20% computation time reduction per epoch, and (iii) improved reconstruction performance over baseline network models trained from scratch. This few-shot transfer learning approach can potentially be applied in other microscopic imaging methods, helping to generalize to new types of samples without the need for extensive training time and data.Comment: 10 Pages, 3 Figure

Virtual staining of defocused autofluorescence images of unlabeled tissue using deep neural networks

Deep learning-based virtual staining was developed to introduce image contrast to label-free tissue sections, digitally matching the histological staining, which is time-consuming, labor-intensive, and destructive to tissue. Standard virtual staining requires high autofocusing precision during the whole slide imaging of label-free tissue, which consumes a significant portion of the total imaging time and can lead to tissue photodamage. Here, we introduce a fast virtual staining framework that can stain defocused autofluorescence images of unlabeled tissue, achieving equivalent performance to virtual staining of in-focus label-free images, also saving significant imaging time by lowering the microscope's autofocusing precision. This framework incorporates a virtual-autofocusing neural network to digitally refocus the defocused images and then transforms the refocused images into virtually stained images using a successive network. These cascaded networks form a collaborative inference scheme: the virtual staining model regularizes the virtual-autofocusing network through a style loss during the training. To demonstrate the efficacy of this framework, we trained and blindly tested these networks using human lung tissue. Using 4x fewer focus points with 2x lower focusing precision, we successfully transformed the coarsely-focused autofluorescence images into high-quality virtually stained H&E images, matching the standard virtual staining framework that used finely-focused autofluorescence input images. Without sacrificing the staining quality, this framework decreases the total image acquisition time needed for virtual staining of a label-free whole-slide image (WSI) by ~32%, together with a ~89% decrease in the autofocusing time, and has the potential to eliminate the laborious and costly histochemical staining process in pathology.Comment: 26 Pages, 5 Figure

Table_4_Metagenomics next-generation sequencing for the diagnosis of central nervous system infection: A systematic review and meta-analysis.docx

ObjectiveIt is widely acknowledged that central nervous system (CNS) infection is a serious infectious disease accompanied by various complications. However, the accuracy of current detection methods is limited, leading to delayed diagnosis and treatment. In recent years, metagenomic next-generation sequencing (mNGS) has been increasingly adopted to improve the diagnostic yield. The present study sought to evaluate the value of mNGS in CNS infection diagnosis.MethodsFollowing the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) 2022 guidelines, we searched relevant articles published in seven databases, including PubMed, Web of Science, and Cochrane Library, published from January 2014 to January 2022. High-quality articles related to mNGS applications in the CNS infection diagnosis were included. The comparison between mNGS and the gold standard of CNS infection, such as culture, PCR or serology, and microscopy, was conducted to obtain true positive (TP), true negative (TN), false positive (FP), and false negative (FN) values, which were extracted for sensitivity and specificity calculation.ResultsA total of 272 related studies were retrieved and strictly selected according to the inclusion and exclusion criteria. Finally, 12 studies were included for meta-analysis and the pooled sensitivity was 77% (95% CI: 70–82%, I2 = 39.69%) and specificity was 96% (95% CI: 93–98%, I2 = 72.07%). Although no significant heterogeneity in sensitivity was observed, a sub-group analysis was conducted based on the pathogen, region, age, and sample pretreatment method to ascertain potential confounders. The area under the curve (AUC) of the summary receiver operating characteristic curve (SROC) of mNGS for CNS infection was 0.91 (95% CI: 0.88–0.93). Besides, Deek's Funnel Plot Asymmetry Test indicated no publication bias in the included studies (Figure 3, p > 0.05).ConclusionOverall, mNGS exhibits good sensitivity and specificity for diagnosing CNS infection and diagnostic performance during clinical application by assisting in identifying the pathogen. However, the efficacy remains inconsistent, warranting subsequent studies for further performance improvement during its clinical application.Study registration numberINPLASY202120002</p

Table_3_Metagenomics next-generation sequencing for the diagnosis of central nervous system infection: A systematic review and meta-analysis.DOCX

ObjectiveIt is widely acknowledged that central nervous system (CNS) infection is a serious infectious disease accompanied by various complications. However, the accuracy of current detection methods is limited, leading to delayed diagnosis and treatment. In recent years, metagenomic next-generation sequencing (mNGS) has been increasingly adopted to improve the diagnostic yield. The present study sought to evaluate the value of mNGS in CNS infection diagnosis.MethodsFollowing the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) 2022 guidelines, we searched relevant articles published in seven databases, including PubMed, Web of Science, and Cochrane Library, published from January 2014 to January 2022. High-quality articles related to mNGS applications in the CNS infection diagnosis were included. The comparison between mNGS and the gold standard of CNS infection, such as culture, PCR or serology, and microscopy, was conducted to obtain true positive (TP), true negative (TN), false positive (FP), and false negative (FN) values, which were extracted for sensitivity and specificity calculation.ResultsA total of 272 related studies were retrieved and strictly selected according to the inclusion and exclusion criteria. Finally, 12 studies were included for meta-analysis and the pooled sensitivity was 77% (95% CI: 70–82%, I2 = 39.69%) and specificity was 96% (95% CI: 93–98%, I2 = 72.07%). Although no significant heterogeneity in sensitivity was observed, a sub-group analysis was conducted based on the pathogen, region, age, and sample pretreatment method to ascertain potential confounders. The area under the curve (AUC) of the summary receiver operating characteristic curve (SROC) of mNGS for CNS infection was 0.91 (95% CI: 0.88–0.93). Besides, Deek's Funnel Plot Asymmetry Test indicated no publication bias in the included studies (Figure 3, p > 0.05).ConclusionOverall, mNGS exhibits good sensitivity and specificity for diagnosing CNS infection and diagnostic performance during clinical application by assisting in identifying the pathogen. However, the efficacy remains inconsistent, warranting subsequent studies for further performance improvement during its clinical application.Study registration numberINPLASY202120002</p