20 research outputs found
GedankenNet: Self-supervised learning of hologram reconstruction using physics consistency
The past decade has witnessed transformative applications of deep learning in
various computational imaging, sensing and microscopy tasks. Due to the
supervised learning schemes employed, most of these methods depend on
large-scale, diverse, and labeled training data. The acquisition and
preparation of such training image datasets are often laborious and costly,
also leading to biased estimation and limited generalization to new types of
samples. Here, we report a self-supervised learning model, termed GedankenNet,
that eliminates the need for labeled or experimental training data, and
demonstrate its effectiveness and superior generalization on hologram
reconstruction tasks. Without prior knowledge about the sample types to be
imaged, the self-supervised learning model was trained using a
physics-consistency loss and artificial random images that are synthetically
generated without any experiments or resemblance to real-world samples. After
its self-supervised training, GedankenNet successfully generalized to
experimental holograms of various unseen biological samples, reconstructing the
phase and amplitude images of different types of objects using experimentally
acquired test holograms. Without access to experimental data or the knowledge
of real samples of interest or their spatial features, GedankenNet's
self-supervised learning achieved complex-valued image reconstructions that are
consistent with the Maxwell's equations, meaning that its output inference and
object solutions accurately represent the wave propagation in free-space. This
self-supervised learning of image reconstruction tasks opens up new
opportunities for various inverse problems in holography, microscopy and
computational imaging fields.Comment: 30 pages, 6 Figure
Virtual histological staining of unlabeled autopsy tissue
Histological examination is a crucial step in an autopsy; however, the
traditional histochemical staining of post-mortem samples faces multiple
challenges, including the inferior staining quality due to autolysis caused by
delayed fixation of cadaver tissue, as well as the resource-intensive nature of
chemical staining procedures covering large tissue areas, which demand
substantial labor, cost, and time. These challenges can become more pronounced
during global health crises when the availability of histopathology services is
limited, resulting in further delays in tissue fixation and more severe
staining artifacts. Here, we report the first demonstration of virtual staining
of autopsy tissue and show that a trained neural network can rapidly transform
autofluorescence images of label-free autopsy tissue sections into brightfield
equivalent images that match hematoxylin and eosin (H&E) stained versions of
the same samples, eliminating autolysis-induced severe staining artifacts
inherent in traditional histochemical staining of autopsied tissue. Our virtual
H&E model was trained using >0.7 TB of image data and a data-efficient
collaboration scheme that integrates the virtual staining network with an image
registration network. The trained model effectively accentuated nuclear,
cytoplasmic and extracellular features in new autopsy tissue samples that
experienced severe autolysis, such as COVID-19 samples never seen before, where
the traditional histochemical staining failed to provide consistent staining
quality. This virtual autopsy staining technique can also be extended to
necrotic tissue, and can rapidly and cost-effectively generate artifact-free
H&E stains despite severe autolysis and cell death, also reducing labor, cost
and infrastructure requirements associated with the standard histochemical
staining.Comment: 24 Pages, 7 Figure
Roadmap on label-free super-resolution imaging
Label-free super-resolution (LFSR) imaging relies on light-scattering processes in nanoscale objects without a need for fluorescent (FL) staining required in super-resolved FL microscopy. The objectives of this Roadmap are to present a comprehensive vision of the developments, the state-of-the-art in this field, and to discuss the resolution boundaries and hurdles that need to be overcome to break the classical diffraction limit of the label-free imaging. The scope of this Roadmap spans from the advanced interference detection techniques, where the diffraction-limited lateral resolution is combined with unsurpassed axial and temporal resolution, to techniques with true lateral super-resolution capability that are based on understanding resolution as an information science problem, on using novel structured illumination, near-field scanning, and nonlinear optics approaches, and on designing superlenses based on nanoplasmonics, metamaterials, transformation optics, and microsphere-assisted approaches. To this end, this Roadmap brings under the same umbrella researchers from the physics and biomedical optics communities in which such studies have often been developing separately. The ultimate intent of this paper is to create a vision for the current and future developments of LFSR imaging based on its physical mechanisms and to create a great opening for the series of articles in this field
Roadmap on Label-Free Super-resolution Imaging
Label-free super-resolution (LFSR) imaging relies on light-scattering processes in nanoscale objects without a need for fluorescent (FL) staining required in super-resolved FL microscopy. The objectives of this Roadmap are to present a comprehensive vision of the developments, the state-of-the-art in this field, and to discuss the resolution boundaries and hurdles that need to be overcome to break the classical diffraction limit of the label-free imaging. The scope of this Roadmap spans from the advanced interference detection techniques, where the diffraction-limited lateral resolution is combined with unsurpassed axial and temporal resolution, to techniques with true lateral super-resolution capability that are based on understanding resolution as an information science problem, on using novel structured illumination, near-field scanning, and nonlinear optics approaches, and on designing superlenses based on nanoplasmonics, metamaterials, transformation optics, and microsphere-assisted approaches. To this end, this Roadmap brings under the same umbrella researchers from the physics and biomedical optics communities in which such studies have often been developing separately. The ultimate intent of this paper is to create a vision for the current and future developments of LFSR imaging based on its physical mechanisms and to create a great opening for the series of articles in this field.Peer reviewe
Few-shot Transfer Learning for Holographic Image Reconstruction using a Recurrent Neural Network
Deep learning-based methods in computational microscopy have been shown to be
powerful but in general face some challenges due to limited generalization to
new types of samples and requirements for large and diverse training data.
Here, we demonstrate a few-shot transfer learning method that helps a
holographic image reconstruction deep neural network rapidly generalize to new
types of samples using small datasets. We pre-trained a convolutional recurrent
neural network on a large dataset with diverse types of samples, which serves
as the backbone model. By fixing the recurrent blocks and transferring the rest
of the convolutional blocks of the pre-trained model, we reduced the number of
trainable parameters by ~90% compared with standard transfer learning, while
achieving equivalent generalization. We validated the effectiveness of this
approach by successfully generalizing to new types of samples using small
holographic datasets for training, and achieved (i) ~2.5-fold convergence speed
acceleration, (ii) ~20% computation time reduction per epoch, and (iii)
improved reconstruction performance over baseline network models trained from
scratch. This few-shot transfer learning approach can potentially be applied in
other microscopic imaging methods, helping to generalize to new types of
samples without the need for extensive training time and data.Comment: 10 Pages, 3 Figure
Virtual staining of defocused autofluorescence images of unlabeled tissue using deep neural networks
Deep learning-based virtual staining was developed to introduce image
contrast to label-free tissue sections, digitally matching the histological
staining, which is time-consuming, labor-intensive, and destructive to tissue.
Standard virtual staining requires high autofocusing precision during the whole
slide imaging of label-free tissue, which consumes a significant portion of the
total imaging time and can lead to tissue photodamage. Here, we introduce a
fast virtual staining framework that can stain defocused autofluorescence
images of unlabeled tissue, achieving equivalent performance to virtual
staining of in-focus label-free images, also saving significant imaging time by
lowering the microscope's autofocusing precision. This framework incorporates a
virtual-autofocusing neural network to digitally refocus the defocused images
and then transforms the refocused images into virtually stained images using a
successive network. These cascaded networks form a collaborative inference
scheme: the virtual staining model regularizes the virtual-autofocusing network
through a style loss during the training. To demonstrate the efficacy of this
framework, we trained and blindly tested these networks using human lung
tissue. Using 4x fewer focus points with 2x lower focusing precision, we
successfully transformed the coarsely-focused autofluorescence images into
high-quality virtually stained H&E images, matching the standard virtual
staining framework that used finely-focused autofluorescence input images.
Without sacrificing the staining quality, this framework decreases the total
image acquisition time needed for virtual staining of a label-free whole-slide
image (WSI) by ~32%, together with a ~89% decrease in the autofocusing time,
and has the potential to eliminate the laborious and costly histochemical
staining process in pathology.Comment: 26 Pages, 5 Figure
Table_4_Metagenomics next-generation sequencing for the diagnosis of central nervous system infection: A systematic review and meta-analysis.docx
ObjectiveIt is widely acknowledged that central nervous system (CNS) infection is a serious infectious disease accompanied by various complications. However, the accuracy of current detection methods is limited, leading to delayed diagnosis and treatment. In recent years, metagenomic next-generation sequencing (mNGS) has been increasingly adopted to improve the diagnostic yield. The present study sought to evaluate the value of mNGS in CNS infection diagnosis.MethodsFollowing the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) 2022 guidelines, we searched relevant articles published in seven databases, including PubMed, Web of Science, and Cochrane Library, published from January 2014 to January 2022. High-quality articles related to mNGS applications in the CNS infection diagnosis were included. The comparison between mNGS and the gold standard of CNS infection, such as culture, PCR or serology, and microscopy, was conducted to obtain true positive (TP), true negative (TN), false positive (FP), and false negative (FN) values, which were extracted for sensitivity and specificity calculation.ResultsA total of 272 related studies were retrieved and strictly selected according to the inclusion and exclusion criteria. Finally, 12 studies were included for meta-analysis and the pooled sensitivity was 77% (95% CI: 70â82%, I2 = 39.69%) and specificity was 96% (95% CI: 93â98%, I2 = 72.07%). Although no significant heterogeneity in sensitivity was observed, a sub-group analysis was conducted based on the pathogen, region, age, and sample pretreatment method to ascertain potential confounders. The area under the curve (AUC) of the summary receiver operating characteristic curve (SROC) of mNGS for CNS infection was 0.91 (95% CI: 0.88â0.93). Besides, Deek's Funnel Plot Asymmetry Test indicated no publication bias in the included studies (Figure 3, p > 0.05).ConclusionOverall, mNGS exhibits good sensitivity and specificity for diagnosing CNS infection and diagnostic performance during clinical application by assisting in identifying the pathogen. However, the efficacy remains inconsistent, warranting subsequent studies for further performance improvement during its clinical application.Study registration numberINPLASY202120002</p
Table_3_Metagenomics next-generation sequencing for the diagnosis of central nervous system infection: A systematic review and meta-analysis.DOCX
ObjectiveIt is widely acknowledged that central nervous system (CNS) infection is a serious infectious disease accompanied by various complications. However, the accuracy of current detection methods is limited, leading to delayed diagnosis and treatment. In recent years, metagenomic next-generation sequencing (mNGS) has been increasingly adopted to improve the diagnostic yield. The present study sought to evaluate the value of mNGS in CNS infection diagnosis.MethodsFollowing the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) 2022 guidelines, we searched relevant articles published in seven databases, including PubMed, Web of Science, and Cochrane Library, published from January 2014 to January 2022. High-quality articles related to mNGS applications in the CNS infection diagnosis were included. The comparison between mNGS and the gold standard of CNS infection, such as culture, PCR or serology, and microscopy, was conducted to obtain true positive (TP), true negative (TN), false positive (FP), and false negative (FN) values, which were extracted for sensitivity and specificity calculation.ResultsA total of 272 related studies were retrieved and strictly selected according to the inclusion and exclusion criteria. Finally, 12 studies were included for meta-analysis and the pooled sensitivity was 77% (95% CI: 70â82%, I2 = 39.69%) and specificity was 96% (95% CI: 93â98%, I2 = 72.07%). Although no significant heterogeneity in sensitivity was observed, a sub-group analysis was conducted based on the pathogen, region, age, and sample pretreatment method to ascertain potential confounders. The area under the curve (AUC) of the summary receiver operating characteristic curve (SROC) of mNGS for CNS infection was 0.91 (95% CI: 0.88â0.93). Besides, Deek's Funnel Plot Asymmetry Test indicated no publication bias in the included studies (Figure 3, p > 0.05).ConclusionOverall, mNGS exhibits good sensitivity and specificity for diagnosing CNS infection and diagnostic performance during clinical application by assisting in identifying the pathogen. However, the efficacy remains inconsistent, warranting subsequent studies for further performance improvement during its clinical application.Study registration numberINPLASY202120002</p