227 research outputs found
3D exemplar-based image inpainting in electron microscopy
In electron microscopy (EM) a common problem is the non-availability of data, which causes artefacts in reconstructions. In this thesis the goal is to generate artificial data where missing in EM by using exemplar-based inpainting (EBI). We implement an accelerated 3D version tailored to applications in EM, which reduces reconstruction times from days to minutes. We develop intelligent sampling strategies to find optimal data as input for reconstruction methods. Further, we investigate approaches to reduce electron dose and acquisition time. Sparse sampling followed by inpainting is the most promising approach. As common evaluation measures may lead to misinterpretation of results in EM and falsify a subsequent analysis, we propose to use application driven metrics and demonstrate this in a segmentation task. A further application of our technique is the artificial generation of projections in tiltbased EM. EBI is used to generate missing projections, such that the full angular range is covered. Subsequent reconstructions are significantly enhanced in terms of resolution, which facilitates further analysis of samples. In conclusion, EBI proves promising when used as an additional data generation step to tackle the non-availability of data in EM, which is evaluated in selected applications. Enhancing adaptive sampling methods and refining EBI, especially considering the mutual influence, promotes higher throughput in EM using less electron dose while not lessening quality.Ein häufig vorkommendes Problem in der Elektronenmikroskopie (EM) ist die Nichtverfügbarkeit von Daten, was zu Artefakten in Rekonstruktionen führt. In dieser Arbeit ist es das Ziel fehlende Daten in der EM künstlich zu erzeugen, was durch Exemplar-basiertes Inpainting (EBI) realisiert wird. Wir implementieren eine auf EM zugeschnittene beschleunigte 3D Version, welche es ermöglicht, Rekonstruktionszeiten von Tagen auf Minuten zu reduzieren. Wir entwickeln intelligente Abtaststrategien, um optimale Datenpunkte für die Rekonstruktion zu erhalten. Ansätze zur Reduzierung von Elektronendosis und Aufnahmezeit werden untersucht. Unterabtastung gefolgt von Inpainting führt zu den besten Resultaten. Evaluationsmaße zur Beurteilung der Rekonstruktionsqualität helfen in der EM oft nicht und können zu falschen Schlüssen führen, weswegen anwendungsbasierte Metriken die bessere Wahl darstellen. Dies demonstrieren wir anhand eines Beispiels. Die künstliche Erzeugung von Projektionen in der neigungsbasierten Elektronentomographie ist eine weitere Anwendung. EBI wird verwendet um fehlende Projektionen zu generieren. Daraus resultierende Rekonstruktionen weisen eine deutlich erhöhte Auflösung auf. EBI ist ein vielversprechender Ansatz, um nicht verfügbare Daten in der EM zu generieren. Dies wird auf Basis verschiedener Anwendungen gezeigt und evaluiert. Adaptive Aufnahmestrategien und EBI können also zu einem höheren Durchsatz in der EM führen, ohne die Bildqualität merklich zu verschlechtern
Self-Supervised Super-Resolution Approach for Isotropic Reconstruction of 3D Electron Microscopy Images from Anisotropic Acquisition
Three-dimensional electron microscopy (3DEM) is an essential technique to
investigate volumetric tissue ultra-structure. Due to technical limitations and
high imaging costs, samples are often imaged anisotropically, where resolution
in the axial direction () is lower than in the lateral directions .
This anisotropy 3DEM can hamper subsequent analysis and visualization tasks. To
overcome this limitation, we propose a novel deep-learning (DL)-based
self-supervised super-resolution approach that computationally reconstructs
isotropic 3DEM from the anisotropic acquisition. The proposed DL-based
framework is built upon the U-shape architecture incorporating
vision-transformer (ViT) blocks, enabling high-capability learning of local and
global multi-scale image dependencies. To train the tailored network, we employ
a self-supervised approach. Specifically, we generate pairs of anisotropic and
isotropic training datasets from the given anisotropic 3DEM data. By feeding
the given anisotropic 3DEM dataset in the trained network through our proposed
framework, the isotropic 3DEM is obtained. Importantly, this isotropic
reconstruction approach relies solely on the given anisotropic 3DEM dataset and
does not require pairs of co-registered anisotropic and isotropic 3DEM training
datasets. To evaluate the effectiveness of the proposed method, we conducted
experiments using three 3DEM datasets acquired from brain. The experimental
results demonstrated that our proposed framework could successfully reconstruct
isotropic 3DEM from the anisotropic acquisition
Multi-modal registration for correlative microscopy using image analogies
Correlative microscopy is a methodology combining the functionality of light microscopy with the high resolution of electron microscopy and other microscopy technologies for the same biological specimen. In this paper, we propose an image registration method for correlative microscopy, which is challenging due to the distinct appearance of biological structures when imaged with different modalities. Our method is based on image analogies and allows to transform images of a given modality into the appearance-space of another modality. Hence, the registration between two different types of microscopy images can be transformed to a mono-modality image registration. We use a sparse representation model to obtain image analogies. The method makes use of corresponding image training patches of two different imaging modalities to learn a dictionary capturing appearance relations. We test our approach on backscattered electron (BSE) scanning electron microscopy (SEM)/confocal and transmission electron microscopy (TEM)/confocal images. We perform rigid, affine, and deformable registration via B-splines and show improvements over direct registration using both mutual information and sum of squared differences similarity measures to account for differences in image appearance
Stretched sinograms for limited-angle tomographic reconstruction with neural networks
We present a direct method for limited angle tomographic reconstruction using
convolutional networks. The key to our method is to first stretch every tilt
view in the direction perpendicular to the tilt axis by the secant of the tilt
angle. These stretched views are then fed into a 2-D U-Net which directly
outputs the 3-D reconstruction. We train our networks by minimizing the mean
squared error between the network's generated reconstruction and a ground truth
3-D volume. To demonstrate and evaluate our method, we synthesize tilt views
from a 3-D image of fly brain tissue acquired with Focused Ion Beam Scanning
Electron Microscopy. We compare our method to using a U-Net to directly
reconstruct the unstretched tilt views and show that this simple stretching
procedure leads to significantly better reconstructions. We also compare to
using a network to clean up reconstructions generated by backprojection and
filtered backprojection, and find that this simple stretching procedure also
gives lower mean squared error on previously unseen images
- …