Contributions To Automatic Particle Identification In Electron Micrographs: Algorithms, Implementation, And Applications

Three dimensional reconstruction of large macromolecules like viruses at resolutions below 8 Ã… - 10 Ã… requires a large set of projection images and the particle identification step becomes a bottleneck. Several automatic and semi-automatic particle detection algorithms have been developed along the years. We present a general technique designed to automatically identify the projection images of particles. The method utilizes Markov random field modelling of the projected images and involves a preprocessing of electron micrographs followed by image segmentation and post processing for boxing of the particle projections. Due to the typically extensive computational requirements for extracting hundreds of thousands of particle projections, parallel processing becomes essential. We present parallel algorithms and load balancing schemes for our algorithms. The lack of a standard benchmark for relative performance analysis of particle identification algorithms has prompted us to develop a benchmark suite. Further, we present a collection of metrics for the relative performance analysis of particle identification algorithms on the micrograph images in the suite, and discuss the design of the benchmark suite

Pushing the resolution limit by correcting the Ewald sphere effect in single-particle Cryo-EM reconstructions

The Ewald sphere effect is generally neglected when using the Central Projection Theorem for cryo electron microscopy single-particle reconstructions. This can reduce the resolution of a reconstruction. Here we estimate the attainable resolution and report a “block-based” reconstruction method for extending the resolution limit. We find the Ewald sphere effect limits the resolution of large objects, especially large viruses. After processing two real datasets of large viruses, we show that our procedure can extend the resolution for both datasets and can accommodate the flexibility associated with large protein complexes

3D SEM Surface Reconstruction: An Optimized, Adaptive, and Intelligent Approach

Structural analysis of microscopic objects is a longstanding topic in several scientific disciplines, including biological, mechanical, and material sciences. The scanning electron microscope (SEM), as a promising imaging equipment has been around to determine the surface properties (e.g., compositions or geometries) of specimens by achieving increased magnification, contrast, and resolution greater than one nanometer. Whereas SEM micrographs still remain two-dimensional (2D), many research and educational questions truly require knowledge and information about their three-dimensional (3D) surface structures. Having 3D surfaces from SEM images would provide true anatomic shapes of micro samples which would allow for quantitative measurements and informative visualization of the systems being investigated. In this research project, we novel design and develop an optimized, adaptive, and intelligent multi-view approach named 3DSEM++ for 3D surface reconstruction of SEM images, making a 3D SEM dataset publicly and freely available to the research community. The work is expected to stimulate more interest and draw attention from the computer vision and multimedia communities to the fast-growing SEM application area

New computational methods toward atomic resolution in single particle cryo-electron microscopy

Tesis doctoral inédita leída en la Universidad Autónoma de Madrid, Escuela Politécnica Superior, Departamento de Ingeniería Informática. Fecha de lectura: 22-06-2016Structural information of macromolecular complexes provides key insights into the way they carry out their biological functions. In turn, Electron microscopy (EM) is an essential tool to study the structure and function of biological macromolecules at a medium-high resolution. In this context, Single-Particle Analysis (SPA), as an EM modality, is able to yield Three-Dimensional (3-D) structural information for large biological complexes at near atomic resolution by combining many thousands of projection images. However, these views su er from low Signal-to-Noise Ratios (SNRs), since an extremely low total electron dose is used during exposure to reduce radiation damage and preserve the functional structure of macromolecules. In recent years, the emergence of Direct Detection Devices (DDDs) has opened up the possibility of obtaining images with higher SNRs. These detectors provide a set of frames instead of just one micrograph, which makes it possible to study the behavior of frozen hydrated specimens as a function of electron dose and rate. In this way, it has become apparent that biological specimens embedded in a solid matrix of amorphous ice move during imaging, resulting in Beam-Induced Motion (BIM). Therefore, alignment of frames should be added to the classical standard data processing work ow of single-particle reconstruction, which includes: particle selection, particle alignment, particle classi cation, 3-D reconstruction, and model re nement. In this thesis, we propose new algorithms and improvements for three important steps of this work ow: movie alignment, particles selection, and 3-D reconstruction. For movie alignment, a methodology based on a robust to noise optical ow approach is proposed that can e ciently correct for local movements and provide quantitative analysis of the BIM pattern. We then introduce a method for automatic particle selection in micrographs that uses some new image features to train two classi ers to learn from the user the kind of particles he is interested in. Finally, for 3-D reconstruction, we introduce a gridding-based direct Fourier method that uses a weighting technique to compute a uniform sampled Fourier transform. The algorithms are fully implemented in the open-source Xmipp package (http://xmipp.cnb.csic.es

Novel computational methods for in vitro and in situ cryo-electron microscopy

Over the past decade, advances in microscope hardware and image data processing algorithms have made cryo-electron microscopy (cryo-EM) a dominant technique for protein structure determination. Near-atomic resolution can now be obtained for many challenging in vitro samples using single-particle analysis (SPA), while sub-tomogram averaging (STA) can obtain sub-nanometer resolution for large protein complexes in a crowded cellular environment. Reaching high resolution requires large amounts of im-age data. Modern transmission electron microscopes (TEMs) automate the acquisition process and can acquire thousands of micrographs or hundreds of tomographic tilt se-ries over several days without intervention. In a first step, the data must be pre-processed: Micrographs acquired as movies are cor-rected for stage and beam-induced motion. For tilt series, additional alignment of all micrographs in 3D is performed using gold- or patch-based fiducials. Parameters of the contrast-transfer function (CTF) are estimated to enable its reversal during SPA refine-ment. Finally, individual protein particles must be located and extracted from the aligned micrographs. Current pre-processing algorithms, especially those for particle picking, are not robust enough to enable fully unsupervised operation. Thus, pre-processing is start-ed after data collection, and takes several days due to the amount of supervision re-quired. Pre-processing the data in parallel to acquisition with more robust algorithms would save time and allow to discover bad samples and microscope settings early on. Warp is a new software for cryo-EM data pre-processing. It implements new algorithms for motion correction, CTF estimation, tomogram reconstruction, as well as deep learn-ing-based approaches to particle picking and image denoising. The algorithms are more accurate and robust, enabling unsupervised operation. Warp integrates all pre-processing steps into a pipeline that is executed on-the-fly during data collection. Inte-grated with SPA tools, the pipeline can produce 2D and 3D classes less than an hour into data collection for favorable samples. Here I describe the implementation of the new algorithms, and evaluate them on various movie and tilt series data sets. I show that un-supervised pre-processing of a tilted influenza hemagglutinin trimer sample with Warp and refinement in cryoSPARC can improve previously published resolution from 3.9 Å to 3.2 Å. Warp’s algorithms operate in a reference-free manner to improve the image resolution at the pre-processing stage when no high-resolution maps are available for the particles yet. Once 3D maps have been refined, they can be used to go back to the raw data and perform reference-based refinement of sample motion and CTF in movies and tilt series. M is a new tool I developed to solve this task in a multi-particle framework. Instead of following the SPA assumption that every particle is single and independent, M models all particles in a field of view as parts of a large, physically connected multi-particle system. This allows M to optimize hyper-parameters of the system, such as sample motion and deformation, or higher-order aberrations in the CTF. Because M models these effects accurately and optimizes all hyper-parameters simultaneously with particle alignments, it can surpass previous reference-based frame and tilt series alignment tools. Here I de-scribe the implementation of M, evaluate it on several data sets, and demonstrate that the new algorithms achieve equally high resolution with movie and tilt series data of the same sample. Most strikingly, the combination of Warp, RELION and M can resolve 70S ribosomes bound to an antibiotic at 3.5 Å inside vitrified Mycoplasma pneumoniae cells, marking a major advance in resolution for in situ imaging

Doctor of Philosophy

dissertationDigital image processing has wide ranging applications in combustion research. The analysis of digital images is used in practically every scale of studying combustion phenomena from the scale of individual atoms to diagnosing and controlling large-scale combustors. Digital image processing is one of the fastest-growing scientific areas in the world today. From being able to reconstruct low-resolution grayscale images from transmitted signals, the capabilities have grown to enabling machines carrying out tasks that would normally require human vision, perception, and reasoning. Certain applications in combustion science benefit greatly from recent advances in image processing. Unfortunately, since the two fields - combustion and image processing research - stand relatively far from each other, the most recent results are often not known well enough in the areas where they may be applied with great benefits. This work aims to improve the accuracy and reliability of certain measurements in combustion science by selecting, adapting, and implementing the appropriate techniques originally developed in the image processing area. A number of specific applications were chosen that cover a wide range of physical scales of combustion phenomena, and specific image processing methodologies were proposed to improve or enable measurements in studying such phenomena. The selected applications include the description and quantification of combustion-derived carbon nanostructure, the three-dimensional optical diagnostics of combusting pulverized-coal particles and the optical flow velocimetry and quantitative radiation imaging of a pilot-scale oxy-coal flame. In the field of the structural analysis of soot, new structural parameters were derived and the extraction and fidelity of existing ones were improved. In the field of pulverized-coal combustion, the developed methodologies allow for studying the detailed mechanisms of particle combustion in three dimensions. At larger scales, the simultaneous measurement of flame velocity, spectral radiation, and pyrometric properties were realized

Validação de heterogeneidade estrutural em dados de Crio-ME por comitês de agrupadores

Orientadores: Fernando José Von Zuben, Rodrigo Villares PortugalDissertação (mestrado) - Universidade Estadual de Campinas, Faculdade de Engenharia Elétrica e de ComputaçãoResumo: Análise de Partículas Isoladas é uma técnica que permite o estudo da estrutura tridimensional de proteínas e outros complexos macromoleculares de interesse biológico. Seus dados primários consistem em imagens de microscopia eletrônica de transmissão de múltiplas cópias da molécula em orientações aleatórias. Tais imagens são bastante ruidosas devido à baixa dose de elétrons utilizada. Reconstruções 3D podem ser obtidas combinando-se muitas imagens de partículas em orientações similares e estimando seus ângulos relativos. Entretanto, estados conformacionais heterogêneos frequentemente coexistem na amostra, porque os complexos moleculares podem ser flexíveis e também interagir com outras partículas. Heterogeneidade representa um desafio na reconstrução de modelos 3D confiáveis e degrada a resolução dos mesmos. Entre os algoritmos mais populares usados para classificação estrutural estão o agrupamento por k-médias, agrupamento hierárquico, mapas autoorganizáveis e estimadores de máxima verossimilhança. Tais abordagens estão geralmente entrelaçadas à reconstrução dos modelos 3D. No entanto, trabalhos recentes indicam ser possível inferir informações a respeito da estrutura das moléculas diretamente do conjunto de projeções 2D. Dentre estas descobertas, está a relação entre a variabilidade estrutural e manifolds em um espaço de atributos multidimensional. Esta dissertação investiga se um comitê de algoritmos de não-supervisionados é capaz de separar tais "manifolds conformacionais". Métodos de "consenso" tendem a fornecer classificação mais precisa e podem alcançar performance satisfatória em uma ampla gama de conjuntos de dados, se comparados a algoritmos individuais. Nós investigamos o comportamento de seis algoritmos de agrupamento, tanto individualmente quanto combinados em comitês, para a tarefa de classificação de heterogeneidade conformacional. A abordagem proposta foi testada em conjuntos sintéticos e reais contendo misturas de imagens de projeção da proteína Mm-cpn nos estados "aberto" e "fechado". Demonstra-se que comitês de agrupadores podem fornecer informações úteis na validação de particionamentos estruturais independetemente de algoritmos de reconstrução 3DAbstract: Single Particle Analysis is a technique that allows the study of the three-dimensional structure of proteins and other macromolecular assemblies of biological interest. Its primary data consists of transmission electron microscopy images from multiple copies of the molecule in random orientations. Such images are very noisy due to the low electron dose employed. Reconstruction of the macromolecule can be obtained by averaging many images of particles in similar orientations and estimating their relative angles. However, heterogeneous conformational states often co-exist in the sample, because the molecular complexes can be flexible and may also interact with other particles. Heterogeneity poses a challenge to the reconstruction of reliable 3D models and degrades their resolution. Among the most popular algorithms used for structural classification are k-means clustering, hierarchical clustering, self-organizing maps and maximum-likelihood estimators. Such approaches are usually interlaced with the reconstructions of the 3D models. Nevertheless, recent works indicate that it is possible to infer information about the structure of the molecules directly from the dataset of 2D projections. Among these findings is the relationship between structural variability and manifolds in a multidimensional feature space. This dissertation investigates whether an ensemble of unsupervised classification algorithms is able to separate these "conformational manifolds". Ensemble or "consensus" methods tend to provide more accurate classification and may achieve satisfactory performance across a wide range of datasets, when compared with individual algorithms. We investigate the behavior of six clustering algorithms both individually and combined in ensembles for the task of structural heterogeneity classification. The approach was tested on synthetic and real datasets containing a mixture of images from the Mm-cpn chaperonin in the "open" and "closed" states. It is shown that cluster ensembles can provide useful information in validating the structural partitionings independently of 3D reconstruction methodsMestradoEngenharia de ComputaçãoMestre em Engenharia Elétric

Single particle 2D Electron crystallography for membrane protein structure determination

Proteins embedded into or attached to the cellular membrane perform crucial biological functions. Despite such importance, they remain among the most challenging targets of structural biology. Dedicated methods for membrane protein structure determination have been devised since decades, however with only partial success if compared to soluble proteins. One of these methods is 2D electron crystallography, in which the proteins are periodically arranged into a lipid bilayer. Using transmission electron microscopy to acquire projection images of samples containing such 2D crystals, which are embedded into a thin vitreous ice layer for radiation protection (cryo-EM), computer algorithms can be used to generate a 3D reconstruction of the protein. Unfortunately, in nearly every case, the 2D crystals are not flat and ordered enough to yield high-resolution reconstructions. Single particle analysis, on the other hand, is a technique that aligns projections of proteins isolated in solution in order to obtain a 3D reconstruction with a high success rate in terms of high resolution structures. In this thesis, we couple 2D crystal data processing with single particle analysis algorithms in order to perform a local correction of crystal distortions. We show that this approach not only allows reconstructions of much higher resolution than expected from the diffraction patterns obtained, but also reveals the existence of conformational heterogeneity within the 2D crystals. This structural variability can be linked to protein function, providing novel mechanistic insights and an explanation for why 2D crystals do not diffract to high resolution, in general. We present the computational methods that enable this hybrid approach, as well as other tools that aid several steps of cryo-EM data processing, from storage to postprocessing

CryoFormer: Continuous Reconstruction of 3D Structures from Cryo-EM Data using Transformer-based Neural Representations

High-resolution heterogeneous reconstruction of 3D structures of proteins and other biomolecules using cryo-electron microscopy (cryo-EM) is essential for understanding fundamental processes of life. However, it is still challenging to reconstruct the continuous motions of 3D structures from hundreds of thousands of noisy and randomly oriented 2D cryo-EM images. Existing methods based on coordinate-based neural networks show compelling results to model continuous conformations of 3D structures in the Fourier domain, but they suffer from a limited ability to model local flexible regions and lack interpretability. We propose a novel approach, cryoFormer, that utilizes a transformer-based network architecture for continuous heterogeneous cryo-EM reconstruction. We for the first time directly reconstruct continuous conformations of 3D structures using an implicit feature volume in the 3D spatial domain. A novel deformation transformer decoder further improves reconstruction quality and, more importantly, locates and robustly tackles flexible 3D regions caused by conformations. In experiments, our method outperforms current approaches on three public datasets (1 synthetic and 2 experimental) and a new synthetic dataset of PEDV spike protein. The code and new synthetic dataset will be released for better reproducibility of our results. Project page: https://cryoformer.github.io