Particle size distribution based on deep learning instance segmentation

Abstract. Deep learning has become one of the most important topics in Computer Science, and recently it proved to deliver outstanding performances in the field of Computer Vision, ranging from image classification and object detection to instance segmentation and panoptic segmentation. However, most of these results were obtained on large, publicly available datasets, that exhibit a low level of scene complexity. Less is known about applying deep neural networks to images acquired in industrial settings, where data is available in limited amounts. Moreover, comparing an image-based measurement boosted by deep learning to an established reference method can pave the way towards a shift in industrial measurements. This thesis hypothesizes that the particle size distribution can be estimated by employing a deep neural network to segment the particles of interest. The analysis was performed on two deep neural networks, comparing the results of the instance segmentation and the resulted size distributions. First, the data was manually labelled by selecting apatite and phlogopite particles, formulating the problem as a two-class instance segmentation task. Next, models were trained based on the two architectures and then used for predicting instances of particles on previously unseen images. Ultimately, accumulating the sizes of the predicted particles would result in a particle size distribution for a given dataset. The final results validated the hypothesis to some extent and showed that tackling difficult and complex challenges in the industry by leveraging state-of-the-art deep learning neural networks leads to promising results. The system was able to correctly identify most of the particles, even in challenging situations. The resulted particle size distribution was also compared to a reference measurement obtained by the laser diffraction method, but still further research and experiments are required in order to properly compare the two methods. The two evaluated architectures yielded great results, with relatively small amounts of annotated data

Validação de heterogeneidade estrutural em dados de Crio-ME por comitês de agrupadores

Orientadores: Fernando José Von Zuben, Rodrigo Villares PortugalDissertação (mestrado) - Universidade Estadual de Campinas, Faculdade de Engenharia Elétrica e de ComputaçãoResumo: Análise de Partículas Isoladas é uma técnica que permite o estudo da estrutura tridimensional de proteínas e outros complexos macromoleculares de interesse biológico. Seus dados primários consistem em imagens de microscopia eletrônica de transmissão de múltiplas cópias da molécula em orientações aleatórias. Tais imagens são bastante ruidosas devido à baixa dose de elétrons utilizada. Reconstruções 3D podem ser obtidas combinando-se muitas imagens de partículas em orientações similares e estimando seus ângulos relativos. Entretanto, estados conformacionais heterogêneos frequentemente coexistem na amostra, porque os complexos moleculares podem ser flexíveis e também interagir com outras partículas. Heterogeneidade representa um desafio na reconstrução de modelos 3D confiáveis e degrada a resolução dos mesmos. Entre os algoritmos mais populares usados para classificação estrutural estão o agrupamento por k-médias, agrupamento hierárquico, mapas autoorganizáveis e estimadores de máxima verossimilhança. Tais abordagens estão geralmente entrelaçadas à reconstrução dos modelos 3D. No entanto, trabalhos recentes indicam ser possível inferir informações a respeito da estrutura das moléculas diretamente do conjunto de projeções 2D. Dentre estas descobertas, está a relação entre a variabilidade estrutural e manifolds em um espaço de atributos multidimensional. Esta dissertação investiga se um comitê de algoritmos de não-supervisionados é capaz de separar tais "manifolds conformacionais". Métodos de "consenso" tendem a fornecer classificação mais precisa e podem alcançar performance satisfatória em uma ampla gama de conjuntos de dados, se comparados a algoritmos individuais. Nós investigamos o comportamento de seis algoritmos de agrupamento, tanto individualmente quanto combinados em comitês, para a tarefa de classificação de heterogeneidade conformacional. A abordagem proposta foi testada em conjuntos sintéticos e reais contendo misturas de imagens de projeção da proteína Mm-cpn nos estados "aberto" e "fechado". Demonstra-se que comitês de agrupadores podem fornecer informações úteis na validação de particionamentos estruturais independetemente de algoritmos de reconstrução 3DAbstract: Single Particle Analysis is a technique that allows the study of the three-dimensional structure of proteins and other macromolecular assemblies of biological interest. Its primary data consists of transmission electron microscopy images from multiple copies of the molecule in random orientations. Such images are very noisy due to the low electron dose employed. Reconstruction of the macromolecule can be obtained by averaging many images of particles in similar orientations and estimating their relative angles. However, heterogeneous conformational states often co-exist in the sample, because the molecular complexes can be flexible and may also interact with other particles. Heterogeneity poses a challenge to the reconstruction of reliable 3D models and degrades their resolution. Among the most popular algorithms used for structural classification are k-means clustering, hierarchical clustering, self-organizing maps and maximum-likelihood estimators. Such approaches are usually interlaced with the reconstructions of the 3D models. Nevertheless, recent works indicate that it is possible to infer information about the structure of the molecules directly from the dataset of 2D projections. Among these findings is the relationship between structural variability and manifolds in a multidimensional feature space. This dissertation investigates whether an ensemble of unsupervised classification algorithms is able to separate these "conformational manifolds". Ensemble or "consensus" methods tend to provide more accurate classification and may achieve satisfactory performance across a wide range of datasets, when compared with individual algorithms. We investigate the behavior of six clustering algorithms both individually and combined in ensembles for the task of structural heterogeneity classification. The approach was tested on synthetic and real datasets containing a mixture of images from the Mm-cpn chaperonin in the "open" and "closed" states. It is shown that cluster ensembles can provide useful information in validating the structural partitionings independently of 3D reconstruction methodsMestradoEngenharia de ComputaçãoMestre em Engenharia Elétric

Structure-aware image denoising, super-resolution, and enhancement methods

Denoising, super-resolution and structure enhancement are classical image processing applications. The motive behind their existence is to aid our visual analysis of raw digital images. Despite tremendous progress in these fields, certain difficult problems are still open to research. For example, denoising and super-resolution techniques which possess all the following properties, are very scarce: They must preserve critical structures like corners, should be robust to the type of noise distribution, avoid undesirable artefacts, and also be fast. The area of structure enhancement also has an unresolved issue: Very little efforts have been put into designing models that can tackle anisotropic deformations in the image acquisition process. In this thesis, we design novel methods in the form of partial differential equations, patch-based approaches and variational models to overcome the aforementioned obstacles. In most cases, our methods outperform the existing approaches in both quality and speed, despite being applicable to a broader range of practical situations.Entrauschen, Superresolution und Strukturverbesserung sind klassische Anwendungen der Bildverarbeitung. Ihre Existenz bedingt sich in dem Bestreben, die visuelle Begutachtung digitaler Bildrohdaten zu unterstützen. Trotz erheblicher Fortschritte in diesen Feldern bedürfen bestimmte schwierige Probleme noch weiterer Forschung. So sind beispielsweise Entrauschungsund Superresolutionsverfahren, welche alle der folgenden Eingenschaften besitzen, sehr selten: die Erhaltung wichtiger Strukturen wie Ecken, Robustheit bezüglich der Rauschverteilung, Vermeidung unerwünschter Artefakte und niedrige Laufzeit. Auch im Gebiet der Strukturverbesserung liegt ein ungelöstes Problem vor: Bisher wurde nur sehr wenig Forschungsaufwand in die Entwicklung von Modellen investieret, welche anisotrope Deformationen in bildgebenden Verfahren bewältigen können. In dieser Arbeit entwerfen wir neue Methoden in Form von partiellen Differentialgleichungen, patch-basierten Ansätzen und Variationsmodellen um die oben erwähnten Hindernisse zu überwinden. In den meisten Fällen übertreffen unsere Methoden nicht nur qualitativ die bisher verwendeten Ansätze, sondern lösen die gestellten Aufgaben auch schneller. Zudem decken wir mit unseren Modellen einen breiteren Bereich praktischer Fragestellungen ab

Computer Vision Approaches to Liquid-Phase Transmission Electron Microscopy

Electron microscopy (EM) is a technique that exploits the interaction between electron and matter to produce high resolution images down to atomic level. In order to avoid undesired scattering in the electron path, EM samples are conventionally imaged in solid state under vacuum conditions. Recently, this limit has been overcome by the realization of liquid-phase electron microscopy (LP EM), a technique that enables the analysis of samples in their liquid native state. LP EM paired with a high frame rate acquisition direct detection camera allows tracking the motion of particles in liquids, as well as their temporal dynamic processes. In this research work, LP EM is adopted to image the dynamics of particles undergoing Brownian motion, exploiting their natural rotation to access all the particle views, in order to reconstruct their 3D structure via tomographic techniques. However, specific computer vision-based tools were designed around the limitations of LP EM in order to elaborate the results of the imaging process. Consequently, different deblurring and denoising approaches were adopted to improve the quality of the images. Therefore, the processed LP EM images were adopted to reconstruct the 3D model of the imaged samples. This task was performed by developing two different methods: Brownian tomography (BT) and Brownian particle analysis (BPA). The former tracks in time a single particle, capturing its dynamics evolution over time. The latter is an extension in time of the single particle analysis (SPA) technique. Conventionally it is paired to cryo-EM to reconstruct 3D density maps starting from thousands of EM images by capturing hundreds of particles of the same species frozen on a grid. On the contrary, BPA has the ability to process image sequences that may not contain thousands of particles, but instead monitors individual particle views across consecutive frames, rather than across a single frame

Analysis of the Architecture of the Nuclear Pore Complex by 3D super-resolution fluorescence microscopy

Nuclear pore complexes (NPCs), embedded in the two nuclear membranes, are the unique gateways that mediate all the traffic between the nucleus and the cytoplasm. In higher eukaryotes, each NPC is composed of multiple copies of approximately 30 different proteins termed nucleoporins and has a mass of over 100 MDa. In recent years, substantial effort has been devoted to the structural and functional characterization of this essential molecular machine in eukaryotic cells. Even though a pseudo-atomic model of the scaffold of the NPC has been produced, many details of the structure still remain elusive due to the enormous size, complexity and conformational dynamics of the NPC. In addition, we have little structural understanding of how such a large machine is assembled and what dynamic structural changes underlie its functions. During my PhD work, I established a methodology that can determine the 3D architecture of the NPC by combining single-molecule localization microscopy in a 4Pi detection scheme with computational classification and 3D single particle averaging. This new approach is able to resolve the structure of the NPC with molecular specificity and nano-scale resolution in situ in human cells. I here present the reconstruction of a 3D molecular map that integrates both scaffold and flexible nucleoporins and that allows us to address dynamic components of the NPC that have been inaccessible for atomic resolution methods to date. My findings indicate that the peripheral regions of the NPC can assume very different conformational states and that even the overall scaffold structure of the NPC is more flexible than previously assumed. The methodology I have established opens the exciting possibility to address novel structural, functional and assembly aspects of this fundamental cellular machine and can be applied to interrogate the 3D architecture of other large protein complexes and organelles inside cells

Life In Motion: Visualizing Biomacromolecules By Time-Resolved Serial Femtosecond Crystallography

abstract: Time-resolved serial femtosecond crystallography is an emerging method that allows for structural discovery to be performed on biomacromolecules during their dynamic trajectory through a reaction pathway after activation. This is performed by triggering a reaction on an ensemble of molecules in nano- or microcrystals and then using femtosecond X-ray laser pulses produced by an X-ray free electron laser to collect near-instantaneous data on the crystal. A full data set can be collected by merging a sufficient number of these patterns together and multiple data sets can be collected at different points along the reaction pathway by manipulating the delay time between reaction initiation and the probing X-rays. In this way, these ‘snapshot’ structures can be viewed in series to make a molecular movie, allowing for atomic visualization of a molecule in action and, thereby, a structural basis for the mechanism and function of a given biomacromolecule. This dissertation presents results towards this end, including the successful implementations of the first diffusive mixing chemoactivated reactions and ultrafast dynamics in the femtosecond regime. The primary focus is on photosynthetic membrane proteins and enzymatic drug targets, in pursuit of strategies for sustainable energy and medical advancement by gaining understanding of the structure-function relationships evolved in nature. In particular, photosystem I, photosystem II, the complex of photosystem I and ferredoxin, and 3-deoxy-D-manno-2-octulosonate-8-phosphate synthase are reported on, from purification and isolation, to crystallogenesis, to experimental design and data collection and subsequent interpretation of results and novel insights gained.Dissertation/ThesisDoctoral Dissertation Chemistry 201

NASA SBIR abstracts of 1990 phase 1 projects

The research objectives of the 280 projects placed under contract in the National Aeronautics and Space Administration (NASA) 1990 Small Business Innovation Research (SBIR) Phase 1 program are described. The basic document consists of edited, non-proprietary abstracts of the winning proposals submitted by small businesses in response to NASA's 1990 SBIR Phase 1 Program Solicitation. The abstracts are presented under the 15 technical topics within which Phase 1 proposals were solicited. Each project was assigned a sequential identifying number from 001 to 280, in order of its appearance in the body of the report. The document also includes Appendixes to provide additional information about the SBIR program and permit cross-reference in the 1990 Phase 1 projects by company name, location by state, principal investigator, NASA field center responsible for management of each project, and NASA contract number

Radar Technology

In this book “Radar Technology”, the chapters are divided into four main topic areas: Topic area 1: “Radar Systems” consists of chapters which treat whole radar systems, environment and target functional chain. Topic area 2: “Radar Applications” shows various applications of radar systems, including meteorological radars, ground penetrating radars and glaciology. Topic area 3: “Radar Functional Chain and Signal Processing” describes several aspects of the radar signal processing. From parameter extraction, target detection over tracking and classification technologies. Topic area 4: “Radar Subsystems and Components” consists of design technology of radar subsystem components like antenna design or waveform design

The Transmission Electron Microscope

The book "The Transmission Electron Microscope" contains a collection of research articles submitted by engineers and scientists to present an overview of different aspects of TEM from the basic mechanisms and diagnosis to the latest advancements in the field. The book presents descriptions of electron microscopy, models for improved sample sizing and handling, new methods of image projection, and experimental methodologies for nanomaterials studies. The selection of chapters focuses on transmission electron microscopy used in material characterization, with special emphasis on both the theoretical and experimental aspect of modern electron microscopy techniques. I believe that a broad range of readers, such as students, scientists and engineers will benefit from this book