COCO_TS Dataset: Pixel-level Annotations Based on Weak Supervision for Scene Text Segmentation

The absence of large scale datasets with pixel-level supervisions is a significant obstacle for the training of deep convolutional networks for scene text segmentation. For this reason, synthetic data generation is normally employed to enlarge the training dataset. Nonetheless, synthetic data cannot reproduce the complexity and variability of natural images. In this paper, a weakly supervised learning approach is used to reduce the shift between training on real and synthetic data. Pixel-level supervisions for a text detection dataset (i.e. where only bounding-box annotations are available) are generated. In particular, the COCO-Text-Segmentation (COCO_TS) dataset, which provides pixel-level supervisions for the COCO-Text dataset, is created and released. The generated annotations are used to train a deep convolutional neural network for semantic segmentation. Experiments show that the proposed dataset can be used instead of synthetic data, allowing us to use only a fraction of the training samples and significantly improving the performances

analysis of brain nmr images for age estimation with deep learning

Abstract During the last decade, deep learning and Convolutional Neural Networks (CNNs) have produced a devastating impact on computer vision, yielding exceptional results on a variety of problems, including analysis of medical images. Recently, these techniques have been extended to 3D images with the downside of a large increase in the computational load. In particular, state-of-the-art CNNs have been used for brain Nuclear Magnetic Resonance (NMR) imaging, with the aim of estimating the patients' age. In fact, a large discrepancy between the real and the estimated age is a clear alarm for the onset of neurodegenerative diseases, such as some types of early dementia and Alzheimer's disease. In this paper, we propose an effective alternative to 3D convolutions that guarantees a significant reduction of the computational requirements for this kind of analysis. The proposed architectures achieve comparable results with the competitor 3D methods, requiring only a fraction of the training time and GPU memory

A multi-stage GAN for multi-organ chest X-ray image generation and segmentation

Multi-organ segmentation of X-ray images is of fundamental importance for computer aided diagnosis systems. However, the most advanced semantic segmentation methods rely on deep learning and require a huge amount of labeled images, which are rarely available due to both the high cost of human resources and the time required for labeling. In this paper, we present a novel multi-stage generation algorithm based on Generative Adversarial Networks (GANs) that can produce synthetic images along with their semantic labels and can be used for data augmentation. The main feature of the method is that, unlike other approaches, generation occurs in several stages, which simplifies the procedure and allows it to be used on very small datasets. The method has been evaluated on the segmentation of chest radiographic images, showing promising results. The multistage approach achieves state-of-the-art and, when very few images are used to train the GANs, outperforms the corresponding single-stage approach

Collective behavior and morphological complexity in Pseudomonas aeruginosa

In nature, many animals are capable of performing complex behaviors without centralized coordination. A well-studied focus is collective motion in flocks of birds and shoals of fish, both of which are capable of changing collective behavior as a function of individuals responding to their local environment. Similarly, despite their microscopic individual size, groups of bacteria are capable of collectively responding to and restructuring their environment. In this thesis I focus on the gamma proteobacterium Pseudomonas aeruginosa (PA), a well-studied opportunistic pathogen that is known to engage in complex collective behaviors, often controlled by a form of cell-cell communication mediated by diffusible signal molecules called quorum sensing (QS). First, I query the sensing capacity of QS, quantifying the ability to sense cell density by tracking QS-regulated secreted protease (lasB) expression on the population and single-cell scale. We find that PA can deliver a graded behavioral response (or ‘reaction norm’) to fine-scale variation in population density and show that populations generate graded responses to environmental variation through shifts in the proportion of cells responding and the intensity of responses. Given this ability of PA to quantitatively respond to discrete density environments, I then ask how the molecular machinery of QS shapes the reaction norms to changing density, via signal synthase knockout and complementation experiments. We find that the wildtype reaction norm is robust to the addition of density-independent signal supplements and more broadly, that a positive reaction norm to density is robust to multiple combinations of gene deletion and density-independent signal supplementation. Switching from QS control of a single gene (lasB), I turn to a complex multigenic and multicellular trait of colony growth. Using a collection of diverse environmental and clinical PA isolates, we develop a colony image library of 69 strains in four-fold replication. We then use a combination of image processing techniques to quantify colony morphology and complexity and find that, under common laboratory conditions, morphology and complexity form a robust, repeatable phenotype on the level of individual strains. Based on this replicable visual “fingerprint” per strain, we reasoned that colony image data could be used to classify previously unseen colony images to the strain level. Using a combination of transfer learning and data augmentation we trained a neural network to classify strains, resulting in high-level accuracy (94%). These results indicate that not only do PA strains have characteristic, replicable ‘fingerprints’, but also that these ‘fingerprints’ are learnable and classifiable. These results could provide a basis for predicting other strain-dependent behaviors including virulence or antibiotic resistance. Overall, these results highlight that complex and heterogeneous single-cell behaviors can produce robust and consistent patterns on the collective scale of environmental sensing and colony growth.Ph.D

Encoding and decoding information within native and engineered bacterial swarm patterns

Pattern formation, or the generation of coordinated, emergent behavior, is ubiquitous in nature. Researchers have long sought to understand the mechanisms behind such systems as zebra stripes, repeating flower petals, and fingers on hands, within fields such as physics and developmental biology. Notably, a diverse array of bacteria species naturally self-organize into durable macroscale patterns on solid surfaces via swarming motility—a highly coordinated, rapid movement of bacteria powered by flagella. Meanwhile, researchers in the synthetic biology field, which aims to rationally engineer living organisms for biotechnological applications, have been engineering synthetic pattern formation in microbes over the last several decades. Engineering swarming is an untapped opportunity to increase the scale and robustness of coordinated synthetic microbial systems. In this thesis, we expand the field of engineered pattern formation by applying the tools of synthetic biology and deep learning to engineer and characterize the swarming of Proteus mirabilis, which natively forms a centimeter-scale ring pattern. We engineer P. mirabilis to “write” external inputs into visible spatial records. Specifically, we engineer tunable expression of swarming-related genes that modify pattern features, and we develop quantitative approaches to decoding. Next, we develop a dual-input system that modulates two swarming-related genes simultaneously, and we apply convolutional neural networks (CNNs) to decode the resulting patterns with over 90% top-3 accuracy. We separately show growing colonies can record dynamic environmental changes which can be decoded with a U-Net model. We show the robustness of the engineered strains’ readout to fluctuations in temperature and environmental water samples. Lastly, we engineer strains which sense and respond to heavy metals. Our pCopA-flgM strain records the presence of 0 to 50 mM aqueous copper with decreased colony ring width. We conclude in this chapter that engineering native swarm patterns can thus be applied for building bacterial recorders with a visible macroscale readout. In parallel, to better characterize the swarm patterns of P. mirabilis, we develop a pipeline using deep learning approaches to segment colony images. We develop easy-to-use, semi-automated ground truth annotation and preprocessing methods. We separately segment the (1) colony background from agar and (2) the internal colony ring boundaries. The first task is achieved with a patch-classification approach; in the process, we find that the combination of the trained CNN and the “majority voting” method of label fusion achieves a test DICE score of 93% and correctly segments even faint outer swarm rings. The second task is accomplished with a U-Net which achieves over 83% test DICE. We show that our trained models easily segment a set of colonies generated at two relevant conditions, enabling automated analysis of features such as area and ring width. We apply our pipeline to analyze the more complex patterns of our engineered strains, such as the pCopA-flgM strain. The work in this chapter altogether advances the ability to analyze swarm patterns of P. mirabilis. We also aim to expand the use of our colony-characterization approaches beyond P. mirabilis to other microbes. Therefore, we present our work using deep learning to classify a set of Bacillus species isolated from soil samples. We generate datasets of the species grown under different conditions and apply transfer learning to train well-known CNN architectures such as ResNet and Inception to classify these datasets. This approach allows the models to easily learn these small datasets, and the models generalize to correctly predict a species which forms branching patterns regardless of exact growth condition. We visualize the attributions of the models with the integrated gradients method and find that model predictions are attributable to colony regions. This work sets the stage for classification, segmentation, and characterization of a wider array of microbial species with distinctive macroscale colony morphologies. Finally, we conclude by discussing ongoing efforts to expand upon the work presented in this thesis towards the sensing of dynamic inputs such as light, engineering of species other than P. mirabilis, and further optimization of the system of an engineered swarm pattern as a macroscale biosensor readout. Such work can contribute not only to the fields of synthetic pattern formation and the study of bacterial swarming, but also to the fields of engineered living materials and bio-inspired design

A deep learning approach to bacterial colony segmentation

In this paper, we introduce a new method for the segmentation of bacterial colonies in solid agar plate images. The proposed approach comprises two contributions. First, a simple but nonetheless effective engine is devised to generate synthetic plate images. This engine overlays bacterial colony patches to existing background images, taking into account both the local appearance of the background and the intrinsic opacity of the bacterial colonies. Therefore, a scalable alternative to the human ground–truth supervision—often difficult to obtain in medical imaging, due to privacy issues and scarcity of data—is provided. Then, synthetic generated data, together with few annotated images, were used to train a Fully–Convolutional Network. Such network is actually effective in separating bacterial colonies from the background. Finally, we discuss the role of the generation of synthetic images, conducting experiments that show how their inclusion improves the performances of the segmentation network, producing very encouraging results