Structured output prediction for semantic perception in autonomous vehicles

A key challenge in the realization of autonomous vehicles is the machine's ability to perceive its surrounding environment. This task is tackled through a model that partitions vehicle camera input into distinct semantic classes, by taking into account visual contextual cues. The use of structured machine learning models is investigated, which not only allow for complex input, but also arbitrarily structured output. Towards this goal, an outdoor road scene dataset is constructed with accompanying fine-grained image labelings. For coherent segmentation, a structured predictor is modeled to encode label distributions conditioned on the input images. After optimizing this model through max-margin learning, based on an ontological loss function, efficient classification is realized via graph cuts inference using alpha-expansion. Both quantitative and qualitative analyses demonstrate that by taking into account contextual relations between pixel segmentation regions within a second-degree neighborhood, spurious label assignments are filtered out, leading to highly accurate semantic segmentations for outdoor scenes

Joint segmentation and classification of retinal arteries/veins from fundus images

Objective Automatic artery/vein (A/V) segmentation from fundus images is required to track blood vessel changes occurring with many pathologies including retinopathy and cardiovascular pathologies. One of the clinical measures that quantifies vessel changes is the arterio-venous ratio (AVR) which represents the ratio between artery and vein diameters. This measure significantly depends on the accuracy of vessel segmentation and classification into arteries and veins. This paper proposes a fast, novel method for semantic A/V segmentation combining deep learning and graph propagation. Methods A convolutional neural network (CNN) is proposed to jointly segment and classify vessels into arteries and veins. The initial CNN labeling is propagated through a graph representation of the retinal vasculature, whose nodes are defined as the vessel branches and edges are weighted by the cost of linking pairs of branches. To efficiently propagate the labels, the graph is simplified into its minimum spanning tree. Results The method achieves an accuracy of 94.8% for vessels segmentation. The A/V classification achieves a specificity of 92.9% with a sensitivity of 93.7% on the CT-DRIVE database compared to the state-of-the-art-specificity and sensitivity, both of 91.7%. Conclusion The results show that our method outperforms the leading previous works on a public dataset for A/V classification and is by far the fastest. Significance The proposed global AVR calculated on the whole fundus image using our automatic A/V segmentation method can better track vessel changes associated to diabetic retinopathy than the standard local AVR calculated only around the optic disc.Comment: Preprint accepted in Artificial Intelligence in Medicin

Map-Guided Curriculum Domain Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation

We address the problem of semantic nighttime image segmentation and improve the state-of-the-art, by adapting daytime models to nighttime without using nighttime annotations. Moreover, we design a new evaluation framework to address the substantial uncertainty of semantics in nighttime images. Our central contributions are: 1) a curriculum framework to gradually adapt semantic segmentation models from day to night through progressively darker times of day, exploiting cross-time-of-day correspondences between daytime images from a reference map and dark images to guide the label inference in the dark domains; 2) a novel uncertainty-aware annotation and evaluation framework and metric for semantic segmentation, including image regions beyond human recognition capability in the evaluation in a principled fashion; 3) the Dark Zurich dataset, comprising 2416 unlabeled nighttime and 2920 unlabeled twilight images with correspondences to their daytime counterparts plus a set of 201 nighttime images with fine pixel-level annotations created with our protocol, which serves as a first benchmark for our novel evaluation. Experiments show that our map-guided curriculum adaptation significantly outperforms state-of-the-art methods on nighttime sets both for standard metrics and our uncertainty-aware metric. Furthermore, our uncertainty-aware evaluation reveals that selective invalidation of predictions can improve results on data with ambiguous content such as our benchmark and profit safety-oriented applications involving invalid inputs.Comment: IEEE T-PAMI 202

The Emergence of Canalization and Evolvability in an Open-Ended, Interactive Evolutionary System

Natural evolution has produced a tremendous diversity of functional organisms. Many believe an essential component of this process was the evolution of evolvability, whereby evolution speeds up its ability to innovate by generating a more adaptive pool of offspring. One hypothesized mechanism for evolvability is developmental canalization, wherein certain dimensions of variation become more likely to be traversed and others are prevented from being explored (e.g. offspring tend to have similarly sized legs, and mutations affect the length of both legs, not each leg individually). While ubiquitous in nature, canalization almost never evolves in computational simulations of evolution. Not only does that deprive us of in silico models in which to study the evolution of evolvability, but it also raises the question of which conditions give rise to this form of evolvability. Answering this question would shed light on why such evolvability emerged naturally and could accelerate engineering efforts to harness evolution to solve important engineering challenges. In this paper we reveal a unique system in which canalization did emerge in computational evolution. We document that genomes entrench certain dimensions of variation that were frequently explored during their evolutionary history. The genetic representation of these organisms also evolved to be highly modular and hierarchical, and we show that these organizational properties correlate with increased fitness. Interestingly, the type of computational evolutionary experiment that produced this evolvability was very different from traditional digital evolution in that there was no objective, suggesting that open-ended, divergent evolutionary processes may be necessary for the evolution of evolvability.Comment: SI can be found at: http://www.evolvingai.org/files/SI_0.zi

Reoccurring neural stem cell divisions in the adult zebrafish telencephalon are sufficient for the emergence of aggregated spatiotemporal patterns

Regulation of quiescence and cell cycle entry is pivotal for the maintenance of stem cell populations. Regulatory mechanisms, however, are poorly understood. In particular, it is unclear how the activity of single stem cells is coordinated within the population or if cells divide in a purely random fashion. We addressed this issue by analyzing division events in an adult neural stem cell (NSC) population of the zebrafish telencephalon. Spatial statistics and mathematical modeling of over 80,000 NSCs in 36 brain hemispheres revealed weakly aggregated, nonrandom division patterns in space and time. Analyzing divisions at 2 time points allowed us to infer cell cycle and S-phase lengths computationally. Interestingly, we observed rapid cell cycle reentries in roughly 15% of newly born NSCs. In agent-based simulations of NSC populations, this redividing activity sufficed to induce aggregated spatiotemporal division patterns that matched the ones observed experimentally. In contrast, omitting redivisions leads to a random spatiotemporal distribution of dividing cells. Spatiotemporal aggregation of dividing stem cells can thus emerge solely from the cell's history

Polar Auxin Transport And Auxin Induced Development : Root System And Signaling Molecules Give The Clue

Nearly every developmental phase and every growth process of a plant is affected by auxin. Auxin is transported over long distance via the phloem but also polarly from cell-to-cell, along the entire plant body; from the shoot tip downwards and from the root tip upwards. This rather unique feature is not shared by any other known signaling molecule in plants. Auxin transport is always directional, energy dependent, and substrate specific. The mechanism of polar auxin transport (PAT) is described by the chemiosmotic model, proposed some 33years ago by Raven and independently by Rubery and Sheldrake. Since then it has obtained paradigm status receiving only marginal modifications by the discovery of some molecular components such as auxin influx and efflux transporters. However, the current study is describing and discussing observations, which contradict several predictions of the classical chemiosmotic theory of PAT, which makes it desirable to update our view of cellular auxin efflux. For example, the chemiosmotic theory cannot explain the rapid inhibition of PAT by Brefeldin A, an inhibitor of secretion. Under these conditions, i.e., while PAT is inhibited, PIN auxin transporters are still present at the plasma membrane. This observation is the motivation and starting point in the current study to investigate the relationship between PAT and endosomal membrane recycling with some consideration of auxin action on root hair formation, and upstream effectors of auxin such as D´orenone and indole butyric acid (IBA). A new IAA specific antibody has been used here to re-investigate several key aspects of polar auxin transport. The conclusions from this work are: (1) Endosomes and vesicle recycling are essential parts of the auxin transport machinery. Auxin is enriched at cross wall domains (end poles) of IAA transporting cells, but not of cells impaired in PAT either due to inhibitors or to genetic lesions. (2) The mere presence of PIN proteins at the plasma membrane is not sufficient to sustain auxin transport. (3) Continuous F-actin-dependent vesicle recycling between the plasma membrane and endosomes is necessary for polar positioning of PIN. It is further shown, that auxin transport provides vectorial information for the localization of root hair initiation sites close to apical ends of hair-forming cells (ends that are oriented towards the root tip). Once initiated, root hair tip growth is based on a threshold level of auxin in the trichoblasts. Auxin export out of trichoblasts is driven by the PIN2 auxin efflux transporter to the next basal epidermis cell. This study provides evidence, that one cleavage product of ß-carotene, called D’orenone, inhibits root hair formation and modifies the root architecture. Analysis of cellular changes after D´orenone treatment reveal an effect of this substance on PAT. In the D´orenone treated roots more PIN2 protein is found in root apices and the PIN2 expression zone is increased in size and shifted basally (away from the root apex). Root hair growth inhibition by D´orenone is rescued by externally applied IAA. Moreover, PIN2 knock-out mutants are insensitive towards D´orenone. Taken together, D´orenone targets root hair growth by upregulating PIN2 and thereby increasing the rate of PAT. It is important to note, that auxin based modulation of root architecture involves second messengers such as reactive oxygen species and reactive nitrogen species (ROS/RNS). It is shown here, that mutants which are insensitive to the naturally occuring auxin, indole butyric acid (IBA), proove to be useful experimental tools to examine the relevance of the ROS/RNS dependent activity mechanism. Apparently there are two pathways involved in the action of IBA. One is that IBA is converted to IAA, which then feeds IAA into intracellular signalling pathways, and the other is that the IBA-conversion induces ROS/RNS production, though in a similar pattern as IAA within root tissues. Both generated IAA and alongside the IBA-to-IAA conversion produced nitric oxide are necessary for any IBA related root architecture altering effects. The comparison of IBA and IAA activity in this study implicates that all mentioned auxin-dependent changes of root growth and development are formed by complex feedback interactions between auxin and stress-related signalling molecules, which together underly environment-dependent phenotypic plasticity of the plant root system architecture.</p