Scenic: A Language for Scenario Specification and Scene Generation

We propose a new probabilistic programming language for the design and analysis of perception systems, especially those based on machine learning. Specifically, we consider the problems of training a perception system to handle rare events, testing its performance under different conditions, and debugging failures. We show how a probabilistic programming language can help address these problems by specifying distributions encoding interesting types of inputs and sampling these to generate specialized training and test sets. More generally, such languages can be used for cyber-physical systems and robotics to write environment models, an essential prerequisite to any formal analysis. In this paper, we focus on systems like autonomous cars and robots, whose environment is a "scene", a configuration of physical objects and agents. We design a domain-specific language, Scenic, for describing "scenarios" that are distributions over scenes. As a probabilistic programming language, Scenic allows assigning distributions to features of the scene, as well as declaratively imposing hard and soft constraints over the scene. We develop specialized techniques for sampling from the resulting distribution, taking advantage of the structure provided by Scenic's domain-specific syntax. Finally, we apply Scenic in a case study on a convolutional neural network designed to detect cars in road images, improving its performance beyond that achieved by state-of-the-art synthetic data generation methods.Comment: 41 pages, 36 figures. Full version of a PLDI 2019 paper (extending UC Berkeley EECS Department Tech Report No. UCB/EECS-2018-8

Learning to Act Properly: Predicting and Explaining Affordances from Images

We address the problem of affordance reasoning in diverse scenes that appear in the real world. Affordances relate the agent's actions to their effects when taken on the surrounding objects. In our work, we take the egocentric view of the scene, and aim to reason about action-object affordances that respect both the physical world as well as the social norms imposed by the society. We also aim to teach artificial agents why some actions should not be taken in certain situations, and what would likely happen if these actions would be taken. We collect a new dataset that builds upon ADE20k, referred to as ADE-Affordance, which contains annotations enabling such rich visual reasoning. We propose a model that exploits Graph Neural Networks to propagate contextual information from the scene in order to perform detailed affordance reasoning about each object. Our model is showcased through various ablation studies, pointing to successes and challenges in this complex task

Generalized Multivariate Extreme Value Models for Explicit Route Choice Sets

This paper analyses a class of route choice models with closed-form probability expressions, namely, Generalized Multivariate Extreme Value (GMEV) models. A large group of these models emerge from different utility formulas that combine systematic utility and random error terms. Twelve models are captured in a single discrete choice framework. The additive utility formula leads to the known logit family, being multinomial, path-size, paired combinatorial and link-nested. For the multiplicative formulation only the multinomial and path-size weibit models have been identified; this study also identifies the paired combinatorial and link-nested variations, and generalizes the path-size variant. Furthermore, a new traveller's decision rule based on the multiplicative utility formula with a reference route is presented. Here the traveller chooses exclusively based on the differences between routes. This leads to four new GMEV models. We assess the models qualitatively based on a generic structure of route utility with random foreseen travel times, for which we empirically identify that the variance of utility should be different from thus far assumed for multinomial probit and logit-kernel models. The expected travellers' behaviour and model-behaviour under simple network changes are analysed. Furthermore, all models are estimated and validated on an illustrative network example with long distance and short distance origin-destination pairs. The new multiplicative models based on differences outperform the additive models in both tests