Towards Analyzing Semantic Robustness of Deep Neural Networks

Despite the impressive performance of Deep Neural Networks (DNNs) on various vision tasks, they still exhibit erroneous high sensitivity toward semantic primitives (e.g. object pose). We propose a theoretically grounded analysis for DNN robustness in the semantic space. We qualitatively analyze different DNNs' semantic robustness by visualizing the DNN global behavior as semantic maps and observe interesting behavior of some DNNs. Since generating these semantic maps does not scale well with the dimensionality of the semantic space, we develop a bottom-up approach to detect robust regions of DNNs. To achieve this, we formalize the problem of finding robust semantic regions of the network as optimizing integral bounds and we develop expressions for update directions of the region bounds. We use our developed formulations to quantitatively evaluate the semantic robustness of different popular network architectures. We show through extensive experimentation that several networks, while trained on the same dataset and enjoying comparable accuracy, do not necessarily perform similarly in semantic robustness. For example, InceptionV3 is more accurate despite being less semantically robust than ResNet50. We hope that this tool will serve as a milestone towards understanding the semantic robustness of DNNs.Comment: Presented at European conference on computer vision (ECCV 2020) Workshop on Adversarial Robustness in the Real World ( https://eccv20-adv-workshop.github.io/ ) [best paper award]. The code is available at https://github.com/ajhamdi/semantic-robustnes

Compositional Falsification of Cyber-Physical Systems with Machine Learning Components

Cyber-physical systems (CPS), such as automotive systems, are starting to include sophisticated machine learning (ML) components. Their correctness, therefore, depends on properties of the inner ML modules. While learning algorithms aim to generalize from examples, they are only as good as the examples provided, and recent efforts have shown that they can produce inconsistent output under small adversarial perturbations. This raises the question: can the output from learning components can lead to a failure of the entire CPS? In this work, we address this question by formulating it as a problem of falsifying signal temporal logic (STL) specifications for CPS with ML components. We propose a compositional falsification framework where a temporal logic falsifier and a machine learning analyzer cooperate with the aim of finding falsifying executions of the considered model. The efficacy of the proposed technique is shown on an automatic emergency braking system model with a perception component based on deep neural networks

Formal Verification of Neural Network Controlled Autonomous Systems

In this paper, we consider the problem of formally verifying the safety of an autonomous robot equipped with a Neural Network (NN) controller that processes LiDAR images to produce control actions. Given a workspace that is characterized by a set of polytopic obstacles, our objective is to compute the set of safe initial conditions such that a robot trajectory starting from these initial conditions is guaranteed to avoid the obstacles. Our approach is to construct a finite state abstraction of the system and use standard reachability analysis over the finite state abstraction to compute the set of the safe initial states. The first technical problem in computing the finite state abstraction is to mathematically model the imaging function that maps the robot position to the LiDAR image. To that end, we introduce the notion of imaging-adapted sets as partitions of the workspace in which the imaging function is guaranteed to be affine. We develop a polynomial-time algorithm to partition the workspace into imaging-adapted sets along with computing the corresponding affine imaging functions. Given this workspace partitioning, a discrete-time linear dynamics of the robot, and a pre-trained NN controller with Rectified Linear Unit (ReLU) nonlinearity, the second technical challenge is to analyze the behavior of the neural network. To that end, we utilize a Satisfiability Modulo Convex (SMC) encoding to enumerate all the possible segments of different ReLUs. SMC solvers then use a Boolean satisfiability solver and a convex programming solver and decompose the problem into smaller subproblems. To accelerate this process, we develop a pre-processing algorithm that could rapidly prune the space feasible ReLU segments. Finally, we demonstrate the efficiency of the proposed algorithms using numerical simulations with increasing complexity of the neural network controller

The isotopic composition of water vapor: from discrete to continuous measurements. A focus on calibration methods

The water residence time in the atmosphere is approximately nine days, the shortest residence time in any major reservoir of the whole water cycle on the planet. Nevertheless, water vapor is a key factor in climate and hydrology due to its dynamic behavior. The isotopic composition of water vapor can highlight several processes of the water cycle that link the water reservoirs to the atmosphere (Galewsky et al., 2016). In the past, the isotopic composition of water vapor was generally inferred from precipitation data, assuming isotopic equilibrium between rain and water vapor. This assumption works well when precipitation is abundant but gives misleading results when precipitation is scarce. A common method to determine the isotopic composition of water vapor is the cryotrapping technique, proposed by Craig et al., (1963). Cryotrapping consists in freezing all the moisture content of the air (to avoid fractionation) and analyze the liquid sample with the regular mass spectrometry technique. This process includes the designing of customized cold traps and usually requires several man-hours due to the long sampling time (2 - 8 hours per sample). With the advent of the laser absorption spectrometry (LAS) technique is now possible to determine the isotopic composition of water vapor with sampling time down to seconds. This novel technique increases our knowledge about the isotopic composition of water vapor and gives a substantial help in our understanding of the water cycle, both on global and local scales. However, the continuous measurement of isotopic composition of water vapor requires a specific method to calibrate the large amount of data resulting as the output of a Cavity Ring-Down Spectroscopy (CRDS) analyzer. This includes the production of vapor with known isotopic composition, determination of the response of the analyzer to different humidity levels and correction of the instrumental drift. In this work, we present a summary of potential calibration techniques for continuous measurements of the isotopic composition of water vapor. The study goes in-depth on the developing of a customized calibration unit for a commercial CRDS analyzer (Picarro L1102i). Continuous measurements will be compared to water vapor samples collected with cryotraps and several continuous measurements will be presented highlighting sub-daily processes in the atmospheric boundary layer

Use of XR-QA2 radiochromic films for quantitative imaging of a synchrotron radiation beam

This work investigates the use of XR-QA2 radiochromic films for quantitative imaging of a synchrotron radiation (SR) beam. Pieces (200 7 30 mm2) of XR-QA2 film were irradiated in a plane transverse to the beam axis, at the SYRMEP beamline at ELETTRA (Trieste), with a monochromatic beam of size 170 7 3.94 mm2 (H 7 V) and energy of 28, 35, 38 or 40 keV. The response was calibrated in terms of average air kerma (1\uf02d20 mGy), measured with a calibrated ionization chamber. Films were digitized in reflectance mode using a flatbed scanner. The 16-bit red channel was used. The net\uf020reflectance was then converted to photon fluence per unit air kerma (mm-2 mGy-1). The SR beam profile was acquired also with a scintillator (GOS) based, fiberoptic coupled CCD camera as well as with a scintillator based flat panel detector. Horizontal profiles obtained with the two modalities were compared, evaluated in a ROI of 17.71 7 0.59 mm2, across the beam centre. Once corrected for flat field, the CCD profile was scaled in order to have the same average value as the normalized profile acquired with the gafchromic film. The same procedure was followed for the beam images acquired with the flat panel detector. Horizontal and vertical line profiles acquired with the radiochromic film show an uneven 2D distribution of the beam intensity, with variations in the order of 15\uf02d20% in the horizontal direction, while the statistical uncertainties evaluated for the radiochromic dose measurements were 6% at 28 keV. Larger variations up to 64% were observed in the vertical direction. The response of the radiochromic film is comparable to that of the other imaging detectors, within less than 5% variation

NNV: The Neural Network Verification Tool for Deep Neural Networks and Learning-Enabled Cyber-Physical Systems

This paper presents the Neural Network Verification (NNV) software tool, a set-based verification framework for deep neural networks (DNNs) and learning-enabled cyber-physical systems (CPS). The crux of NNV is a collection of reachability algorithms that make use of a variety of set representations, such as polyhedra, star sets, zonotopes, and abstract-domain representations. NNV supports both exact (sound and complete) and over-approximate (sound) reachability algorithms for verifying safety and robustness properties of feed-forward neural networks (FFNNs) with various activation functions. For learning-enabled CPS, such as closed-loop control systems incorporating neural networks, NNV provides exact and over-approximate reachability analysis schemes for linear plant models and FFNN controllers with piecewise-linear activation functions, such as ReLUs. For similar neural network control systems (NNCS) that instead have nonlinear plant models, NNV supports over-approximate analysis by combining the star set analysis used for FFNN controllers with zonotope-based analysis for nonlinear plant dynamics building on CORA. We evaluate NNV using two real-world case studies: the first is safety verification of ACAS Xu networks and the second deals with the safety verification of a deep learning-based adaptive cruise control system

Verisig: verifying safety properties of hybrid systems with neural network controllers

This paper presents Verisig, a hybrid system approach to verifying safety properties of closed-loop systems using neural networks as controllers. We focus on sigmoid-based networks and exploit the fact that the sigmoid is the solution to a quadratic differential equation, which allows us to transform the neural network into an equivalent hybrid system. By composing the network’s hybrid system with the plant’s, we transform the problem into a hybrid system verification problem which can be solved using state-of-theart reachability tools. We show that reachability is decidable for networks with one hidden layer and decidable for general networks if Schanuel’s conjecture is true. We evaluate the applicability and scalability of Verisig in two case studies, one from reinforcement learning and one in which the neural network is used to approximate a model predictive controller

A Nine-year series of daily oxygen and hydrogen isotopic composition of precipitation at Concordia station, East Antarctica

The atmospheric processes determining the isotopic composition of precipitation on the Antarctic plateau are yet to be fully understood, as well as the post-depositional processes altering the snow pristine isotopic signal. Improving the comprehension of these physical mechanisms is of crucial importance for interpreting the isotopic records from ice cores drilled in the low accumulation area of Antarctica, e.g., the upcoming Beyond EPICA drilling at Little Dome C. Up to now, few records of the isotopic composition of precipitation in Antarctica are available, most of them limited in time or sampling frequency. Here we present a 9-year long δ18O and δD record (2008-2016) of precipitation at Concordia base, East Antarctica. The snow is collected daily on a raised platform (1 m), positioned in the clean area of the station; the precipitation collection is still being carried out each year by the winter over personnel. A significant positive correlation between isotopes in precipitation and 2-m air temperature is observed at both seasonal and interannual scale; the lowest temperature and isotopic values are usually recorded during winters characterized by a strongly positive Southern Annular Mode index. To improve the understanding of the mechanisms governing the isotopic composition of precipitation, we compare the isotopic data of Concordia samples with on-site observations, meteorological data from the Dome C AWS of the University of Wisconsin-Madison, as well as with high-resolution simulation results from the isotope-enabled atmospheric general circulation models ECHAM5-wiso and ECHAM6-wiso, nudged with the ERA-Interim and ERA5 reanalyses respectively

Ten years of isotopic composition of precipitation at Concordia Station, East Antarctica

Oxygen and Hydrogen isotopic composition (delta18O and deltaD) in ice cores has been widely used as a proxy for reconstructing past temperature variations. However, the atmospheric dynamics determining the precipitation isotopic composition on the Antarctic Plateau are yet to be fully understood, as well as the post-depositional processes modifying the pristine snow isotopic signal: both are fundamental for the interpretation of the isotopic records from deep Antarctic ice cores drilled in low accumulation areas in order to improve past temperature reconstructions. Since 2008, daily precipitation has been continuously collected by the winter-over personnel on raised surfaces (height: 1 m) placed in the clean area of Concordia Station on the East Antarctic plateau. Each sample has been analyzed for 18O, D and deuterium excess (d): this represents a unique record, still ongoing, for the isotopic composition of precipitation in inland Antarctica. In order to better comprehend the relationship between local temperature and the isotopic signal of precipitation, temperature data (T2m) from the Dome C Automatic Weather Station of the Programma Nazionale di Ricerche in Antartide (PNRA) were correlated with precipitation sample delta18O, deltaD and d from 2008 to 2017. A significant positive correlation between delta18O and deltaD of precipitation and T2m is observed when using both daily and monthly-averaged data. The measured precipitation isotopic data were also compared to the simulated delta18O, deltaD and d from the isotope-enabled atmospheric general circulation models ECHAM5-wiso and ECHAM6-wiso, with the latter showing significant improvement in simulating the isotopic data of precipitation