JALAD: Joint Accuracy- and Latency-Aware Deep Structure Decoupling for Edge-Cloud Execution

Recent years have witnessed a rapid growth of deep-network based services and applications. A practical and critical problem thus has emerged: how to effectively deploy the deep neural network models such that they can be executed efficiently. Conventional cloud-based approaches usually run the deep models in data center servers, causing large latency because a significant amount of data has to be transferred from the edge of network to the data center. In this paper, we propose JALAD, a joint accuracy- and latency-aware execution framework, which decouples a deep neural network so that a part of it will run at edge devices and the other part inside the conventional cloud, while only a minimum amount of data has to be transferred between them. Though the idea seems straightforward, we are facing challenges including i) how to find the best partition of a deep structure; ii) how to deploy the component at an edge device that only has limited computation power; and iii) how to minimize the overall execution latency. Our answers to these questions are a set of strategies in JALAD, including 1) A normalization based in-layer data compression strategy by jointly considering compression rate and model accuracy; 2) A latency-aware deep decoupling strategy to minimize the overall execution latency; and 3) An edge-cloud structure adaptation strategy that dynamically changes the decoupling for different network conditions. Experiments demonstrate that our solution can significantly reduce the execution latency: it speeds up the overall inference execution with a guaranteed model accuracy loss.Comment: conference, copyright transfered to IEE

Verification of Uncertain POMDPs Using Barrier Certificates

We consider a class of partially observable Markov decision processes (POMDPs) with uncertain transition and/or observation probabilities. The uncertainty takes the form of probability intervals. Such uncertain POMDPs can be used, for example, to model autonomous agents with sensors with limited accuracy, or agents undergoing a sudden component failure, or structural damage [1]. Given an uncertain POMDP representation of the autonomous agent, our goal is to propose a method for checking whether the system will satisfy an optimal performance, while not violating a safety requirement (e.g. fuel level, velocity, and etc.). To this end, we cast the POMDP problem into a switched system scenario. We then take advantage of this switched system characterization and propose a method based on barrier certificates for optimality and/or safety verification. We then show that the verification task can be carried out computationally by sum-of-squares programming. We illustrate the efficacy of our method by applying it to a Mars rover exploration example.Comment: 8 pages, 4 figure

Robustness Verification for Classifier Ensembles

We give a formal verification procedure that decides whether a classifier ensemble is robust against arbitrary randomized attacks. Such attacks consist of a set of deterministic attacks and a distribution over this set. The robustness-checking problem consists of assessing, given a set of classifiers and a labelled data set, whether there exists a randomized attack that induces a certain expected loss against all classifiers. We show the NP-hardness of the problem and provide an upper bound on the number of attacks that is sufficient to form an optimal randomized attack. These results provide an effective way to reason about the robustness of a classifier ensemble. We provide SMT and MILP encodings to compute optimal randomized attacks or prove that there is no attack inducing a certain expected loss. In the latter case, the classifier ensemble is provably robust. Our prototype implementation verifies multiple neural-network ensembles trained for image-classification tasks. The experimental results using the MILP encoding are promising both in terms of scalability and the general applicability of our verification procedure

Artificial Intelligence and Machine Learning: A Perspective on Integrated Systems Opportunities and Challenges for Multi-Domain Operations

This paper provides a perspective on historical background, innovation and applications of Artificial Intelligence (AI) and Machine Learning (ML), data successes and systems challenges, national security interests, and mission opportunities for system problems. AI and ML today are used interchangeably, or together as AI/ML, and are ubiquitous among many industries and applications. The recent explosion, based on a confluence of new ML algorithms, large data sets, and fast and cheap computing, has demonstrated impressive results in classification and regression and used for prediction, and decision-making. Yet, AI/ML today lacks a precise definition, and as a technical discipline, it has grown beyond its origins in computer science. Even though there are impressive feats, primarily of ML, there still is much work needed in order to see the systems benefits of AI, such as perception, reasoning, planning, acting, learning, communicating, and abstraction. Recent national security interests in AI/ML have focused on problems including multidomain operations (MDO), and this has renewed the focus on a systems view of AI/ML. This paper will address the solutions for systems from an AI/ML perspective and that these solutions will draw from methods in AI and ML, as well as computational methods in control, estimation, communication, and information theory, as in the early days of cybernetics. Along with the focus on developing technology, this paper will also address the challenges of integrating these AI/ML systems for warfare