Boosting-based Construction of BDDs for Linear Threshold Functions and Its Application to Verification of Neural Networks

Understanding the characteristics of neural networks is important but difficult due to their complex structures and behaviors. Some previous work proposes to transform neural networks into equivalent Boolean expressions and apply verification techniques for characteristics of interest. This approach is promising since rich results of verification techniques for circuits and other Boolean expressions can be readily applied. The bottleneck is the time complexity of the transformation. More precisely, (i) each neuron of the network, i.e., a linear threshold function, is converted to a Binary Decision Diagram (BDD), and (ii) they are further combined into some final form, such as Boolean circuits. For a linear threshold function with $n$ variables, an existing method takes $O(n2^{\frac{n}{2}})$ time to construct an ordered BDD of size $O(2^{\frac{n}{2}})$ consistent with some variable ordering. However, it is non-trivial to choose a variable ordering producing a small BDD among $n!$ candidates. We propose a method to convert a linear threshold function to a specific form of a BDD based on the boosting approach in the machine learning literature. Our method takes $O(2^n \text{poly}(1/\rho))$ time and outputs BDD of size $O(\frac{n^2}{\rho^4}\ln{\frac{1}{\rho}})$, where $\rho$ is the margin of some consistent linear threshold function. Our method does not need to search for good variable orderings and produces a smaller expression when the margin of the linear threshold function is large. More precisely, our method is based on our new boosting algorithm, which is of independent interest. We also propose a method to combine them into the final Boolean expression representing the neural network

On Training Neurons with Bounded Compilations

Knowledge compilation offers a formal approach to explaining and verifying the behavior of machine learning systems, such as neural networks. Unfortunately, compiling even an individual neuron into a tractable representation such as an Ordered Binary Decision Diagram (OBDD), is an NP-hard problem. In this thesis, we consider the problem of training a neuron from data, subject to the constraint that it has a compact representation as an OBDD. Our approach is based on the observation that a neuron can be compiled into an OBDD in polytime if (1) the neuron has integer weights, and (2) its aggregate weight is bounded. Unfortunately, we first show that it is also NP-hard to train a neuron, subject to these two constraints. On the other hand, we show that if we train a neuron generatively, rather than discriminatively, a neuron with bounded aggregate weight can be trained in pseudo-polynomial time. Hence, we propose the first efficient algorithm for training a neuron that is guaranteed to have a compact representation as an OBDD. Empirically, we show that our approach can train neurons with higher accuracy and more compact OBDDs

A Study of the Learnability of Relational Properties: Model Counting Meets Machine Learning (MCML)

This paper introduces the MCML approach for empirically studying the learnability of relational properties that can be expressed in the well-known software design language Alloy. A key novelty of MCML is quantification of the performance of and semantic differences among trained machine learning (ML) models, specifically decision trees, with respect to entire (bounded) input spaces, and not just for given training and test datasets (as is the common practice). MCML reduces the quantification problems to the classic complexity theory problem of model counting, and employs state-of-the-art model counters. The results show that relatively simple ML models can achieve surprisingly high performance (accuracy and F1-score) when evaluated in the common setting of using training and test datasets - even when the training dataset is much smaller than the test dataset - indicating the seeming simplicity of learning relational properties. However, MCML metrics based on model counting show that the performance can degrade substantially when tested against the entire (bounded) input space, indicating the high complexity of precisely learning these properties, and the usefulness of model counting in quantifying the true performance

Three Modern Roles for Logic in AI

We consider three modern roles for logic in artificial intelligence, which are based on the theory of tractable Boolean circuits: (1) logic as a basis for computation, (2) logic for learning from a combination of data and knowledge, and (3) logic for reasoning about the behavior of machine learning systems.Comment: To be published in PODS 202

The Assurance of Bayesian Networks for Mission Critical Systems

A prerequisite for the assurance of any mission-critical system is a comprehensive understanding of a system’s properties and behaviours. This is a challenging proposition for many AI-based Systems (AISs). Their functionality is often dictated by factors that are often outside the scope of the assurance concerns typical of conventional software systems. These distinctions have implications for all phases of the design, development, deployment and operation of AISs. They pose serious problems for existing software assurance standards, guidelines and techniques: the application of existing practices to an AIS will fail to expose or mitigate numerous system aspects that can contribute to hazardous system behaviours. This thesis introduces a number of techniques that aim to support the resolution of these problems for Bayesian Network-based Systems (BNSs). This class of system has been deployed in many applications, ranging from medical diagnostic systems to naviga- tional controls aboard autonomous systems. To date, there is no published literature on the deployment of these systems in directly safety-critical roles. This thesis introduces ap- proaches aimed at addressing three particular challenges. Firstly, it proposes a framework for conceptualising and communicating the distinctions between BNSs and conventional software systems and uses this framework to generate and refine a set of BNS verification and validation objectives. Secondly, it introduces an assurance-focussed BNS analysis technique that can provide targeted information on mission-critical aspects of a BNS. Finally, it outlines an approach for describing how BNS-specific safety evidence relates to BNS aspects, and how the evidence can be used to derive sufficient confidence in a mission-critical BNS. These contributions are then evaluated in the context of a case study that indicates the utility of the proposed techniques, and how these can be used to comprehensively structure and target the unconventional assurance concerns associated with the development of a mission-critical BNS