Compositional Falsification of Cyber-Physical Systems with Machine Learning Components

Cyber-physical systems (CPS), such as automotive systems, are starting to include sophisticated machine learning (ML) components. Their correctness, therefore, depends on properties of the inner ML modules. While learning algorithms aim to generalize from examples, they are only as good as the examples provided, and recent efforts have shown that they can produce inconsistent output under small adversarial perturbations. This raises the question: can the output from learning components can lead to a failure of the entire CPS? In this work, we address this question by formulating it as a problem of falsifying signal temporal logic (STL) specifications for CPS with ML components. We propose a compositional falsification framework where a temporal logic falsifier and a machine learning analyzer cooperate with the aim of finding falsifying executions of the considered model. The efficacy of the proposed technique is shown on an automatic emergency braking system model with a perception component based on deep neural networks

Detecting facial features automatically

Text in English; Abstract: English and TurkishIncludes bibliographical references (leaves 57-58)xii, 59 leavesThere are many algorithms and approaches in object detection world. Many of them are based on Viola Jones algorithm. According to our observations, the features which help to detect an object are very critical for the success of this algorithm. These features are usually created manually. In this thesis we explore automatic extraction of Haar-like features. We describe the design and construction of a completely automated face detector for gray scale images. Finally, we illustrate the performance of our algorithm on various databases.Obje tespit etmek icin bir çok algoritma ve yaklaşım vardır. Bunların çoğu Viola Jones algoritmasına dayanır. Bizim edindiğimiz tecrübelere göre, obje tespitinde temel konu o objeye ait özniteliklerdir. Bu öznitelikler genellikle manuel olarak oluşturulur. Bu tezde biz Haar-like özniteliklerin otomatik çıkarımları üzerine araştırma yaptık. Gri tonlamalı resimler için tamamıyla otomatikleştirilmiş bir yüz algılayıcısı tasarlayıp bunu uyguladık. Nihayetinde, tasarladığımız algoritmanın farklı veribankaları üzerindeki performansını gösterdik

HUMAN ROBOT INTERACTION THROUGH SEMANTIC INTEGRATION OF MULTIPLE MODALITIES, DIALOG MANAGEMENT, AND CONTEXTS

The hypothesis for this research is that applying the Human Computer Interaction (HCI) concepts of using multiple modalities, dialog management, context, and semantics to Human Robot Interaction (HRI) will improve the performance of Instruction Based Learning (IBL) compared to only using speech. We tested the hypothesis by simulating a domestic robot that can be taught to clean a house using a multi-modal interface. We used a method of semantically integrating the inputs from multiple modalities and contexts that multiplies a confidence score for each input by a Fusion Weight, sums the products, and then uses the input with the highest product sum. We developed an algorithm for determining the Fusion Weights. We concluded that different modalities, contexts, and modes of dialog management impact human robot interaction; however, which combination is better depends on the importance of the accuracy of learning what is taught versus the succinctness of the dialog between the user and the robot

Internally Rewarded Reinforcement Learning

We study a class of reinforcement learning problems where the reward signals for policy learning are generated by a discriminator that is dependent on and jointly optimized with the policy. This interdependence between the policy and the discriminator leads to an unstable learning process because reward signals from an immature discriminator are noisy and impede policy learning, and conversely, an under-optimized policy impedes discriminator learning. We call this learning setting \textit{Internally Rewarded Reinforcement Learning} (IRRL) as the reward is not provided directly by the environment but \textit{internally} by the discriminator. In this paper, we formally formulate IRRL and present a class of problems that belong to IRRL. We theoretically derive and empirically analyze the effect of the reward function in IRRL and based on these analyses propose the clipped linear reward function. Experimental results show that the proposed reward function can consistently stabilize the training process by reducing the impact of reward noise, which leads to faster convergence and higher performance compared with baselines in diverse tasks.Comment: Accepted at ICML 2023. Project webpage at https://ir-rl.github.i

Supervised Classification: Quite a Brief Overview

The original problem of supervised classification considers the task of automatically assigning objects to their respective classes on the basis of numerical measurements derived from these objects. Classifiers are the tools that implement the actual functional mapping from these measurements---also called features or inputs---to the so-called class label---or output. The fields of pattern recognition and machine learning study ways of constructing such classifiers. The main idea behind supervised methods is that of learning from examples: given a number of example input-output relations, to what extent can the general mapping be learned that takes any new and unseen feature vector to its correct class? This chapter provides a basic introduction to the underlying ideas of how to come to a supervised classification problem. In addition, it provides an overview of some specific classification techniques, delves into the issues of object representation and classifier evaluation, and (very) briefly covers some variations on the basic supervised classification task that may also be of interest to the practitioner

Enabling Explainable Fusion in Deep Learning with Fuzzy Integral Neural Networks

Information fusion is an essential part of numerous engineering systems and biological functions, e.g., human cognition. Fusion occurs at many levels, ranging from the low-level combination of signals to the high-level aggregation of heterogeneous decision-making processes. While the last decade has witnessed an explosion of research in deep learning, fusion in neural networks has not observed the same revolution. Specifically, most neural fusion approaches are ad hoc, are not understood, are distributed versus localized, and/or explainability is low (if present at all). Herein, we prove that the fuzzy Choquet integral (ChI), a powerful nonlinear aggregation function, can be represented as a multi-layer network, referred to hereafter as ChIMP. We also put forth an improved ChIMP (iChIMP) that leads to a stochastic gradient descent-based optimization in light of the exponential number of ChI inequality constraints. An additional benefit of ChIMP/iChIMP is that it enables eXplainable AI (XAI). Synthetic validation experiments are provided and iChIMP is applied to the fusion of a set of heterogeneous architecture deep models in remote sensing. We show an improvement in model accuracy and our previously established XAI indices shed light on the quality of our data, model, and its decisions.Comment: IEEE Transactions on Fuzzy System

Hyperdrive: A Multi-Chip Systolically Scalable Binary-Weight CNN Inference Engine

Deep neural networks have achieved impressive results in computer vision and machine learning. Unfortunately, state-of-the-art networks are extremely compute and memory intensive which makes them unsuitable for mW-devices such as IoT end-nodes. Aggressive quantization of these networks dramatically reduces the computation and memory footprint. Binary-weight neural networks (BWNs) follow this trend, pushing weight quantization to the limit. Hardware accelerators for BWNs presented up to now have focused on core efficiency, disregarding I/O bandwidth and system-level efficiency that are crucial for deployment of accelerators in ultra-low power devices. We present Hyperdrive: a BWN accelerator dramatically reducing the I/O bandwidth exploiting a novel binary-weight streaming approach, which can be used for arbitrarily sized convolutional neural network architecture and input resolution by exploiting the natural scalability of the compute units both at chip-level and system-level by arranging Hyperdrive chips systolically in a 2D mesh while processing the entire feature map together in parallel. Hyperdrive achieves 4.3 TOp/s/W system-level efficiency (i.e., including I/Os)---3.1x higher than state-of-the-art BWN accelerators, even if its core uses resource-intensive FP16 arithmetic for increased robustness

Machine Conscious Architecture for State Exploitation and Decision Making

This research addressed a critical limitation in the area of computational intelligence by developing a general purpose architecture for information processing and decision making. Traditional computational intelligence methods are best suited for well-defined problems with extensive, long-term knowledge of the environmental and operational conditions the system will encounter during operation. These traditional approaches typically generate quick answers (i.e., reflexive responses) using pattern recognition methods. Most pattern recognition techniques are static processes which consist of a predefined series of computations. For these pattern recognition approaches to be effective, training data is required from all anticipated environments and operating conditions. The proposed framework, Conscious Architecture for State Exploitation (CASE), is a general purpose architecture designed to mimic key characteristics of human information processing. CASE combines low- and high-level cognitive processes into a common framework to enable goal-based decision making. The CASE approach is to generate artificial phenomenal states (i.e., generate qualia = consciousness) into a shared computational process to enhance goal-based decision making and adaptation. That is, this approach allows for the appropriate decision and corresponding adaptive behavior as the goals and environmental factors change. To demonstrate the engineering advantages of CASE, it was used in an airframe application to autonomously monitor the integrity of a flight critical structural component. In this demonstration, CASE automatically generated a timely maintenance recommendation when unacceptable cracking was detected. Over the lifetime of the investigated component, operational availability increased by a minimum of 10.7%, operational cost decreased by 79%, and maintenance intervals (i.e., MTBM) increased by a minimum of 900%