Multiparty Dynamics and Failure Modes for Machine Learning and Artificial Intelligence

An important challenge for safety in machine learning and artificial intelligence systems is a~set of related failures involving specification gaming, reward hacking, fragility to distributional shifts, and Goodhart's or Campbell's law. This paper presents additional failure modes for interactions within multi-agent systems that are closely related. These multi-agent failure modes are more complex, more problematic, and less well understood than the single-agent case, and are also already occurring, largely unnoticed. After motivating the discussion with examples from poker-playing artificial intelligence (AI), the paper explains why these failure modes are in some senses unavoidable. Following this, the paper categorizes failure modes, provides definitions, and cites examples for each of the modes: accidental steering, coordination failures, adversarial misalignment, input spoofing and filtering, and goal co-option or direct hacking. The paper then discusses how extant literature on multi-agent AI fails to address these failure modes, and identifies work which may be useful for the mitigation of these failure modes.Comment: 12 Pages, This version re-submitted to Big Data and Cognitive Computing, Special Issue "Artificial Superintelligence: Coordination & Strategy

Evolutionary dataset optimisation: learning algorithm quality through evolution

In this paper we propose a new method for learning how algorithms perform. Classically, algorithms are compared on a finite number of existing (or newly simulated) benchmark data sets based on some fixed metrics. The algorithm(s) with the smallest value of this metric are chosen to be the `best performing'. We offer a new approach to flip this paradigm. We instead aim to gain a richer picture of the performance of an algorithm by generating artificial data through genetic evolution, the purpose of which is to create populations of datasets for which a particular algorithm performs well. These data sets can be studied to learn as to what attributes lead to a particular progress of a given algorithm. Following a detailed description of the algorithm as well as a brief description of an open source implementation, a number of numeric experiments are presented to show the performance of the method which we call Evolutionary Dataset Optimisation

A Causality-Aware Pattern Mining Scheme for Group Activity Recognition in a Pervasive Sensor Space

Human activity recognition (HAR) is a key challenge in pervasive computing and its solutions have been presented based on various disciplines. Specifically, for HAR in a smart space without privacy and accessibility issues, data streams generated by deployed pervasive sensors are leveraged. In this paper, we focus on a group activity by which a group of users perform a collaborative task without user identification and propose an efficient group activity recognition scheme which extracts causality patterns from pervasive sensor event sequences generated by a group of users to support as good recognition accuracy as the state-of-the-art graphical model. To filter out irrelevant noise events from a given data stream, a set of rules is leveraged to highlight causally related events. Then, a pattern-tree algorithm extracts frequent causal patterns by means of a growing tree structure. Based on the extracted patterns, a weighted sum-based pattern matching algorithm computes the likelihoods of stored group activities to the given test event sequence by means of matched event pattern counts for group activity recognition. We evaluate the proposed scheme using the data collected from our testbed and CASAS datasets where users perform their tasks on a daily basis and validate its effectiveness in a real environment. Experiment results show that the proposed scheme performs higher recognition accuracy and with a small amount of runtime overhead than the existing schemes

Fuzzy Finite State Machine for human activity modelling and recognition

Independent living is a housing arrangement designed exclusively for older adults to support them with their Activity of Daily Living (ADL) in a safe and secure environment. The provision of independent living would reduce the cost of social care while elderly residents are kept in their own homes. Therefore, there is a need for an automated system to monitor the residents to be able to understand their activities and only when abnormal activities are identified, provide human support to resolve the issue. Three main approaches are used for gathering data representing the human’s activities; ambient sensory device-based, wearable sensory device-based and camera vision device-based. Ambient sensory devices-based systems use sensors such as Passive Infra-Red (PIR) and door entry sensors to capture a user’s presence or absence within a specific area and record them as binary information. Gathering data using these sensory devices are widely accepted, as they are unobtrusive and it does not affect the ADLs. However, wearable sensory devices-based and camera vision device-based approaches are undesirable to many users especially for the older adults users as they more often forget to wear them and due to some privacy concerns. Recognising and modelling human activities from unobtrusive sensors is a topic addressed in Ambient Intelligence (AmI) research. The research proposed in this thesis aims to recognise and model human activities in an indoor environment based on ambient sensory device-based data. Different methods including statistical, machine learning and deep learning techniques are already researched to address the challenges of recognising and modelling human activities. The research in this thesis is mainly focusing on the application of Fuzzy Finite State Machine (FFSM) for human activities modelling and proposes ways for enhancing the FFSM performance to improve the accuracy of human activity modelling. In this thesis, three novel contributions are made which are outlined as follows; Firstly, a framework is proposed for combining the learning abilities of Neural Networks (NNs), Long Short-Term Memory (LSTM) neural network and Convolutional Neural Networks (CNNs) with the existing FFSM for human activity modelling and recognition. These models are referred to as NN-FFSM, LSTM-FFSM and CNN-FFSM. Secondly, to obtain the optimal feature representation from the acquired sensory information, relevant features are extracted and fuzzified with the selected membership degrees, these features are then applied to the different enhanced FFSM models. Thirdly, binary data gathered from the ambient sensors including PIR and door entry sensors are represented as greyscale images. A pre-trained Deep Convolutional Neural Network (DCNN) such as AlexNet is used to select and extract features from the generated greyscale image for each activity. The selected features are then used as inputs to Adaptive Boosting (AdaBoost) and Fuzzy C-means (FCM) classifiers for modelling and recognising the ADL for a single user. The proposed enhanced FFSM models were tested and evaluated using two different datasets representing the ADL for a single user. The first dataset was collected at the Smart Home facilities at NTU and the second dataset is a public dataset collected from CASAS smart home project

Recognizing Complex Activities by a Probabilistic Interval-Based Model

A key challenge in complex activity recognition is the fact that a complex activity can often be performed in several different ways, with each consisting of its own configuration of atomic actions and their temporal dependencies. This leads us to define an atomic activity-based probabilistic framework that employs Allen's interval relations to represent local temporal dependencies. The framework introduces a latent variable from the Chinese Restaurant Process to explicitly characterize these unique internal configurations of a particular complex activity as a variable number of tables.It can be analytically shown that the resulting interval network satisfies the transitivity property, and as a result, all local temporal dependencies can be retained and are globally consistent.Empirical evaluations on benchmark datasets suggest our approach significantly outperforms the state-of-the-art methods