Enhanced fuzzy finite state machine for human activity modelling and recognition

A challenging key aspect of modelling and recognising human activity is to design a model that can deal with the uncertainty in human behaviour. Several machine learning and deep learning techniques are employed to model the Activity of Daily Living (ADL) representing the human activity. This paper proposes an enhanced Fuzzy Finite State Machine (FFSM) model by combining the classical FFSM with Long Short-Term Memory (LSTM) neural network and Convolutional Neural Network (CNN). The learning capability in the LSTM and CNN allows the system to learn the relationship in the temporal human activity data and to identify the parameters of the rule-based system as building blocks of the FFSM through time steps in the learning mode. The learned parameters are then used for generating the fuzzy rules that govern the transitions between the system’s states representing activities. The proposed enhanced FFSMs were tested and evaluated using two different datasets; a real dataset collected by our research group and a public dataset collected from CASAS smart home project. Using LSTM-FFSM, the experimental results achieved 95.7% and 97.6% for the first dataset and the second dataset, respectively. Once CNN-FFSM was applied to both datasets, the obtained results were 94.2% and 99.3%, respectively

Platonic model of mind as an approximation to neurodynamics

Hierarchy of approximations involved in simplification of microscopic theories, from sub-cellural to the whole brain level, is presented. A new approximation to neural dynamics is described, leading to a Platonic-like model of mind based on psychological spaces. Objects and events in these spaces correspond to quasi-stable states of brain dynamics and may be interpreted from psychological point of view. Platonic model bridges the gap between neurosciences and psychological sciences. Static and dynamic versions of this model are outlined and Feature Space Mapping, a neurofuzzy realization of the static version of Platonic model, described. Categorization experiments with human subjects are analyzed from the neurodynamical and Platonic model points of view

Integrating Symbolic and Neural Processing in a Self-Organizing Architechture for Pattern Recognition and Prediction

British Petroleum (89A-1204); Defense Advanced Research Projects Agency (N00014-92-J-4015); National Science Foundation (IRI-90-00530); Office of Naval Research (N00014-91-J-4100); Air Force Office of Scientific Research (F49620-92-J-0225

Fuzzy Finite State Machine for human activity modelling and recognition

Independent living is a housing arrangement designed exclusively for older adults to support them with their Activity of Daily Living (ADL) in a safe and secure environment. The provision of independent living would reduce the cost of social care while elderly residents are kept in their own homes. Therefore, there is a need for an automated system to monitor the residents to be able to understand their activities and only when abnormal activities are identified, provide human support to resolve the issue. Three main approaches are used for gathering data representing the human’s activities; ambient sensory device-based, wearable sensory device-based and camera vision device-based. Ambient sensory devices-based systems use sensors such as Passive Infra-Red (PIR) and door entry sensors to capture a user’s presence or absence within a specific area and record them as binary information. Gathering data using these sensory devices are widely accepted, as they are unobtrusive and it does not affect the ADLs. However, wearable sensory devices-based and camera vision device-based approaches are undesirable to many users especially for the older adults users as they more often forget to wear them and due to some privacy concerns. Recognising and modelling human activities from unobtrusive sensors is a topic addressed in Ambient Intelligence (AmI) research. The research proposed in this thesis aims to recognise and model human activities in an indoor environment based on ambient sensory device-based data. Different methods including statistical, machine learning and deep learning techniques are already researched to address the challenges of recognising and modelling human activities. The research in this thesis is mainly focusing on the application of Fuzzy Finite State Machine (FFSM) for human activities modelling and proposes ways for enhancing the FFSM performance to improve the accuracy of human activity modelling. In this thesis, three novel contributions are made which are outlined as follows; Firstly, a framework is proposed for combining the learning abilities of Neural Networks (NNs), Long Short-Term Memory (LSTM) neural network and Convolutional Neural Networks (CNNs) with the existing FFSM for human activity modelling and recognition. These models are referred to as NN-FFSM, LSTM-FFSM and CNN-FFSM. Secondly, to obtain the optimal feature representation from the acquired sensory information, relevant features are extracted and fuzzified with the selected membership degrees, these features are then applied to the different enhanced FFSM models. Thirdly, binary data gathered from the ambient sensors including PIR and door entry sensors are represented as greyscale images. A pre-trained Deep Convolutional Neural Network (DCNN) such as AlexNet is used to select and extract features from the generated greyscale image for each activity. The selected features are then used as inputs to Adaptive Boosting (AdaBoost) and Fuzzy C-means (FCM) classifiers for modelling and recognising the ADL for a single user. The proposed enhanced FFSM models were tested and evaluated using two different datasets representing the ADL for a single user. The first dataset was collected at the Smart Home facilities at NTU and the second dataset is a public dataset collected from CASAS smart home project

Hierarchical Attention Network for Action Segmentation

The temporal segmentation of events is an essential task and a precursor for the automatic recognition of human actions in the video. Several attempts have been made to capture frame-level salient aspects through attention but they lack the capacity to effectively map the temporal relationships in between the frames as they only capture a limited span of temporal dependencies. To this end we propose a complete end-to-end supervised learning approach that can better learn relationships between actions over time, thus improving the overall segmentation performance. The proposed hierarchical recurrent attention framework analyses the input video at multiple temporal scales, to form embeddings at frame level and segment level, and perform fine-grained action segmentation. This generates a simple, lightweight, yet extremely effective architecture for segmenting continuous video streams and has multiple application domains. We evaluate our system on multiple challenging public benchmark datasets, including MERL Shopping, 50 salads, and Georgia Tech Egocentric datasets, and achieves state-of-the-art performance. The evaluated datasets encompass numerous video capture settings which are inclusive of static overhead camera views and dynamic, ego-centric head-mounted camera views, demonstrating the direct applicability of the proposed framework in a variety of settings.Comment: Published in Pattern Recognition Letter

Consciosusness in Cognitive Architectures. A Principled Analysis of RCS, Soar and ACT-R

This report analyses the aplicability of the principles of consciousness developed in the ASys project to three of the most relevant cognitive architectures. This is done in relation to their aplicability to build integrated control systems and studying their support for general mechanisms of real-time consciousness.\ud To analyse these architectures the ASys Framework is employed. This is a conceptual framework based on an extension for cognitive autonomous systems of the General Systems Theory (GST).\ud A general qualitative evaluation criteria for cognitive architectures is established based upon: a) requirements for a cognitive architecture, b) the theoretical framework based on the GST and c) core design principles for integrated cognitive conscious control systems

AI Solutions for MDS: Artificial Intelligence Techniques for Misuse Detection and Localisation in Telecommunication Environments

This report considers the application of Articial Intelligence (AI) techniques to the problem of misuse detection and misuse localisation within telecommunications environments. A broad survey of techniques is provided, that covers inter alia rule based systems, model-based systems, case based reasoning, pattern matching, clustering and feature extraction, articial neural networks, genetic algorithms, arti cial immune systems, agent based systems, data mining and a variety of hybrid approaches. The report then considers the central issue of event correlation, that is at the heart of many misuse detection and localisation systems. The notion of being able to infer misuse by the correlation of individual temporally distributed events within a multiple data stream environment is explored, and a range of techniques, covering model based approaches, `programmed' AI and machine learning paradigms. It is found that, in general, correlation is best achieved via rule based approaches, but that these suffer from a number of drawbacks, such as the difculty of developing and maintaining an appropriate knowledge base, and the lack of ability to generalise from known misuses to new unseen misuses. Two distinct approaches are evident. One attempts to encode knowledge of known misuses, typically within rules, and use this to screen events. This approach cannot generally detect misuses for which it has not been programmed, i.e. it is prone to issuing false negatives. The other attempts to `learn' the features of event patterns that constitute normal behaviour, and, by observing patterns that do not match expected behaviour, detect when a misuse has occurred. This approach is prone to issuing false positives, i.e. inferring misuse from innocent patterns of behaviour that the system was not trained to recognise. Contemporary approaches are seen to favour hybridisation, often combining detection or localisation mechanisms for both abnormal and normal behaviour, the former to capture known cases of misuse, the latter to capture unknown cases. In some systems, these mechanisms even work together to update each other to increase detection rates and lower false positive rates. It is concluded that hybridisation offers the most promising future direction, but that a rule or state based component is likely to remain, being the most natural approach to the correlation of complex events. The challenge, then, is to mitigate the weaknesses of canonical programmed systems such that learning, generalisation and adaptation are more readily facilitated