Gesture Recognition from Data Streams of Human Motion Sensor Using Accelerated PSO Swarm Search Feature Selection Algorithm

Human motion sensing technology gains tremendous popularity nowadays with practical applications such as video surveillance for security, hand signing, and smart-home and gaming. These applications capture human motions in real-time from video sensors, the data patterns are nonstationary and ever changing. While the hardware technology of such motion sensing devices as well as their data collection process become relatively mature, the computational challenge lies in the real-time analysis of these live feeds. In this paper we argue that traditional data mining methods run short of accurately analyzing the human activity patterns from the sensor data stream. The shortcoming is due to the algorithmic design which is not adaptive to the dynamic changes in the dynamic gesture motions. The successor of these algorithms which is known as data stream mining is evaluated versus traditional data mining, through a case of gesture recognition over motion data by using Microsoft Kinect sensors. Three different subjects were asked to read three comic strips and to tell the stories in front of the sensor. The data stream contains coordinates of articulation points and various positions of the parts of the human body corresponding to the actions that the user performs. In particular, a novel technique of feature selection using swarm search and accelerated PSO is proposed for enabling fast preprocessing for inducing an improved classification model in real-time. Superior result is shown in the experiment that runs on this empirical data stream. The contribution of this paper is on a comparative study between using traditional and data stream mining algorithms and incorporation of the novel improved feature selection technique with a scenario where different gesture patterns are to be recognized from streaming sensor data

Hierarchical Modelling and Recognition of Activities of Daily Living

Activity recognition is becoming an increasingly important task in artificial intelligence. Successful activity recognition systems must be able to model and recognise activities ranging from simple short activities spanning a few seconds to complex longer activities spanning minutes or hours. We define activities as a set of qualitatively interesting interactions between people, objects and the environment. Accurate activity recognition is a desirable task in many scenarios such as surveillance, smart environments, robotic vision etc. In the domain of robotic vision specifically, there is now an increasing interest in autonomous robots that are able to operate without human intervention for long periods of time. The goal of this research is to build activity recognition approaches for such systems that are able to model and recognise simple short activities as well as complex longer activities arising from long-term autonomous operation of intelligent systems. The research makes the following key contributions: 1. We present a qualitative and quantitative representation to model simple activities as observed by autonomous systems. 2. We present a hierarchical framework to efficiently model complex activities that comprise of many sub-activities at varying levels of granularity. Simple activities are modelled using a discriminative model where a combined feature space, consisting of qualitative and quantitative spatio-temporal features, is generated in order to encode various aspects of the activity. Qualitative features are computed using qualitative spatio-temporal relations between human subjects and objects in order to abstractly represent the simple activity. Unlike current state-of-the-art approaches, our approach uses significantly fewer assumptions and does not require any knowledge about object types, their affordances, or the constituent activities of an activity. The optimal and most discriminating features are then extracted, using an entropy-based feature selection process, to best represent the training data. A novel approach for building models of complex long-term activities is presented as well. The proposed approach builds a hierarchical activity model from mark-up of activities acquired from multiple annotators in a video corpus. Multiple human annotators identify activities at different levels of conceptual granularity. Our method automatically infers a ‘part-of’ hierarchical activity model from this data using semantic similarity of textual annotations and temporal consistency. We then consolidate hierarchical structures learned from different training videos into a generalised hierarchical model represented as an extended grammar describing the over all activity. We then describe an inference mechanism to interpret new instances of activities. Simple short activity classes are first recognised using our previously learned generalised model. Given a test video, simple activities are detected as a stream of temporally complex low-level actions. We then use the learned extended grammar to infer the higher-level activities as a hierarchy over the low-level action input stream. We make use of three publicly available datasets to validate our two approaches of modelling simple to complex activities. These datasets have been annotated by multiple annotators through crowd-sourcing and in-house annotations. They consist of daily activity videos such as ‘cleaning microwave’, ‘having lunch in a restaurant’, ‘working in an office’ etc. The activities in these datasets have all been marked up at multiple levels of abstraction by multiple annotators, however no information on the ‘part-of’ relationship between activities is provided. The complexity of the videos and their annotations allows us to demonstrate the effectiveness of the proposed methods

Explainable shared control in assistive robotics

Shared control plays a pivotal role in designing assistive robots to complement human capabilities during everyday tasks. However, traditional shared control relies on users forming an accurate mental model of expected robot behaviour. Without this accurate mental image, users may encounter confusion or frustration whenever their actions do not elicit the intended system response, forming a misalignment between the respective internal models of the robot and human. The Explainable Shared Control paradigm introduced in this thesis attempts to resolve such model misalignment by jointly considering assistance and transparency. There are two perspectives of transparency to Explainable Shared Control: the human's and the robot's. Augmented reality is presented as an integral component that addresses the human viewpoint by visually unveiling the robot's internal mechanisms. Whilst the robot perspective requires an awareness of human "intent", and so a clustering framework composed of a deep generative model is developed for human intention inference. Both transparency constructs are implemented atop a real assistive robotic wheelchair and tested with human users. An augmented reality headset is incorporated into the robotic wheelchair and different interface options are evaluated across two user studies to explore their influence on mental model accuracy. Experimental results indicate that this setup facilitates transparent assistance by improving recovery times from adverse events associated with model misalignment. As for human intention inference, the clustering framework is applied to a dataset collected from users operating the robotic wheelchair. Findings from this experiment demonstrate that the learnt clusters are interpretable and meaningful representations of human intent. This thesis serves as a first step in the interdisciplinary area of Explainable Shared Control. The contributions to shared control, augmented reality and representation learning contained within this thesis are likely to help future research advance the proposed paradigm, and thus bolster the prevalence of assistive robots.Open Acces

Automating Abell's theory of comparative narratives

The purpose of this thesis is to demonstrate the progress that has been made towards the goal of producing a prototype computer model of Abell's Theory of Comparative Narratives, and subsequently, designing metrics to rigorously measure Abell's concept of 'closeness' of texts. The production of such a model does not simply involve the mechanical (though distinctly non-trivial) transference of Abell's theory from paper to machine; various facets of the theory are not of a sufficiently high specification for a computer model and the fulfilment of such a computer model requires attention to these areas, specifically: i) a repeatable method of comparing the structures of individual events; ii) a consistent procedure of comparing the overall structure of a pair of texts, following on from Abell's basic concept of paths of social determination. iii) metrics to demonstrate that the solutions proposed do indeed address the shortcomings of Abell's theory. In order to preserve the qualitative nature of the theory and to demonstrate its potential real-world uses, the computer model attempts to avoid complex mathematics as far as possible and to produce transparent, non-expert results

Recognition of complex human activities in multimedia streams using machine learning and computer vision

Modelling human activities observed in multimedia streams as temporal sequences of their constituent actions has been the object of much research effort in recent years. However, most of this work concentrates on tasks where the action vocabulary is relatively small and/or each activity can be performed in a limited number of ways. In this Thesis, a novel and robust framework for modelling and analysing composite, prolonged activities arising in tasks which can be effectively executed in a variety of ways is proposed. Additionally, the proposed framework is designed to handle cognitive tasks, which cannot be captured using conventional types of sensors. It is shown that the proposed methodology is able to efficiently analyse and recognise complex activities arising in such tasks and also detect potential errors in their execution. To achieve this, a novel activity classification method comprising a feature selection stage based on the novel Key Actions Discovery method and a classification stage based on the combination of Random Forests and Hierarchical Hidden Markov Models is introduced. Experimental results captured in several scenarios arising from real-life applications, including a novel application to a bridge design problem, show that the proposed framework offers higher classification accuracy compared to current activity identification schemes

Modeling Memes: A Memetic View of Affordance Learning

This research employed systems social science inquiry to build a synthesis model that would be useful for modeling meme evolution. First, a formal definition of memes was proposed that balanced both ontological adequacy and empirical observability. Based on this definition, a systems model for meme evolution was synthesized from Shannon Information Theory and elements of Bandura\u27s Social Cognitive Learning Theory. Research in perception, social psychology, learning, and communication were incorporated to explain the cognitive and environmental processes guiding meme evolution. By extending the PMFServ cognitive architecture, socio-cognitive agents were created who could simulate social learning of Gibson affordances. The PMFServ agent based model was used to examine two scenarios: a simulation to test for potential memes inside the Stanford Prison Experiment and a simulation of pro-US and anti-US meme competition within the fictional Hamariyah Iraqi village. The Stanford Prison Experiment simulation was designed, calibrated, and tested using the original Stanford Prison Experiment archival data. This scenario was used to study potential memes within a real-life context. The Stanford Prison Experiment simulation was complemented by internal and external validity testing. The Hamariyah Iraqi village was used to analyze meme competition in a fictional village based upon US Marine Corps human terrain data. This simulation demonstrated how the implemented system can infer the personality traits and contextual factors that cause certain agents to adopt pro-US or anti-US memes, using Gaussian mixture clustering analysis and cross-cluster analysis. Finally, this research identified significant gaps in empirical science with respect to studying memes. These roadblocks and their potential solutions are explored in the conclusions of this work

A Risk And Trust Security Framework For The Pervasive Mobile Environment

A pervasive mobile computing environment is typically composed of multiple fixed and mobile entities that interact autonomously with each other with very little central control. Many of these interactions may occur between entities that have not interacted with each other previously. Conventional security models are inadequate for regulating access to data and services, especially when the identities of a dynamic and growing community of entities are not known in advance. In order to cope with this drawback, entities may rely on context data to make security and trust decisions. However, risk is introduced in this process due to the variability and uncertainty of context information. Moreover, by the time the decisions are made, the context data may have already changed and, in which case, the security decisions could become invalid.With this in mind, our goal is to develop mechanisms or models, to aid trust decision-making by an entity or agent (the truster), when the consequences of its decisions depend on context information from other agents (the trustees). To achieve this, in this dissertation, we have developed ContextTrust a framework to not only compute the risk associated with a context variable, but also to derive a trust measure for context data producing agents. To compute the context data risk, ContextTrust uses Monte Carlo based method to model the behavior of a context variable. Moreover, ContextTrust makes use of time series classifiers and other simple statistical measures to derive an entity trust value.We conducted empirical analyses to evaluate the performance of ContextTrust using two real life data sets. The evaluation results show that ContextTrust can be effective in helping entities render security decisions

Considerations in designing a cybernetic simple 'learning' model; and an overview of the problem of modelling learning

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Learning is viewed as a central feature of living systems and must be manifested in any artifact that claims to exhibit general intelligence. The central aims of the thesis are twofold: (1) - To review and critically assess the empirical and theoretical aspects of learning as have been addressed in a multitude of disciplines, with the aim of extracting fundamental features and elements. (2) - To develop a more systematic approach to the cybernetic modelling of learning than has been achieved hitherto. In pursuit of aim (1) above the following discussions are included: Historical and Philosophical backgrounds; Natural learning, both physiological and psychological aspects; Hierarchies of learning identified in the evolutionary, functional and developmental senses; An extensive section on the general problem of modelling of learning and the formal tools, is included as a link between aims (1) and (2). Following this a systematic and historically oriented study of cybernetic and other related approaches to the problem of modelling of learning is presented. This then leads to the development of a state-of-the-art general purpose experimental cybernetic learning model. The programming and use of this model is also fully described, including an elaborate scheme for the manifestation of simple learning

Genetic evolution and equivalence of some complex systems: fractals, cellular automata and lindenmayer systems

Tesis doctoral inédita leída en la Universidad Autónoma de Madrid. Escuela Politécnica Superior, Departamento de Ingeniería informática.26-04-200