Recognition of human activity and the state of an assembly task using vision and inertial sensor fusion methods

Reliable human machine interfaces is key to accomplishing the goals of Industry 4.0. This work proposes the late fusion of a visual recognition and human action recognition (HAR) classifier. Vision is used to recognise the number of screws assembled into a mock part while HAR from body worn Inertial Measurement Units (IMUs) classifies actions done to assemble the part. Convolutional Neural Network (CNN) methods are used in both modes of classification before various late fusion methods are analysed for prediction of a final state estimate. The fusion methods investigated are mean, weighted average, Support Vector Machine (SVM), Bayesian, Artificial Neural Network (ANN) and Long Short Term Memory (LSTM). The results show the LSTM fusion method to perform best, with accuracy of 93% compared to 81% for IMU and 77% for visual sensing. Development of sensor fusion methods such as these is key to reliable Human Machine Interaction (HMI

Recognition of human activity and the state of an assembly task using vision and inertial sensor fusion methods

Reliable human machine interfaces is key to accomplishing the goals of Industry 4.0. This work proposes the late fusion of a visual recognition and human action recognition (HAR) classifier. Vision is used to recognise the number of screws assembled into a mock part while HAR from body worn Inertial Measurement Units (IMUs) classifies actions done to assemble the part. Convolutional Neural Network (CNN) methods are used in both modes of classification before various late fusion methods are analysed for prediction of a final state estimate. The fusion methods investigated are mean, weighted average, Support Vector Machine (SVM), Bayesian, Artificial Neural Network (ANN) and Long Short Term Memory (LSTM). The results show the LSTM fusion method to perform best, with accuracy of 93% compared to 81% for IMU and 77% for visual sensing. Development of sensor fusion methods such as these is key to reliable Human Machine Interaction (HMI

Artificial co-drivers as a universal enabling technology for future intelligent vehicles and transportation systems

This position paper introduces the concept of artificial “co-drivers” as an enabling technology for future intelligent transportation systems. In Sections I and II, the design principles of co-drivers are introduced and framed within general human–robot interactions. Several contributing theories and technologies are reviewed, specifically those relating to relevant cognitive architectures, human-like sensory-motor strategies, and the emulation theory of cognition. In Sections III and IV, we present the co-driver developed for the EU project interactIVe as an example instantiation of this notion, demonstrating how it conforms to the given guidelines. We also present substantive experimental results and clarify the limitations and performance of the current implementation. In Sections IV and V, we analyze the impact of the co-driver technology. In particular, we identify a range of application fields, showing how it constitutes a universal enabling technology for both smart vehicles and cooperative systems, and naturally sets out a program for future research

Game-Theoretic Safety Assurance for Human-Centered Robotic Systems

In order for autonomous systems like robots, drones, and self-driving cars to be reliably introduced into our society, they must have the ability to actively account for safety during their operation. While safety analysis has traditionally been conducted offline for controlled environments like cages on factory floors, the much higher complexity of open, human-populated spaces like our homes, cities, and roads makes it unviable to rely on common design-time assumptions, since these may be violated once the system is deployed. Instead, the next generation of robotic technologies will need to reason about safety online, constructing high-confidence assurances informed by ongoing observations of the environment and other agents, in spite of models of them being necessarily fallible.This dissertation aims to lay down the necessary foundations to enable autonomous systems to ensure their own safety in complex, changing, and uncertain environments, by explicitly reasoning about the gap between their models and the real world. It first introduces a suite of novel robust optimal control formulations and algorithmic tools that permit tractable safety analysis in time-varying, multi-agent systems, as well as safe real-time robotic navigation in partially unknown environments; these approaches are demonstrated on large-scale unmanned air traffic simulation and physical quadrotor platforms. After this, it draws on Bayesian machine learning methods to translate model-based guarantees into high-confidence assurances, monitoring the reliability of predictive models in light of changing evidence about the physical system and surrounding agents. This principle is first applied to a general safety framework allowing the use of learning-based control (e.g. reinforcement learning) for safety-critical robotic systems such as drones, and then combined with insights from cognitive science and dynamic game theory to enable safe human-centered navigation and interaction; these techniques are showcased on physical quadrotors—flying in unmodeled wind and among human pedestrians—and simulated highway driving. The dissertation ends with a discussion of challenges and opportunities ahead, including the bridging of safety analysis and reinforcement learning and the need to ``close the loop'' around learning and adaptation in order to deploy increasingly advanced autonomous systems with confidence

Artificial co-drivers as a universal enabling technology for future intelligent vehicles and transportation systems

This position paper introduces the concept of artificial “co-drivers” as an enabling technology for future intelligent transportation systems. In Sections I and II, the design principles of co-drivers are introduced and framed within general human–robot interactions. Several contributing theories and technologies are reviewed, specifically those relating to relevant cognitive architectures, human-like sensory-motor strategies, and the emulation theory of cognition. In Sections III and IV, we present the co-driver developed for the EU project interactIVe as an example instantiation of this notion, demonstrating how it conforms to the given guidelines. We also present substantive experimental results and clarify the limitations and performance of the current implementation. In Sections IV and V, we analyze the impact of the co-driver technology. In particular, we identify a range of application fields, showing how it constitutes a universal enabling technology for both smart vehicles and cooperative systems, and naturally sets out a program for future research

Artificial co-drivers as a universal enabling technology for future intelligent vehicles and transportation systems

This position paper introduces the concept of artificial “co-drivers” as an enabling technology for future intelligent transportation systems. In Sections I and II, the design principles of co-drivers are introduced and framed within general human–robot interactions. Several contributing theories and technologies are reviewed, specifically those relating to relevant cognitive architectures, human-like sensory-motor strategies, and the emulation theory of cognition. In Sections III and IV, we present the co-driver developed for the EU project interactIVe as an example instantiation of this notion, demonstrating how it conforms to the given guidelines. We also present substantive experimental results and clarify the limitations and performance of the current implementation. In Sections IV and V, we analyze the impact of the co-driver technology. In particular, we identify a range of application fields, showing how it constitutes a universal enabling technology for both smart vehicles and cooperative systems, and naturally sets out a program for future research