143 research outputs found
Slowness learning for curiosity-driven agents
In the absence of external guidance, how can a robot learn to map the many raw pixels of high-dimensional visual inputs to useful action sequences? I study methods that achieve this by making robots self-motivated (curious) to continually build compact representations of sensory inputs that encode different aspects of the changing environment. Previous curiosity-based agents acquired skills by associating intrinsic rewards with world model improvements, and used reinforcement learning (RL) to learn how to get these intrinsic rewards. But unlike in previous implementations, I consider streams of high-dimensional visual inputs, where the world model is a set of compact low-dimensional representations of the high-dimensional inputs. To learn these representations, I use the slowness learning principle, which states that the underlying causes of the changing sensory inputs vary on a much slower time scale than the observed sensory inputs. The representations learned through the slowness learning principle are called slow features (SFs). Slow features have been shown to be useful for RL, since they capture the underlying transition process by extracting spatio-temporal regularities in the raw sensory inputs. However, existing techniques that learn slow features are not readily applicable to curiosity-driven online learning agents, as they estimate computationally expensive covariance matrices from the data via batch processing. The first contribution called the incremental SFA (IncSFA), is a low-complexity, online algorithm that extracts slow features without storing any input data or estimating costly covariance matrices, thereby making it suitable to be used for several online learning applications. However, IncSFA gradually forgets previously learned representations whenever the statistics of the input change. In open-ended online learning, it becomes essential to store learned representations to avoid re- learning previously learned inputs. The second contribution is an online active modular IncSFA algorithm called the curiosity-driven modular incremental slow feature analysis (Curious Dr. MISFA). Curious Dr. MISFA addresses the forgetting problem faced by IncSFA and learns expert slow feature abstractions in order from least to most costly, with theoretical guarantees. The third contribution uses the Curious Dr. MISFA algorithm in a continual curiosity-driven skill acquisition framework that enables robots to acquire, store, and re-use both abstractions and skills in an online and continual manner. I provide (a) a formal analysis of the working of the proposed algorithms; (b) compare them to the existing methods; and (c) use the iCub humanoid robot to demonstrate their application in real-world environments. These contributions together demonstrate that the online implementations of slowness learning make it suitable for an open-ended curiosity-driven RL agent to acquire a repertoire of skills that map the many raw pixels of high-dimensional images to multiple sets of action sequences
Lâauto-exploration des espaces sensorimoteurs chez les robots
Developmental robotics has begun in the last fifteen years to study robots that havea childhoodâcrawling before trying to run, playing before being usefulâand that are basing their decisions upon a lifelong and embodied experience of the real-world. In this context, this thesis studies sensorimotor explorationâthe discovery of a robotâs own body and proximal environmentâduring the early developmental stages, when no prior experience of the world is available. Specifically, we investigate how to generate a diversity of effects in an unknown environment. This approach distinguishes itself by its lack of user-defined reward or fitness function, making it especially suited for integration in self-sufficient platforms. In a first part, we motivate our approach, formalize the exploration problem, define quantitative measures to assess performance, and propose an architectural framework to devise algorithms. through the extensive examination of a multi-joint arm example, we explore some of the fundamental challenges that sensorimotor exploration faces, such as high-dimensionality and sensorimotor redundancy, in particular through a comparison between motor and goal babbling exploration strategies. We propose several algorithms and empirically study their behaviour, investigating the interactions with developmental constraints, external demonstrations and biologicallyinspired motor synergies. Furthermore, because even efficient algorithms can provide disastrous performance when their learning abilities do not align with the environmentâs characteristics, we propose an architecture that can dynamically discriminate among a set of exploration strategies. Even with good algorithms, sensorimotor exploration is still an expensive propositionâ a problem since robots inherently face constraints on the amount of data they are able to gather; each observation takes a non-negligible time to collect. [...] Throughout this thesis, our core contributions are algorithms description and empirical results. In order to allow unrestricted examination and reproduction of all our results, the entire code is made available. Sensorimotor exploration is a fundamental developmental mechanism of biological systems. By decoupling it from learning and studying it in its own right in this thesis, we engage in an approach that casts light on important problems facing robots developing on their own.La robotique dĂ©veloppementale a entrepris, au courant des quinze derniĂšres annĂ©es,dâĂ©tudier les processus dĂ©veloppementaux, similaires Ă ceux des systĂšmes biologiques,chez les robots. Le but est de crĂ©er des robots qui ont une enfanceâqui rampent avant dâessayer de courir, qui jouent avant de travaillerâet qui basent leurs dĂ©cisions sur lâexpĂ©rience de toute une vie, incarnĂ©s dans le monde rĂ©el.Dans ce contexte, cette thĂšse Ă©tudie lâexploration sensorimotriceâla dĂ©couverte pour un robot de son propre corps et de son environnement procheâpendant les premiers stage du dĂ©veloppement, lorsque quâaucune expĂ©rience prĂ©alable du monde nâest disponible. Plus spĂ©cifiquement, cette thĂšse se penche sur comment gĂ©nĂ©rer une diversitĂ© dâeffets dans un environnement inconnu. Cette approche se distingue par son absence de fonction de rĂ©compense ou de fitness dĂ©finie par un expert, la rendant particuliĂšrement apte Ă ĂȘtre intĂ©grĂ©e sur des robots auto-suffisants.Dans une premiĂšre partie, lâapproche est motivĂ©e et le problĂšme de lâexploration est formalisĂ©, avec la dĂ©finition de mesures quantitatives pour Ă©valuer le comportement des algorithmes et dâun cadre architectural pour la crĂ©ation de ces derniers. Via lâexamen dĂ©taillĂ© de lâexemple dâun bras robot Ă multiple degrĂ©s de libertĂ©, la thĂšse explore quelques unes des problĂ©matiques fondamentales que lâexploration sensorimotrice pose, comme la haute dimensionnalitĂ© et la redondance sensorimotrice. Cela est fait en particulier via la comparaison entre deux stratĂ©gies dâexploration: le babillage moteur et le babillage dirigĂ© par les objectifs. Plusieurs algorithmes sont proposĂ©s tour Ă tour et leur comportement est Ă©valuĂ© empiriquement, Ă©tudiant les interactions qui naissent avec les contraintes dĂ©veloppementales, les dĂ©monstrations externes et les synergies motrices. De plus, parce que mĂȘme des algorithmes efficaces peuvent se rĂ©vĂ©ler terriblement inefficaces lorsque leurs capacitĂ©s dâapprentissage ne sont pas adaptĂ©s aux caractĂ©ristiques de leur environnement, une architecture est proposĂ©e qui peut dynamiquement choisir la stratĂ©gie dâexploration la plus adaptĂ©e parmi un ensemble de stratĂ©gies. Mais mĂȘme avec de bons algorithmes, lâexploration sensorimotrice reste une entreprise coĂ»teuseâun problĂšme important, Ă©tant donnĂ© que les robots font face Ă des contraintes fortes sur la quantitĂ© de donnĂ©es quâils peuvent extraire de leur environnement;chaque observation prenant un temps non-nĂ©gligeable Ă rĂ©cupĂ©rer. [...] Ă travers cette thĂšse, les contributions les plus importantes sont les descriptions algorithmiques et les rĂ©sultats expĂ©rimentaux. De maniĂšre Ă permettre la reproduction et la rĂ©examination sans contrainte de tous les rĂ©sultats, lâensemble du code est mis Ă disposition. Lâexploration sensorimotrice est un mĂ©canisme fondamental du dĂ©veloppement des systĂšmes biologiques. La sĂ©parer dĂ©libĂ©rĂ©ment des mĂ©canismes dâapprentissage et lâĂ©tudier pour elle-mĂȘme dans cette thĂšse permet dâĂ©clairer des problĂšmes importants que les robots se dĂ©veloppant seuls seront amenĂ©s Ă affronter
Novel methods for posture-based human action recognition and activity anomaly detection
PhD ThesisArti cial Intelligence (AI) for Human Action Recognition (HAR) and Human
Activity Anomaly Detection (HAAD) is an active and exciting research
eld. Video-based HAR aims to classify human actions and video-based
HAAD aims to detect abnormal human activities within data. However, a
human is an extremely complex subject and a non-rigid object in the video,
which provides great challenges for Computer Vision and Signal Processing.
Relevant applications elds are surveillance and public monitoring, assisted
living, robotics, human-to-robot interaction, prosthetics, gaming, video captioning,
and sports analysis.
The focus of this thesis is on the posture-related HAR and HAAD. The
aim is to design computationally-e cient, machine and deep learning-based
HAR and HAAD methods which can run in multiple humans monitoring
scenarios.
This thesis rstly contributes two novel 3D Histogram of Oriented Gradient
(3D-HOG) driven frameworks for silhouette-based HAR. The 3D-HOG
state-of-the-art limitations, e.g. unweighted local body areas based processing
and unstable performance over di erent training rounds, are addressed.
The proposed methods achieve more accurate results than the
baseline, outperforming the state-of-the-art. Experiments are conducted on
publicly available datasets, alongside newly recorded data.
This thesis also contributes a new algorithm for human poses-based
HAR. In particular, the proposed human poses-based HAR is among the
rst, few, simultaneous attempts which have been conducted at the time.
The proposed HAR algorithm, named ActionXPose, is based on Convolutional
Neural Networks and Long Short-Term Memory. It turns out to be
more reliable and computationally advantageous when compared to human
silhouette-based approaches. The ActionXPose's
exibility also allows crossdatasets
processing and more robustness to occlusions scenarios. Extensive
evaluation on publicly available datasets demonstrates the e cacy of ActionXPose
over the state-of-the-art. Moreover, newly recorded data, i.e.
Intelligent Sensing Lab Dataset (ISLD), is also contributed and exploited to
further test ActionXPose in real-world, non-cooperative scenarios.
The last set of contributions in this thesis regards pose-driven, combined
HAR and HAAD algorithms. Motivated by ActionXPose achievements, this
thesis contributes a new algorithm to simultaneously extract deep-learningbased
features from human-poses, RGB Region of Interests (ROIs) and
detected objects positions. The proposed method outperforms the stateof-
the-art in both HAR and HAAD. The HAR performance is extensively
tested on publicly available datasets, including the contributed ISLD dataset.
Moreover, to compensate for the lack of data in the eld, this thesis
also contributes three new datasets for human-posture and objects-positions
related HAAD, i.e. BMbD, M-BMdD and JBMOPbD datasets
Virtual Reality Games for Motor Rehabilitation
This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any productâs acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion
New deep learning approaches to domain adaptation and their applications in 3D hand pose estimation
This study investigates several methods for using artificial intelligence to give machines the ability to see. It introduced several methods for image recognition that are more accurate and efficient compared to the existing approaches
Towards gestural understanding for intelligent robots
Fritsch JN. Towards gestural understanding for intelligent robots. Bielefeld: UniversitÀt Bielefeld; 2012.A strong driving force of scientific progress in the technical sciences is the quest for systems that assist humans in their daily life and make their life easier and more enjoyable. Nowadays smartphones are probably the most typical instances of such systems. Another class of systems that is getting increasing attention are intelligent robots. Instead of offering a smartphone touch screen to select actions, these systems are intended to offer a more natural human-machine interface to their users. Out of the large range of actions performed by humans, gestures performed with the hands play a very important role especially when humans interact with their direct surrounding like, e.g., pointing to an object or manipulating it. Consequently, a robot has to understand such gestures to offer an intuitive interface. Gestural understanding is, therefore, a key capability on the way to intelligent robots.
This book deals with vision-based approaches for gestural understanding. Over the past two decades, this has been an intensive field of research which has resulted in a variety of algorithms to analyze human hand motions. Following a categorization of different gesture types and a review of other sensing techniques, the design of vision systems that achieve hand gesture understanding for intelligent robots is analyzed. For each of the individual algorithmic steps â hand detection, hand tracking, and trajectory-based gesture recognition â a separate Chapter introduces common techniques and algorithms and provides example methods. The resulting recognition algorithms are considering gestures in isolation and are often not sufficient for interacting with a robot who can only understand such gestures when incorporating the context like, e.g., what object was pointed at or manipulated.
Going beyond a purely trajectory-based gesture recognition by incorporating context is an important prerequisite to achieve gesture understanding and is addressed explicitly in a separate Chapter of this book. Two types of context, user-provided context and situational context, are reviewed and existing approaches to incorporate context for gestural understanding are reviewed. Example approaches for both context types provide a deeper algorithmic insight into this field of research. An overview of recent robots capable of gesture recognition and understanding summarizes the currently realized human-robot interaction quality.
The approaches for gesture understanding covered in this book are manually designed while humans learn to recognize gestures automatically during growing up. Promising research targeted at analyzing developmental learning in children in order to mimic this capability in technical systems is highlighted in the last Chapter completing this book as this research direction may be highly influential for creating future gesture understanding systems
Feature regularization and learning for human activity recognition.
Doctoral Degree. University of KwaZulu-Natal, Durban.Feature extraction is an essential component in the design of human activity
recognition model. However, relying on extracted features alone for learning often makes the model a suboptimal model. Therefore, this research
work seeks to address such potential problem by investigating feature regularization. Feature regularization is used for encapsulating discriminative
patterns that are needed for better and efficient model learning. Firstly, a
within-class subspace regularization approach is proposed for eigenfeatures
extraction and regularization in human activity recognition. In this ap-
proach, the within-class subspace is modelled using more eigenvalues from
the reliable subspace to obtain a four-parameter modelling scheme. This
model enables a better and true estimation of the eigenvalues that are distorted by the small sample size effect. This regularization is done in one
piece, thereby avoiding undue complexity of modelling eigenspectrum differently. The whole eigenspace is used for performance evaluation because
feature extraction and dimensionality reduction are done at a later stage
of the evaluation process. Results show that the proposed approach has
better discriminative capacity than several other subspace approaches for
human activity recognition. Secondly, with the use of likelihood prior probability, a new regularization scheme that improves the loss function of deep
convolutional neural network is proposed. The results obtained from this
work demonstrate that a well regularized feature yields better class discrimination in human activity recognition. The major contribution of the
thesis is the development of feature extraction strategies for determining
discriminative patterns needed for efficient model learning
Human Machine Interaction
In this book, the reader will find a set of papers divided into two sections. The first section presents different proposals focused on the human-machine interaction development process. The second section is devoted to different aspects of interaction, with a special emphasis on the physical interaction
- âŠ