Key body pose detection and movement assessment of fitness performances

Motion segmentation plays an important role in human motion analysis. Understanding the intrinsic features of human activities represents a challenge for modern science. Current solutions usually involve computationally demanding processing and achieve the best results using expensive, intrusive motion capture devices. In this thesis, research has been carried out to develop a series of methods for affordable and effective human motion assessment in the context of stand-up physical exercises. The objective of the research was to tackle the needs for an autonomous system that could be deployed in nursing homes or elderly people's houses, as well as rehabilitation of high profile sport performers. Firstly, it has to be designed so that instructions on physical exercises, especially in the case of elderly people, can be delivered in an understandable way. Secondly, it has to deal with the problem that some individuals may find it difficult to keep up with the programme due to physical impediments. They may also be discouraged because the activities are not stimulating or the instructions are hard to follow. In this thesis, a series of methods for automatic assessment production, as a combination of worded feedback and motion visualisation, is presented. The methods comprise two major steps. First, a series of key body poses are identified upon a model built by a multi-class classifier from a set of frame-wise features extracted from the motion data. Second, motion alignment (or synchronisation) with a reference performance (the tutor) is established in order to produce a second assessment model. Numerical assessment, first, and textual feedback, after, are delivered to the user along with a 3D skeletal animation to enrich the assessment experience. This animation is produced after the demonstration of the expert is transformed to the current level of performance of the user, in order to help encourage them to engage with the programme. The key body pose identification stage follows a two-step approach: first, the principal components of the input motion data are calculated in order to reduce the dimensionality of the input. Then, candidates of key body poses are inferred using multi-class, supervised machine learning techniques from a set of training samples. Finally, cluster analysis is used to refine the result. Key body pose identification is guaranteed to be invariant to the repetitiveness and symmetry of the performance. Results show the effectiveness of the proposed approach by comparing it against Dynamic Time Warping and Hierarchical Aligned Cluster Analysis. The synchronisation sub-system takes advantage of the cyclic nature of the stretches that are part of the stand-up exercises subject to study in order to remove out-of-sequence identified key body poses (i.e., false positives). Two approaches are considered for performing cycle analysis: a sequential, trivial algorithm and a proposed Genetic Algorithm, with and without prior knowledge on cyclic sequence patterns. These two approaches are compared and the Genetic Algorithm with prior knowledge shows a lower rate of false positives, but also a higher false negative rate. The GAs are also evaluated with randomly generated periodic string sequences. The automatic assessment follows a similar approach to that of key body pose identification. A multi-class, multi-target machine learning classifier is trained with features extracted from previous motion alignment. The inferred numerical assessment levels (one per identified key body pose and involved body joint) are translated into human-understandable language via a highly-customisable, context-free grammar. Finally, visual feedback is produced in the form of a synchronised skeletal animation of both the user's performance and the tutor's. If the user's performance is well below a standard then an affine offset transformation of the skeletal motion data series to an in-between performance is performed, in order to prevent dis-encouragement from the user and still provide a reference for improvement. At the end of this thesis, a study of the limitations of the methods in real circumstances is explored. Issues like the gimbal lock in the angular motion data, lack of accuracy of the motion capture system and the escalation of the training set are discussed. Finally, some conclusions are drawn and future work is discussed

Cognitive and Web Phenotypes of the Western Black Widow Spider

Animals with brains create mental representations of their environment and their own position within it, and use these representations to make decisions. The information used to create mental representations, and how animals use representations to make decisions, are functions of an animal’s evolution and ecology. Mental representations can be as simple as remembering the direction and distance home, or as detailed as a human’s mental map of their own home. I used behavior assays designed to reveal the contents of mental representations to investigate how a web spider creates representation of its environment and objects within it, and uses these representations to guide decision making. Studies of how animals create internal representations have focused largely on understanding how animals with acute vision form representations based on visual information. I used western black widow spiders (Latrodectus hesperus) in my experiments, because web spiders have poor vision and rely largely on sensing vibrations transmitted through their webs. I first explored whether black widows form memories of where they capture prey on the web and whether they alter future web building behavior in response to past differences in prey capture location. I found that black widows did not adjust their web architecture in response to past prey capture experience. However, web architecture differed between spider sexes and spider families, and spider families differed in their response to past prey capture experience. I next investigated whether black widows use internal representations when navigating their webs. I used assays in which I displaced spiders on their webs and observed their attempts to return home. I found that black widows used representations—in the form of path integration vectors that contain information about a spider’s distance and direction from home—when navigating. I next directed my attention toward whether black widows direct their attention inward toward their internal representations. This ability is a defining feature of basic consciousness, and to my knowledge would be the first direct evidence of consciousness in an invertebrate. I designed a novel assay of inwardly directed attention by manipulating whether a spider’s representation matched its currently occupied web or not and providing spiders with a salient prey cue that they would only ignore if they were distracted by the mismatch between internal representation and external web. Black widows did sometimes become distracted by such a mismatch, and thus possessed basic consciousness. The assay I used to detect consciousness in black widows is generalizable and could be used to test for consciousness in myriad animals. Finally, I explored whether black widows store specific information about their captured prey in memory. I used an assay of searching effort to determine whether black widows remember prey capture location and relative prey size. Black widows formed memories of captured prey, regardless of prey capture site. However, black widows only differed their search effort in response to changes in relative prey size when they had captured prey on the floor of their enclosure. This could indicate that either black widows only formed memories of prey size after capturing prey on the floor or that black widows only used the remembered prey size to make decisions after capturing prey on the floor, but the assay I used could not differentiate between the two possibilities. My behavioral assays revealed that black widows create mental representations of their web and prey, and that they possess basic consciousness. Each of my experiments considered the evolution and ecology of black widows, and demonstrated that investigations of the contents of animal minds that use behavioral assays in conjunction with knowledge about the animal’s ecology, evolution, and neurobiology are especially fruitful

Humanoid Robots

For many years, the human being has been trying, in all ways, to recreate the complex mechanisms that form the human body. Such task is extremely complicated and the results are not totally satisfactory. However, with increasing technological advances based on theoretical and experimental researches, man gets, in a way, to copy or to imitate some systems of the human body. These researches not only intended to create humanoid robots, great part of them constituting autonomous systems, but also, in some way, to offer a higher knowledge of the systems that form the human body, objectifying possible applications in the technology of rehabilitation of human beings, gathering in a whole studies related not only to Robotics, but also to Biomechanics, Biomimmetics, Cybernetics, among other areas. This book presents a series of researches inspired by this ideal, carried through by various researchers worldwide, looking for to analyze and to discuss diverse subjects related to humanoid robots. The presented contributions explore aspects about robotic hands, learning, language, vision and locomotion

Computational intelligence approaches to robotics, automation, and control [Volume guest editors]

No abstract available

Proof of Concept For the Use of Motion Capture Technology In Athletic Pedagogy

Visualization has long been an important method for conveying complex information. Where information transfer using written and spoken means might amount to 200-250 words per minute, visual media can often convey information at many times this rate. This makes visualization a potentially important tool for education. Athletic instruction, particularly, can involve communication about complex human movement that is not easily conveyed with written or spoken descriptions. Video based instruction can be problematic since video data can contain too much information, thereby making it more difficult for a student to absorb what is cognitively necessary. The lesson is to present the learner what is needed and not more. We present a novel use of motion capture animation as an educational tool for teaching athletic movements. The advantage of motion capture is its ability to accurately represent real human motion in a minimalist context which removes extraneous information normally found in video. Motion capture animation only displays motion information, not additional information regarding the motion context. Producing an “automated coach” would be too large and difficult a problem to solve within the scope of a Master's thesis but we can perform initial steps including producing a useful software tool which performs data analysis on two motion datasets. We believe such a tool would be beneficial to a human coach as an analysis tool and the work would provide some useful understanding of next important steps towards perhaps someday producing an automated coach

What does the honeybee see? And how do we know?

This book is the only account of what the bee, as an example of an insect, actually detects with its eyes. Bees detect some visual features such as edges and colours, but there is no sign that they reconstruct patterns or put together features to form objects. Bees detect motion but have no perception of what it is that moves, and certainly they do not recognize “things” by their shapes. Yet they clearly see well enough to fly and find food with a minute brain. Bee vision is therefore relevant to the construction of simple artificial visual systems, for example for mobile robots. The surprising conclusion is that bee vision is adapted to the recognition of places, not things. In this volume, Adrian Horridge also sets out the curious and contentious history of how bee vision came to be understood, with an account of a century of neglect of old experimental results, errors of interpretation, sharp disagreements, and failures of the scientific method. The design of the experiments and the methods of making inferences from observations are also critically examined, with the conclusion that scientists are often hesitant, imperfect and misleading, ignore the work of others, and fail to consider alternative explanations. The erratic path to understanding makes interesting reading for anyone with an analytical mind who thinks about the methods of science or the engineering of seeing machines

A Posture Sequence Learning System for an Anthropomorphic Robotic Hand

The paper presents a cognitive architecture for posture learning of an anthropomorphic robotic hand. Our approach is aimed to allow the robotic system to perform complex perceptual operations, to interact with a human user and to integrate the perceptions by a cognitive representation of the scene and the observed actions. The anthropomorphic robotic hand imitates the gestures acquired by the vision system in order to learn meaningful movements, to build its knowledge by different conceptual spaces and to perform complex interaction with the human operator

Categorical organization and machine perception of oscillatory motion patterns

Thesis (Ph.D.)--Massachusetts Institute of Technology, Dept. of Architecture, 2000.Includes bibliographical references (p. 126-132).Many animal behaviors consist of using special patterns of motion for communication, with certain types of movements appearing widely across animal species. Oscillatory motions in particular are quite prevalent, where many of these repetitive movements can be characterized by a simple sinusoidal model with very specific and limited parameter values. We develop a computational model of categorical perception of these motion patterns based on their inherent structural regularity. The model proposes the initial construction of a hierarchical ordering of the model parameters to partition them into sub-categorical specializations. This organization is then used to specify the types and layout of localized computations required for the corresponding visual recognition system. The goal here is to do away with ad hoc motion recognition methods of computer vision, and instead exploit the underlying structural description for a motion category as a motivating mechanism for recognition. We implement this framework and present an analysis of the approach with synthetic and real oscillatory motions, and demonstrate its applicability within an interactive artificial life environment. With this categorical foundation for the description and recognition of related motions, we gain insight into the basis and development of a machine vision system designed to recognize these patterns.by James W. Davis.Ph.D

Action recognition in depth videos using nonparametric probabilistic graphical models

Action recognition involves automatically labelling videos that contain human motion with action classes. It has applications in diverse areas such as smart surveillance, human computer interaction and content retrieval. The recent advent of depth sensing technology that produces depth image sequences has offered opportunities to solve the challenging action recognition problem. The depth images facilitate robust estimation of a human skeleton’s 3D joint positions and a high level action can be inferred from a sequence of these joint positions. A natural way to model a sequence of joint positions is to use a graphical model that describes probabilistic dependencies between the observed joint positions and some hidden state variables. A problem with these models is that the number of hidden states must be fixed a priori even though for many applications this number is not known in advance. This thesis proposes nonparametric variants of graphical models with the number of hidden states automatically inferred from data. The inference is performed in a full Bayesian setting by using the Dirichlet Process as a prior over the model’s infinite dimensional parameter space. This thesis describes three original constructions of nonparametric graphical models that are applied in the classification of actions in depth videos. Firstly, the action classes are represented by a Hidden Markov Model (HMM) with an unbounded number of hidden states. The formulation enables information sharing and discriminative learning of parameters. Secondly, a hierarchical HMM with an unbounded number of actions and poses is used to represent activities. The construction produces a simplified model for activity classification by using logistic regression to capture the relationship between action states and activity labels. Finally, the action classes are modelled by a Hidden Conditional Random Field (HCRF) with the number of intermediate hidden states learned from data. Tractable inference procedures based on Markov Chain Monte Carlo (MCMC) techniques are derived for all these constructions. Experiments with multiple benchmark datasets confirm the efficacy of the proposed approaches for action recognition