Dance Teaching by a Robot: Combining Cognitive and Physical Human-Robot Interaction for Supporting the Skill Learning Process

This letter presents a physical human-robot interaction scenario in which a robot guides and performs the role of a teacher within a defined dance training framework. A combined cognitive and physical feedback of performance is proposed for assisting the skill learning process. Direct contact cooperation has been designed through an adaptive impedance-based controller that adjusts according to the partner's performance in the task. In measuring performance, a scoring system has been designed using the concept of progressive teaching (PT). The system adjusts the difficulty based on the user's number of practices and performance history. Using the proposed method and a baseline constant controller, comparative experiments have shown that the PT presents better performance in the initial stage of skill learning. An analysis of the subjects' perception of comfort, peace of mind, and robot performance have shown a significant difference at the p < .01 level, favoring the PT algorithm.Comment: Presented at IEEE International Conference on Robotics and Automation ICRA-201

Guiding a Human Follower with Interaction Forces: Implications on Physical Human-Robot Interaction

This work challenges the common assumption in physical human-robot interaction (pHRI) that the movement intention of a human user can be simply modeled with dynamic equations relating forces to movements, regardless of the user. Studies in physical human-human interaction (pHHI) suggest that interaction forces carry sophisticated information that reveals motor skills and roles in the partnership and even promotes adaptation and motor learning. In this view, simple force-displacement equations often used in pHRI studies may not be sufficient. To test this, this work measured and analyzed the interaction forces (F) between two humans as the leader guided the blindfolded follower on a randomly chosen path. The actual trajectory of the follower was transformed to the velocity commands (V) that would allow a hypothetical robot follower to track the same trajectory. Then, possible analytical relationships between F and V were obtained using neural network training. Results suggest that while F helps predict V, the relationship is not straightforward, that seemingly irrelevant components of F may be important, that force-velocity relationships are unique to each human follower, and that human neural control of movement may affect the prediction of the movement intent. It is suggested that user-specific, stereotype-free controllers may more accurately decode human intent in pHRI

Marker-less human body part detection, labelling and tracking for human activity recognition

This thesis focuses on the development of a real-time and cost effective marker-less computer vision method for significant body point or part detection (i.e., the head, arm, shoulder, knee, and feet), labelling and tracking, and its application to activity recognition. This work comprises of three parts: significantbody point detection and labelling, significant body point tracking, and activity recognition. Implicit body models are proposed based on human anthropometry, kinesiology, and human vision inspired criteria to detect and label significant body points. The key idea of the proposed method is to fit the knowledge from the implicit body models rather than fitting the predefined models in order to detect and label significant body points. The advantages of this method are that it does not require manual annotation, an explicit fitting procedure, and a training (learning) phase, and it is applicable to humans with different anthropometric proportions. The experimental results show that the proposed method robustly detects and labels significant body points in various activities of two different (low and high) resolution data sets. Furthermore, a Particle Filter with memory and feedback is proposed that combines temporal information of the previous observation and estimation with feedback to track significant body points in occlusion. In addition, in order to overcome the problem presented by the most occluded body part, i.e., the arm, a Motion Flow method is proposed. This method considers the human arm as a pendulum attached to the shoulder joint and defines conjectures to track the arm since it is the most occluded body part. The former method is invoked as default and the latter is used as per a user's choice. The experimental results show that the two proposed methods, i.e., Particle Filter and Motion Flow methods, robustly track significant body points in various activities of the above-mentioned two data sets and also enhance the performance of significant body point detection. A hierarchical relaxed partitioning system is then proposed that employs features extracted from the significant body points for activity recognition when multiple overlaps exist in the feature space. The working principle of the proposed method is based on the relaxed hierarchy (postpone uncertain decisions) and hierarchical strategy (group similar or confusing classes) while partitioning each class at different levels of the hierarchy. The advantages of the proposed method lie in its real-time speed, ease of implementation and extension, and non-intensive training. The experimental results show that it acquires valuable features and outperforms relevant state-of-the-art methods while comparable to other methods, i.e., the holistic and local feature approaches. In this context, the contribution of this thesis is three-fold: Pioneering a method for automated human body part detection and labelling. Developing methods for tracking human body parts in occlusion. Designing a method for robust and efficient human action recognition

Bio-inspired robotic control in underactuation: principles for energy efficacy, dynamic compliance interactions and adaptability.

Biological systems achieve energy efficient and adaptive behaviours through extensive autologous and exogenous compliant interactions. Active dynamic compliances are created and enhanced from musculoskeletal system (joint-space) to external environment (task-space) amongst the underactuated motions. Underactuated systems with viscoelastic property are similar to these biological systems, in that their self-organisation and overall tasks must be achieved by coordinating the subsystems and dynamically interacting with the environment. One important question to raise is: How can we design control systems to achieve efficient locomotion, while adapt to dynamic conditions as the living systems do? In this thesis, a trajectory planning algorithm is developed for underactuated microrobotic systems with bio-inspired self-propulsion and viscoelastic property to achieve synchronized motion in an energy efficient, adaptive and analysable manner. The geometry of the state space of the systems is explicitly utilized, such that a synchronization of the generalized coordinates is achieved in terms of geometric relations along the desired motion trajectory. As a result, the internal dynamics complexity is sufficiently reduced, the dynamic couplings are explicitly characterised, and then the underactuated dynamics are projected onto a hyper-manifold. Following such a reduction and characterization, we arrive at mappings of system compliance and integrable second-order dynamics with the passive degrees of freedom. As such, the issue of trajectory planning is converted into convenient nonlinear geometric analysis and optimal trajectory parameterization. Solutions of the reduced dynamics and the geometric relations can be obtained through an optimal motion trajectory generator. Theoretical background of the proposed approach is presented with rigorous analysis and developed in detail for a particular example. Experimental studies are conducted to verify the effectiveness of the proposed method. Towards compliance interactions with the environment, accurate modelling or prediction of nonlinear friction forces is a nontrivial whilst challenging task. Frictional instabilities are typically required to be eliminated or compensated through efficiently designed controllers. In this work, a prediction and analysis framework is designed for the self-propelled vibro-driven system, whose locomotion greatly relies on the dynamic interactions with the nonlinear frictions. This thesis proposes a combined physics-based and analytical-based approach, in a manner that non-reversible characteristic for static friction, presliding as well as pure sliding regimes are revealed, and the frictional limit boundaries are identified. Nonlinear dynamic analysis and simulation results demonstrate good captions of experimentally observed frictional characteristics, quenching of friction-induced vibrations and satisfaction of energy requirements. The thesis also performs elaborative studies on trajectory tracking. Control schemes are designed and extended for a class of underactuated systems with concrete considerations on uncertainties and disturbances. They include a collocated partial feedback control scheme, and an adaptive variable structure control scheme with an elaborately designed auxiliary control variable. Generically, adaptive control schemes using neural networks are designed to ensure trajectory tracking. Theoretical background of these methods is presented with rigorous analysis and developed in detail for particular examples. The schemes promote the utilization of linear filters in the control input to improve the system robustness. Asymptotic stability and convergence of time-varying reference trajectories for the system dynamics are shown by means of Lyapunov synthesis