Speech-Gesture Mapping and Engagement Evaluation in Human Robot Interaction

A robot needs contextual awareness, effective speech production and complementing non-verbal gestures for successful communication in society. In this paper, we present our end-to-end system that tries to enhance the effectiveness of non-verbal gestures. For achieving this, we identified prominently used gestures in performances by TED speakers and mapped them to their corresponding speech context and modulated speech based upon the attention of the listener. The proposed method utilized Convolutional Pose Machine [4] to detect the human gesture. Dominant gestures of TED speakers were used for learning the gesture-to-speech mapping. The speeches by them were used for training the model. We also evaluated the engagement of the robot with people by conducting a social survey. The effectiveness of the performance was monitored by the robot and it self-improvised its speech pattern on the basis of the attention level of the audience, which was calculated using visual feedback from the camera. The effectiveness of interaction as well as the decisions made during improvisation was further evaluated based on the head-pose detection and interaction survey.Comment: 8 pages, 9 figures, Under review in IRC 201

Human-Robot Interaction for Children with Cerebral Palsy: Reflection and Suggestion for Interactive Scenario Design

AbstractThis paper explains how a humanoid robot NAO can be used as an assistive technology in specific therapy for children with cerebral palsy (CP). The role of the robot is to motivate the children as to keep them engaged in therapy. To achieve this, the robot must have appropriate appearance to be able to establish affective engagement between child and robot. In addition, the robot should exhibit the right therapeutic approach of managing children with CP. How the humanoid robot NAO acts as a tool to assist in improving the outcome of conventional therapy especially by imitation learning will also be explained. Four interactive scenarios in human-robot interaction (HRI) were designed based on the measurement items in Gross Motor Functional Measure (GMFM). The scenarios will then be constructed based on suitability that will be executed by the robot. As a result from the discussions between clinicians, therapists and engineers, four interactive scenarios consists of introductory rapport, sit to stand, body balancing and ball kicking activity have been formulated. The study has been performed in collaboration between the Faculty of Medicine and the Faculty of Mechanical Engineering at the Medical Specialist Centre, Discipline of Rehabilitation Medicine, Faculty of Medicine, UiTM Sungai Buloh, Selangor, Malaysia

Development of a Locomotion and Balancing Strategy for Humanoid Robots

The locomotion ability and high mobility are the most distinguished features of humanoid robots. Due to the non-linear dynamics of walking, developing and controlling the locomotion of humanoid robots is a challenging task. In this thesis, we study and develop a walking engine for the humanoid robot, NAO, which is the official robotic platform used in the RoboCup Spl. Aldebaran Robotics, the manufacturing company of NAO provides a walking module that has disadvantages, such as being a black box that does not provide control of the gait as well as the robot walk with a bent knee. The latter disadvantage, makes the gait unnatural, energy inefficient and exert large amounts of torque to the knee joint. Thus creating a walking engine that produces a quality and natural gait is essential for humanoid robots in general and is a factor for succeeding in RoboCup competition. Humanoids robots are required to walk fast to be practical for various life tasks. However, its complex structure makes it prone to falling during fast locomotion. On the same hand, the robots are expected to work in constantly changing environments alongside humans and robots, which increase the chance of collisions. Several human-inspired recovery strategies have been studied and adopted to humanoid robots in order to face unexpected and avoidable perturbations. These strategies include hip, ankle, and stepping, however, the use of the arms as a recovery strategy did not enjoy as much attention. The arms can be employed in different motions for fall prevention. The arm rotation strategy can be employed to control the angular momentum of the body and help to regain balance. In this master\u27s thesis, I developed a detailed study of different ways in which the arms can be used to enhance the balance recovery of the NAO humanoid robot while stationary and during locomotion. I model the robot as a linear inverted pendulum plus a flywheel to account for the angular momentum change at the CoM. I considered the role of the arms in changing the body\u27s moment of inertia which help to prevent the robot from falling or to decrease the falling impact. I propose a control algorithm that integrates the arm rotation strategy with the on-board sensors of the NAO. Additionally, I present a simple method to control the amount of recovery from rotating the arms. I also discuss the limitation of the strategy and how it can have a negative impact if it was misused. I present simulations to evaluate the approach in keeping the robot stable against various disturbance sources. The results show the success of the approach in keeping the NAO stable against various perturbations. Finally,I adopt the arm rotation to stabilize the ball kick, which is a common reason for falling in the soccer humanoid RoboCup competitions

A reliability-based particle filter for humanoid robot self-localization in Robocup Standard Platform League

This paper deals with the problem of humanoid robot localization and proposes a new method for position estimation that has been developed for the RoboCup Standard Platform League environment. Firstly, a complete vision system has been implemented in the Nao robot platform that enables the detection of relevant field markers. The detection of field markers provides some estimation of distances for the current robot position. To reduce errors in these distance measurements, extrinsic and intrinsic camera calibration procedures have been developed and described. To validate the localization algorithm, experiments covering many of the typical situations that arise during RoboCup games have been developed: ranging from degradation in position estimation to total loss of position (due to falls, ‘kidnapped robot’, or penalization). The self-localization method developed is based on the classical particle filter algorithm. The main contribution of this work is a new particle selection strategy. Our approach reduces the CPU computing time required for each iteration and so eases the limited resource availability problem that is common in robot platforms such as Nao. The experimental results show the quality of the new algorithm in terms of localization and CPU time consumption.This work has been supported by the Spanish Science and Innovation Ministry (MICINN) under the CICYT project COBAMI: DPI2011-28507-C02-01/02. The responsibility for the content remains with the authors.Munera Sánchez, E.; Muñoz Alcobendas, M.; Blanes Noguera, F.; Benet Gilabert, G.; Simó Ten, JE. (2013). A reliability-based particle filter for humanoid robot self-localization in Robocup Standard Platform League. Sensors. 13(11):14954-14983. https://doi.org/10.3390/s131114954S1495414983131

Persuasiveness of social robot ‘Nao’ based on gaze and proximity

Social Robots have widely infiltrated the retail and public space. Mainly, social robots are being utilized across a wide range of scenarios to influence decision making, disseminate information, and act as a signage mechanism, under the umbrella of Persuasive Robots or Persuasive Technology. While there have been several studies in the afore-mentioned area, the effect of non-verbal behaviour on persuasive abilities is generally unexplored. Therefore, in this research, we report whether two key non-verbal attributes, namely proximity and gaze, can elicit persuasively, compliance, and specific personality appeals. For this, we conducted a 2 (eye gaze) x 2 (proximity) between-subjects experiment where participants viewed a video-based scenario of the Nao robot. Our initial results did not reveal any significant results based on the non-verbal attributes. However, perceived compliance and persuasion were significantly correlated with knowledge, responsiveness, and trustworthiness. In conclusion, we discuss how the design of a robot could make it more convincing as extensive marketing and brand promotion companies could use robots to enhance their advertisement operations