A new IMMU-based data glove for hand motion capture with optimized sensor layout

The number of people with hand disabilities caused by stroke is increasing every year. Developing a low-cost and easy-to-use data glove to capture the human hand motion can be used to assess the patient’s hand ability in home environment. While a majority of existing hand motion capture methods are too complex to be used for patients in residential settings. This paper proposes a new sensor layout strategy using the inertial and magnetic measurement units and designs a multi-sensor Kalman data fusion algorithm. The sensor layout strategy is optimized according to the inverse kinematics and the developed hand model, and the number of sensors can be significantly reduced from 12 in conventional systems to 6 in our system with the hand motion being completely and accurately reconstructed. Hand motion capture experiments were conducted on a healthy subject using the developed data glove. The hand motion can be restored completely and the hand gesture can be recognized with an accuracy of 85%. The results of a continuous hand movement indicate an average error under 15% compared with the common glove with full sensors. This new set with optimized sensor layout is promising for lower-cost and residential medical applications

Intention Understanding in Human-Robot Interaction Based on Visual-NLP Semantics

With the rapid development of robotic and AI technology in recent years, human-robot interaction has made great advancement, making practical social impact. Verbal commands are one of the most direct and frequently used means for human-robot interaction. Currently, such technology can enable robots to execute pre-defined tasks based on simple and direct and explicit language instructions, e.g., certain keywords must be used and detected. However, that is not the natural way for human to communicate. In this paper, we propose a novel task-based framework to enable the robot to comprehend human intentions using visual semantics information, such that the robot is able to satisfy human intentions based on natural language instructions (total three types, namely clear, vague, and feeling, are defined and tested). The proposed framework includes a language semantics module to extract the keywords despite the explicitly of the command instruction, a visual object recognition module to identify the objects in front of the robot, and a similarity computation algorithm to infer the intention based on the given task. The task is then translated into the commands for the robot accordingly. Experiments are performed and validated on a humanoid robot with a defined task: to pick the desired item out of multiple objects on the table, and hand over to one desired user out of multiple human participants. The results show that our algorithm can interact with different types of instructions, even with unseen sentence structures

Reliability, accuracy, and minimal detectable difference of a mixed concept marker set for finger kinematic evaluation

The study of finger biomechanics requires special tools for accurately recording finger joint data. A marker set to evaluate finger postures during activities of daily living is needed to understand finger biomechanics in order to improve prosthesis design and clinical interventions. The purpose of this study was to evaluate the reliability of a proposed hand marker set (the Warwick marker set) to capture finger kinematics using motion capture. The marker set consisted of the application of two and three marker clusters to the fingers of twelve participants who participated in the tests across two sessions. Calibration markers were applied using a custom palpation technique. Each participant performed a series of range of motion movements and held a set of objects. Intra and inter-session reliability was calculated as well as Standard Error of Measurement (SEM) and Minimal Detectable Difference (MDD). The findings showed varying levels of intra- and inter-session reliability, ranging from poor to excellent. The SEM and MDD values were lower for the intra-session range of motion and grasp evaluation. The reduced reliability can potentially be attributed to skin artifacts, differences in marker placement, and the inherent kinematic variability of finger motion. The proposed marker set shows potential to assess finger postures and analyse activities of daily living, primarily within the context of single session tests

On the human evaluation of universal audio adversarial perturbations

[EN] Human-machine interaction is increasingly dependent on speech communication, mainly due to the remarkable performance of Machine Learning models in speech recognition tasks. However, these models can be fooled by adversarial examples, which are inputs in-tentionally perturbed to produce a wrong prediction without the changes being noticeable to humans. While much research has focused on developing new techniques to generate adversarial perturbations, less attention has been given to aspects that determine whether and how the perturbations are noticed by humans. This question is relevant since high fool-ing rates of proposed adversarial perturbation strategies are only valuable if the perturba-tions are not detectable. In this paper we investigate to which extent the distortion metrics proposed in the literature for audio adversarial examples, and which are commonly applied to evaluate the effectiveness of methods for generating these attacks, are a reliable mea-sure of the human perception of the perturbations. Using an analytical framework, and an experiment in which 36 subjects evaluate audio adversarial examples according to different factors, we demonstrate that the metrics employed by convention are not a reliable measure of the perceptual similarity of adversarial examples in the audio domain.This work was supported by the Basque Government (PRE_2019_1_0128 predoctoral grant, IT1244-19 and project KK-2020/00049 through the ELKARTEK program); the Spanish Ministry of Economy and Competitiveness MINECO (projects TIN2016-78365-R and PID2019-104966GB-I00); and the Spanish Ministry of Science, Innovation and Universities (FPU19/03231 predoctoral grant). The authors would also like to thank to the Intelligent Systems Group (University of the Basque Country UPV/EHU, Spain) for providing the computational resources needed to develop the project, as well as to all the participants that took part in the experiments