AnimGAN: A Spatiotemporally-Conditioned Generative Adversarial Network for Character Animation

Producing realistic character animations is one of the essential tasks in human-AI interactions. Considered as a sequence of poses of a humanoid, the task can be considered as a sequence generation problem with spatiotemporal smoothness and realism constraints. Additionally, we wish to control the behavior of AI agents by giving them what to do and, more specifically, how to do it. We proposed a spatiotemporally-conditioned GAN that generates a sequence that is similar to a given sequence in terms of semantics and spatiotemporal dynamics. Using LSTM-based generator and graph ConvNet discriminator, this system is trained end-to-end on a large gathered dataset of gestures, expressions, and actions. Experiments showed that compared to traditional conditional GAN, our method creates plausible, realistic, and semantically relevant humanoid animation sequences that match user expectations.Comment: Submitted to ICIP 202

A Whole-Body Pose Taxonomy for Loco-Manipulation Tasks

Exploiting interaction with the environment is a promising and powerful way to enhance stability of humanoid robots and robustness while executing locomotion and manipulation tasks. Recently some works have started to show advances in this direction considering humanoid locomotion with multi-contacts, but to be able to fully develop such abilities in a more autonomous way, we need to first understand and classify the variety of possible poses a humanoid robot can achieve to balance. To this end, we propose the adaptation of a successful idea widely used in the field of robot grasping to the field of humanoid balance with multi-contacts: a whole-body pose taxonomy classifying the set of whole-body robot configurations that use the environment to enhance stability. We have revised criteria of classification used to develop grasping taxonomies, focusing on structuring and simplifying the large number of possible poses the human body can adopt. We propose a taxonomy with 46 poses, containing three main categories, considering number and type of supports as well as possible transitions between poses. The taxonomy induces a classification of motion primitives based on the pose used for support, and a set of rules to store and generate new motions. We present preliminary results that apply known segmentation techniques to motion data from the KIT whole-body motion database. Using motion capture data with multi-contacts, we can identify support poses providing a segmentation that can distinguish between locomotion and manipulation parts of an action.Comment: 8 pages, 7 figures, 1 table with full page figure that appears in landscape page, 2015 IEEE/RSJ International Conference on Intelligent Robots and System

Analyzing Whole-Body Pose Transitions in Multi-Contact Motions

When executing whole-body motions, humans are able to use a large variety of support poses which not only utilize the feet, but also hands, knees and elbows to enhance stability. While there are many works analyzing the transitions involved in walking, very few works analyze human motion where more complex supports occur. In this work, we analyze complex support pose transitions in human motion involving locomotion and manipulation tasks (loco-manipulation). We have applied a method for the detection of human support contacts from motion capture data to a large-scale dataset of loco-manipulation motions involving multi-contact supports, providing a semantic representation of them. Our results provide a statistical analysis of the used support poses, their transitions and the time spent in each of them. In addition, our data partially validates our taxonomy of whole-body support poses presented in our previous work. We believe that this work extends our understanding of human motion for humanoids, with a long-term objective of developing methods for autonomous multi-contact motion planning.Comment: 8 pages, IEEE-RAS International Conference on Humanoid Robots (Humanoids) 201

Is hugging a robot weird? Investigating the influence of robot appearance on users' perception of hugging

Humanoid robots are expected to be able to communicate with humans using physical interaction, including hug, which is a common gesture of affection. In order to achieve that, their physical embodiment has to be carefully planned, as a user-friendly design will facilitate interaction and minimise repulsion. In this paper, we investigate the effect of manipulating the visual/tactile appearance of a robot, covering wires and metallic parts with clothes, and the auditory effect by enabling or disabling the connector of the hand. The experiment consists in a hugging interaction between the participants and the humanoid robot ARMAR-IIIb. Results after participation of 24 subjects confirm the positive effect from using clothes to modify the appearance and the negative effect of noise and vibration

Temporal context influences the perceived duration of everyday actions:Assessing the ecological validity of lab-based timing phenomena

Timing is key to accurate performance, for example when learning a new complex sequence by mimicry. However, most timing research utilizes artificial tasks and simple stimuli with clearly marked onset and offset cues. Here we address the question whether existing interval timing findings generalize to real-world timing tasks. In this study, animated video clips of a person performing different everyday actions were presented and participants had to reproduce the main action’s duration. Although reproduced durations are more variable then observed in laboratory studies, the data adheres to two interval timing laws: Relative timing sensitivity is constant across durations (scalar property), and the subjective duration of a previous action influenced the current action’s perceived duration (temporal context effect). Taken together, this demonstrates that laboratory findings generalize, and paves the way for studying interval timing as a component of complex, everyday cognitive performance

Modelo de Rede Neural para avaliação desportiva

Trata-se de uma pesquisa que objetiva comprovar a possibilidade de utilizar um Modelo em Rede Neural capaz de avaliar o movimento desportivo. O diferencial deste estudo encontra-se no fato de a máquina-servidor ser totalmente em cloud, o que torna viável sua futura utilização por dispositivos mobile devido ao não comprometimento da capacidade de processamento destes. Outro fato relevante é o emprego de duas Redes Neurais (Convolucional e Recorrente) na análise do movimento desportivo. Quanto à metodologia investigativa, este trabalho tem por alicerce uma revisão bibliográfica sobre Rede Neurais e estimativa de pose humana. Isso significa que a fundamentação teórica foi desenvolvida tendo por suporte estudos já realizados e publicados sobre a temática. Como resultado, conclui-se que a utilização de redes convolucionais para a análise de estimativa de pose possui uma acurácia satisfatória, mas que carece de tratamento de ruídos para que a análise da execução do movimento desportivo possa ser feita de fato

Analyzing Whole-Body Pose Transitions in Multi-Contact Motions

Abstract-When executing whole-body motions, humans are able to use a large variety of support poses which not only utilize the feet, but also hands, knees and elbows to enhance stability. While there are many works analyzing the transitions involved in walking, very few works analyze human motion where more complex supports occur. In this work, we analyze complex support pose transitions in human motion involving locomotion and manipulation tasks (loco-manipulation). We have applied a method for the detection of human support contacts from motion capture data to a largescale dataset of loco-manipulation motions involving multicontact supports, providing a semantic representation of them. Our results provide a statistical analysis of the used support poses, their transitions and the time spent in each of them. In addition, our data partially validates our taxonomy of wholebody support poses presented in our previous work. We believe that this work extends our understanding of human motion for humanoids, with a long-term objective of developing methods for autonomous multi-contact motion planning

Análise cinemática automática usando openpose e dynamic time warping com aplicações no remo

Trabalho de Conclusão de Curso (graduação)—Universidade de Brasília, Faculdade UnB Gama, Engenharia Eletrônica, 2019.Este trabalho propõe um sistema de baixo custo para analisar automaticamente parâmetros cinemáticos no remo, a partir da captura e processamento de vídeo, usando uma única câmera RGB e sem a necessidade de marcadores no corpo do indivíduo. As coordenadas das articulações são estimadas a cada frame usando a API da OpenPose em conjunto com um filtro offline para contornar as possíveis perdas de frames e oscilações na trajetória. Os ângulos das articulações são obtidos por meio das coordenadas em pixels das articulações estimadas. Suas trajetórias são, então, avaliadas utilizando uma técnica computacional chamada Dynamic Time Warping, a qual realiza uma comparação entre duas séries temporais, uma denominada referência e a outra, alvo. A série de referência consiste em um padrão de remada a ser seguido e é usada como base para avaliar a série alvo. No teste do sistema compara-se cada remada em um treino de cinco minutos de um remador iniciante com uma remada de referência, executada por um remador profissional. Além disso, avalia-se um treino também de cinco minutos do mesmo remador profissional para conferir a consistência em sua própria remada. Por fim, todas as métricas cinemáticas extraídas são exibidas em uma interface para monitorar o movimento do remador e fornecer um feedback. A abordagem proposta permite a análise automática de sessões de treinamento gravadas com câmera simples, e pode ser útil para auxiliar na melhoria de movimento de remadores, principalmente, iniciantes.This work proposes a low cost system to automatically analyze kinematic parameters in rowing, using video capture and processing, with a single RGB camera and without the need for markers on the individual’s body. The coordinates of the joints are estimated in each frame using the OpenPose API together with an offline filter to overcome frame loss and oscillations in the trajectories. The joint angles are obtained by means of the pixel coordinates from the estimated joints. Their trajectories are then evaluated using a computational technique named Dynamic Time Warping, which performs a comparison between two time series, one denominated reference and the other, target. The reference series consists of a rowing pattern to be followed and it is used as basis to evaluate the target series. The system test compares each stroke in a five-minute workout by a novice rower with a reference stroke, executed by a professional rower. In addition, a five-minute workout by the same professional rower is evaluated for consistency in his own stroke. Finally, all extracted kinematic metrics are displayed in an interface to monitor rower movement and provide feedback. The proposed approach allows automatic analysis of simple camera recorded training sessions, and could be useful to assist in improving the movement of rowers, especially unexperienced

Análise cinemática automática usando OpenPose e Dynamic Time Warping com aplicações no remo

Trabalho de Conclusão de Curso (graduação)—Universidade de Brasília, Faculdade UnB Gama, 2019.Este trabalho propõe um sistema de baixo custo para analisar automaticamente parâmetros cinemáticos no remo, a partir da captura e processamento de vídeo, usando uma única câmera RGB e sem a necessidade de marcadores no corpo do indivíduo. As coordenadas das articulações são estimadas a cada frame usando a API da OpenPose em conjunto com um filtro offline para contornar as possíveis perdas de frames e oscilações na trajetória. Os ângulos das articulações são obtidos por meio das coordenadas em pixels das articulações estimadas. Suas trajetórias são, então, avaliadas utilizando uma técnica computacional chamada Dynamic Time Warping, a qual realiza uma comparação entre duas séries temporais, uma denominada referência e a outra, alvo. A série de referência consiste em um padrão de remada a ser seguido e é usada como base para avaliar a série alvo. No teste do sistema compara-se cada remada em um treino de cinco minutos de um remador iniciante com uma remada de referência, executada por um remador profissional. Além disso, avalia-se um treino também de cinco minutos do mesmo remador profissional para conferir a consistência em sua própria remada. Por fim, todas as métricas cinemáticas extraídas são exibidas em uma interface para monitorar o movimento do remador e fornecer um feedback. A abordagem proposta permite a análise automática de sessões de treinamento gravadas com câmera simples, e pode ser útil para auxiliar na melhoria de movimento de remadores, principalmente, iniciantes.This work proposes a low cost system to automatically analyze kinematic parameters in rowing, using video capture and processing, with a single RGB camera and without the need for markers on the individual’s body. The coordinates of the joints are estimated in each frame using the OpenPose API together with an offline filter to overcome frame loss and oscillations in the trajectories. The joint angles are obtained by means of the pixel coordinates from the estimated joints. Their trajectories are then evaluated using a computational technique named Dynamic Time Warping, which performs a comparison between two time series, one denominated reference and the other, target. The reference series consists of a rowing pattern to be followed and it is used as basis to evaluate the target series. The system test compares each stroke in a five-minute workout by a novice rower with a reference stroke, executed by a professional rower. In addition, a five-minute workout by the same professional rower is evaluated for consistency in his own stroke. Finally, all extracted kinematic metrics are displayed in an interface to monitor rower movement and provide feedback. The proposed approach allows automatic analysis of simple camera recorded training sessions, and could be useful to assist in improving the movement of rowers, especially unexperienced