Robust Legged Robot State Estimation Using Factor Graph Optimization

Legged robots, specifically quadrupeds, are becoming increasingly attractive for industrial applications such as inspection. However, to leave the laboratory and to become useful to an end user requires reliability in harsh conditions. From the perspective of state estimation, it is essential to be able to accurately estimate the robot's state despite challenges such as uneven or slippery terrain, textureless and reflective scenes, as well as dynamic camera occlusions. We are motivated to reduce the dependency on foot contact classifications, which fail when slipping, and to reduce position drift during dynamic motions such as trotting. To this end, we present a factor graph optimization method for state estimation which tightly fuses and smooths inertial navigation, leg odometry and visual odometry. The effectiveness of the approach is demonstrated using the ANYmal quadruped robot navigating in a realistic outdoor industrial environment. This experiment included trotting, walking, crossing obstacles and ascending a staircase. The proposed approach decreased the relative position error by up to 55% and absolute position error by 76% compared to kinematic-inertial odometry.Comment: 8 pages, 12 figures. Accepted to RA-L + IROS 2019, July 201

VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera

We present the first real-time method to capture the full global 3D skeletal pose of a human in a stable, temporally consistent manner using a single RGB camera. Our method combines a new convolutional neural network (CNN) based pose regressor with kinematic skeleton fitting. Our novel fully-convolutional pose formulation regresses 2D and 3D joint positions jointly in real time and does not require tightly cropped input frames. A real-time kinematic skeleton fitting method uses the CNN output to yield temporally stable 3D global pose reconstructions on the basis of a coherent kinematic skeleton. This makes our approach the first monocular RGB method usable in real-time applications such as 3D character control---thus far, the only monocular methods for such applications employed specialized RGB-D cameras. Our method's accuracy is quantitatively on par with the best offline 3D monocular RGB pose estimation methods. Our results are qualitatively comparable to, and sometimes better than, results from monocular RGB-D approaches, such as the Kinect. However, we show that our approach is more broadly applicable than RGB-D solutions, i.e. it works for outdoor scenes, community videos, and low quality commodity RGB cameras.Comment: Accepted to SIGGRAPH 201

A survey on human performance capture and animation

With the rapid development of computing technology, three-dimensional (3D) human body models and their dynamic motions are widely used in the digital entertainment industry. Human perfor- mance mainly involves human body shapes and motions. Key research problems include how to capture and analyze static geometric appearance and dynamic movement of human bodies, and how to simulate human body motions with physical e�ects. In this survey, according to main research directions of human body performance capture and animation, we summarize recent advances in key research topics, namely human body surface reconstruction, motion capture and synthesis, as well as physics-based motion sim- ulation, and further discuss future research problems and directions. We hope this will be helpful for readers to have a comprehensive understanding of human performance capture and animatio

Exploring the effectiveness of an augmented reality dressing room

In this paper, we describe our experience with the design of an augmented reality dressing room in which 3D models of a dress are overlaid with a color image from a camera to provide the function of a sort of virtual mirror. In such a way, the customer can move around to understand if a dress suits and fits them well. The project is implemented in Unity 4 Pro in combination with the Microsoft Kinect 2 for the tracking process. Design issues and technical implementation as well as the prospects for further development of the techniques are discussed. To assess the validity of our proposal, we have conducted a user study using 47 participants with different levels of experience with video games and devices used to play them. The empirical method used is qualitative. To this end, we used questionnaire-based surveys. The obtained results suggest that our solution represents a viable means to simulate dressing rooms, and participants in the study found the interaction with our 3D models to be natural for understanding if a dress suits and fits them well. Overall, the participants found our application very useful from a practical point of view

A Sampling Approach to Generating Closely Interacting 3D Pose-pairs from 2D Annotations

We introduce a data-driven method to generate a large number of plausible, closely interacting 3D human pose-pairs, for a given motion category, e.g., wrestling or salsa dance. With much difficulty in acquiring close interactions using 3D sensors, our approach utilizes abundant existing video data which cover many human activities. Instead of treating the data generation problem as one of reconstruction, either through 3D acquisition or direct 2D-to-3D data lifting from video annotations, we present a solution based on Markov Chain Monte Carlo (MCMC) sampling. With a focus on efficient sampling over the space of close interactions, rather than pose spaces, we develop a novel representation called interaction coordinates (IC) to encode both poses and their interactions in an integrated manner. Plausibility of a 3D pose-pair is then defined based on the ICs and with respect to the annotated 2D pose-pairs from video. We show that our sampling-based approach is able to efficiently synthesize a large volume of plausible, closely interacting 3D pose-pairs which provide a good coverage of the input 2D pose-pairs

A survey of real-time crowd rendering

In this survey we review, classify and compare existing approaches for real-time crowd rendering. We first overview character animation techniques, as they are highly tied to crowd rendering performance, and then we analyze the state of the art in crowd rendering. We discuss different representations for level-of-detail (LoD) rendering of animated characters, including polygon-based, point-based, and image-based techniques, and review different criteria for runtime LoD selection. Besides LoD approaches, we review classic acceleration schemes, such as frustum culling and occlusion culling, and describe how they can be adapted to handle crowds of animated characters. We also discuss specific acceleration techniques for crowd rendering, such as primitive pseudo-instancing, palette skinning, and dynamic key-pose caching, which benefit from current graphics hardware. We also address other factors affecting performance and realism of crowds such as lighting, shadowing, clothing and variability. Finally we provide an exhaustive comparison of the most relevant approaches in the field.Peer ReviewedPostprint (author's final draft

Climbing and Walking Robots

With the advancement of technology, new exciting approaches enable us to render mobile robotic systems more versatile, robust and cost-efficient. Some researchers combine climbing and walking techniques with a modular approach, a reconfigurable approach, or a swarm approach to realize novel prototypes as flexible mobile robotic platforms featuring all necessary locomotion capabilities. The purpose of this book is to provide an overview of the latest wide-range achievements in climbing and walking robotic technology to researchers, scientists, and engineers throughout the world. Different aspects including control simulation, locomotion realization, methodology, and system integration are presented from the scientific and from the technical point of view. This book consists of two main parts, one dealing with walking robots, the second with climbing robots. The content is also grouped by theoretical research and applicative realization. Every chapter offers a considerable amount of interesting and useful information

Support tools for 3D game creation

Nowadays, tools for developing videogames are a very important part of the development process in the game industry. Such tools are used to assist game developers in their tasks, allowing them to create functional games while writing a few lines of code. For example, these tools allow the users to import the content for the game, set the game logic, or produce the source code and compile it. There are several tasks and components regarding the development of videogames that may become unproductive, therefore, it’s necessary to automate and/or optimize such tasks. For example, the programming of events or dialogs can be a task that consumes too much time in the development cycle, and a tedious and repetitive task for the programmer. For this reason, the use of tools to support these tasks can be very important to increase productivity and help on the maintenance of the various processes that involve the development of videogames. This dissertation aims to demonstrate the advantages of the use of these kind of tools during the development of videogames, presenting a case study involving the development of a Serious Game entitled Clean World.Atualmente, as ferramentas para o desenvolvimento de jogos são uma parte bastante importante de todo o processo de desenvolvimento. Estas ferramentas servem para assistir os criadores de jogos nas tarefas que realizam, permitindo-lhes a criação de jogos funcionais escrevendo poucas linhas de código. Desenvolver um videojogo sem a utilização de ferramentas especializadas é um processo complexo e que consome bastante tempo, daí a existência de ferramentas que permitem ao utilizador importar os conteúdos para o jogo, definir a lógica de jogo, produzir o código fonte e compilá-lo. Este tipo de software é normalmente utilizado por quem se dedica à criação de jogos como hobby, ou por profissionais que procuram otimizar o processo de desenvolvimento de jogos. Existem várias componentes ao nível do desenvolvimento de videojogos que se tornam pouco produtivas, se não forem automatizados e/ou otimizadas. Por exemplo, a programação de eventos ou de diálogos pode ser uma tarefa que consome demasiado tempo no ciclo de desenvolvimento, para além de ser uma tarefa entediante e repetitiva no ponto de vista do programador. Por este motivo, a utilização de ferramentas pode ser muito importante no que diz respeito ao aumento da produtividade e manutenção dos vários processos que envolvem o desenvolvimento de videojogos. Nesta dissertação pretendemos demonstrar as vantagens da utilização dessas mesmas ferramentas durante o desenvolvimento de videojogos, através da apresentação de um caso de estudo que envolve o desenvolvimento de um Serious Game intitulado Clean World. Em Clean World, foram identificadas determinadas tarefas que se mostraram demasiado repetitivas e entediantes quando programadas por inteiro, como é o caso da adição, modificação ou remoção de componentes como diálogos, quest ou items. Tendo em conta este problema concreto, foram criadas algumas ferramentas de forma a aumentar a produtividade no desenvolvimento do jogo, tornando tarefas repetitivas e entediantes em processos simples e intuitivos. O conjunto de ferramentas é constituído por: Item Manager, Quest Manager, Dialog Manager e Terrain Creator

Contemporary Robotics

This book book is a collection of 18 chapters written by internationally recognized experts and well-known professionals of the field. Chapters contribute to diverse facets of contemporary robotics and autonomous systems. The volume is organized in four thematic parts according to the main subjects, regarding the recent advances in the contemporary robotics. The first thematic topics of the book are devoted to the theoretical issues. This includes development of algorithms for automatic trajectory generation using redudancy resolution scheme, intelligent algorithms for robotic grasping, modelling approach for reactive mode handling of flexible manufacturing and design of an advanced controller for robot manipulators. The second part of the book deals with different aspects of robot calibration and sensing. This includes a geometric and treshold calibration of a multiple robotic line-vision system, robot-based inline 2D/3D quality monitoring using picture-giving and laser triangulation, and a study on prospective polymer composite materials for flexible tactile sensors. The third part addresses issues of mobile robots and multi-agent systems, including SLAM of mobile robots based on fusion of odometry and visual data, configuration of a localization system by a team of mobile robots, development of generic real-time motion controller for differential mobile robots, control of fuel cells of mobile robots, modelling of omni-directional wheeled-based robots, building of hunter- hybrid tracking environment, as well as design of a cooperative control in distributed population-based multi-agent approach. The fourth part presents recent approaches and results in humanoid and bioinspirative robotics. It deals with design of adaptive control of anthropomorphic biped gait, building of dynamic-based simulation for humanoid robot walking, building controller for perceptual motor control dynamics of humans and biomimetic approach to control mechatronic structure using smart materials