Artificial Vision in the Nao Humanoid Robot

Projecte Final de Màster UPC realitzat en col.laboració amb l'Universitat Rovira i Virgili. Departament d'Enginyeria Informàtica i MatemàtiquesRobocup is an international robotic soccer competition held yearly to promote innovative research and application in robotic intelligence. Nao humanoid robot is the new RoboCup Standard Platform robot. This platform is the new Nao robot designed and manufactured by the french company Aldebaran Robotics. The new robot is an advanced platform for developing new computer vision and robotics methods. This Master Thesis is oriented to the study of some fundamental issues for the artificial vision in the Nao humanoid robots. In particular, color representation models, real-time segmentation techniques, object detection and visual sonar approaches are the computer vision techniques applied to Nao robot in this Master Thesis. Also, Nao’s camera model, mathematical robot kinematic and stereo-vision techniques are studied and developed. This thesis also studies the integration between kinematic model and robot perception model to perform RoboCup soccer games and RoboCup technical challenges. This work is focused in the RoboCup environment but all computer vision and robotics algorithms can be easily extended to another robotics fields

Perceção e arquitectura de software para robótica móvel

Doutoramento em Ciências da ComputaçãoWhen developing software for autonomous mobile robots, one has to inevitably tackle some kind of perception. Moreover, when dealing with agents that possess some level of reasoning for executing their actions, there is the need to model the environment and the robot internal state in a way that it represents the scenario in which the robot operates. Inserted in the ATRI group, part of the IEETA research unit at Aveiro University, this work uses two of the projects of the group as test bed, particularly in the scenario of robotic soccer with real robots. With the main objective of developing algorithms for sensor and information fusion that could be used e ectively on these teams, several state of the art approaches were studied, implemented and adapted to each of the robot types. Within the MSL RoboCup team CAMBADA, the main focus was the perception of ball and obstacles, with the creation of models capable of providing extended information so that the reasoning of the robot can be ever more e ective. To achieve it, several methodologies were analyzed, implemented, compared and improved. Concerning the ball, an analysis of ltering methodologies for stabilization of its position and estimation of its velocity was performed. Also, with the goal keeper in mind, work has been done to provide it with information of aerial balls. As for obstacles, a new de nition of the way they are perceived by the vision and the type of information provided was created, as well as a methodology for identifying which of the obstacles are team mates. Also, a tracking algorithm was developed, which ultimately assigned each of the obstacles a unique identi er. Associated with the improvement of the obstacles perception, a new algorithm of estimating reactive obstacle avoidance was created. In the context of the SPL RoboCup team Portuguese Team, besides the inevitable adaptation of many of the algorithms already developed for sensor and information fusion and considering that it was recently created, the objective was to create a sustainable software architecture that could be the base for future modular development. The software architecture created is based on a series of di erent processes and the means of communication among them. All processes were created or adapted for the new architecture and a base set of roles and behaviors was de ned during this work to achieve a base functional framework. In terms of perception, the main focus was to de ne a projection model and camera pose extraction that could provide information in metric coordinates. The second main objective was to adapt the CAMBADA localization algorithm to work on the NAO robots, considering all the limitations it presents when comparing to the MSL team, especially in terms of computational resources. A set of support tools were developed or improved in order to support the test and development in both teams. In general, the work developed during this thesis improved the performance of the teams during play and also the e ectiveness of the developers team when in development and test phases.Durante o desenvolvimento de software para robôs autónomos móveis, e inevitavelmente necessário lidar com algum tipo de perceção. Al em disso, ao lidar com agentes que possuem algum tipo de raciocínio para executar as suas ações, há a necessidade de modelar o ambiente e o estado interno do robô de forma a representar o cenário onde o robô opera. Inserido no grupo ATRI, integrado na unidade de investigação IEETA da Universidade de Aveiro, este trabalho usa dois dos projetos do grupo como plataformas de teste, particularmente no cenário de futebol robótico com robôs reais. Com o principal objetivo de desenvolver algoritmos para fusão sensorial e de informação que possam ser usados eficazmente nestas equipas, v arias abordagens de estado da arte foram estudadas, implementadas e adaptadas para cada tipo de robôs. No âmbito da equipa de RoboCup MSL, CAMBADA, o principal foco foi a perceção da bola e obstáculos, com a criação de modelos capazes de providenciar informação estendida para que o raciocino do robô possa ser cada vez mais eficaz. Para o alcançar, v arias metodologias foram analisadas, implementadas, comparadas e melhoradas. Em relação a bola, foi efetuada uma análise de metodologias de filtragem para estabilização da sua posição e estimação da sua velocidade. Tendo o guarda-redes em mente, foi também realizado trabalho para providenciar informação de bolas no ar. Quanto aos obstáculos, foi criada uma nova definição para a forma como são detetados pela visão e para o tipo de informação fornecida, bem como uma metodologia para identificar quais dos obstáculos são colegas de equipa. Além disso foi desenvolvido um algoritmo de rastreamento que, no final, atribui um identicador único a cada obstáculo. Associado a melhoria na perceção dos obstáculos foi criado um novo algoritmo para realizar desvio reativo de obstáculos. No contexto da equipa de RoboCup SPL, Portuguese Team, al em da inevitável adaptação de vários dos algoritmos j a desenvolvidos para fusão sensorial e de informação, tendo em conta que foi recentemente criada, o objetivo foi criar uma arquitetura sustentável de software que possa ser a base para futuro desenvolvimento modular. A arquitetura de software criada e baseada numa série de processos diferentes e métodos de comunicação entre eles. Todos os processos foram criados ou adaptados para a nova arquitetura e um conjunto base de papeis e comportamentos foi definido para obter uma framework funcional base. Em termos de perceção, o principal foco foi a definição de um modelo de projeção e extração de pose da câmara que consiga providenciar informação em coordenadas métricas. O segundo objetivo principal era adaptar o algoritmo de localização da CAMBADA para funcionar nos robôs NAO, considerando todas as limitações apresentadas quando comparando com a equipa MSL, principalmente em termos de recursos computacionais. Um conjunto de ferramentas de suporte foram desenvolvidas ou melhoradas para auxiliar o teste e desenvolvimento em ambas as equipas. Em geral, o trabalho desenvolvido durante esta tese melhorou o desempenho da equipas durante os jogos e também a eficácia da equipa de programação durante as fases de desenvolvimento e teste

Study of Control Strategies for Robot Ball Catching

La tesi riguarda lo studio di un possibile scenario per la cattura di una palla con un braccio robotico usando tecnologie disponibili e considerando due problemi principali: studiare differenti strategie di controllo per il braccio robotico al fine di catturare la palla (controllo predittivo e prospettivo); implementare un simulatore in ROS che simula il robot reale, includendo un sistema di visione per riconoscere e tracciare la palla usando il sensore Microsoft Kinect, con diverse simulazion

Spatial representation for planning and executing robot behaviors in complex environments

Robots are already improving our well-being and productivity in different applications such as industry, health-care and indoor service applications. However, we are still far from developing (and releasing) a fully functional robotic agent that can autonomously survive in tasks that require human-level cognitive capabilities. Robotic systems on the market, in fact, are designed to address specific applications, and can only run pre-defined behaviors to robustly repeat few tasks (e.g., assembling objects parts, vacuum cleaning). They internal representation of the world is usually constrained to the task they are performing, and does not allows for generalization to other scenarios. Unfortunately, such a paradigm only apply to a very limited set of domains, where the environment can be assumed to be static, and its dynamics can be handled before deployment. Additionally, robots configured in this way will eventually fail if their "handcrafted'' representation of the environment does not match the external world. Hence, to enable more sophisticated cognitive skills, we investigate how to design robots to properly represent the environment and behave accordingly. To this end, we formalize a representation of the environment that enhances the robot spatial knowledge to explicitly include a representation of its own actions. Spatial knowledge constitutes the core of the robot understanding of the environment, however it is not sufficient to represent what the robot is capable to do in it. To overcome such a limitation, we formalize SK4R, a spatial knowledge representation for robots which enhances spatial knowledge with a novel and "functional" point of view that explicitly models robot actions. To this end, we exploit the concept of affordances, introduced to express opportunities (actions) that objects offer to an agent. To encode affordances within SK4R, we define the "affordance semantics" of actions that is used to annotate an environment, and to represent to which extent robot actions support goal-oriented behaviors. We demonstrate the benefits of a functional representation of the environment in multiple robotic scenarios that traverse and contribute different research topics relating to: robot knowledge representations, social robotics, multi-robot systems and robot learning and planning. We show how a domain-specific representation, that explicitly encodes affordance semantics, provides the robot with a more concrete understanding of the environment and of the effects that its actions have on it. The goal of our work is to design an agent that will no longer execute an action, because of mere pre-defined routine, rather, it will execute an actions because it "knows'' that the resulting state leads one step closer to success in its task

A Novel Multi-View Table Tennis Umpiring Framework

This research investigates the development of a low-cost multi-view umpiring framework, as an alternative to the current expensive systems that are almost exclusively restricted to elite professional sports. Table tennis has been selected as the testbed because, while automating the process is challenging, it has many different complex match elements including the service, return and rallies, which are governed by a strict set of regulations. The focus is mainly on the rally element rather than the whole match. Ball detection and tracking in video frames are undertaken to determine reliably the ball position relative to key reference objects like the table surface and net, and the ball’s flight path is used to determine the rally’s status. While a low-cost option has benefits, it is technically challenging due to the limited number of cameras and generally low video resolution used. This thesis presents a portable multi-view umpiring framework that identifies each state change in a rally. It makes three significant contributions to knowledge: i) a reliable ball detection strategy that accurately detects the location of the ball in low-resolution sequences; ii) a novel framework for ball tracking using a multi-view system, and iii) a new state-machine based evaluation system for analysing table tennis rallies. In a series of ten different test scenarios, the system achieved an average of 94% system detection rate and 100% accurate decisions. A test sequence of duration 1 s can be processed in 8 s, leading to a delay of only 7 s, which is considered acceptable for practical purposes. This solution has the potential to reform the way matches are umpired, providing objectivity in resolving disputed decisions. It affords an economic technology for amateur players, while the multi-view facility is extendible to other relevant ball-based sports. Finally, the ball flight path analysis mechanism can be a valuable training tool for skills development

Visual Servoing

The goal of this book is to introduce the visional application by excellent researchers in the world currently and offer the knowledge that can also be applied to another field widely. This book collects the main studies about machine vision currently in the world, and has a powerful persuasion in the applications employed in the machine vision. The contents, which demonstrate that the machine vision theory, are realized in different field. For the beginner, it is easy to understand the development in the vision servoing. For engineer, professor and researcher, they can study and learn the chapters, and then employ another application method

Multi-Robot Systems: Challenges, Trends and Applications

This book is a printed edition of the Special Issue entitled “Multi-Robot Systems: Challenges, Trends, and Applications” that was published in Applied Sciences. This Special Issue collected seventeen high-quality papers that discuss the main challenges of multi-robot systems, present the trends to address these issues, and report various relevant applications. Some of the topics addressed by these papers are robot swarms, mission planning, robot teaming, machine learning, immersive technologies, search and rescue, and social robotics

Exploring Meteorites Mysteries

This teacher's guide with activities supports lessons on the earliest history of the Solar System and meteorite impacts on Earth. Educational levels: Middle school, Intermediate elementary, High school