Humanoid Robot Kick in Motion Ability for Playing Robotic Soccer

Robotics and Artificial Intelligence are two deeply intertwined fields of study, currently experiencing formidable growth. To foster these developments, the RoboCup initiative is a fantastic test bed to experiment new approaches. This dissertation seeks to gather these possibilities to design and implement a humanoid robotic kick system employing deep neural networks, capable of fluidly kicking a ball while walking. This dissertation's work is rooted in the groundwork laid by previous FCPortugal3D teams so as to take the existing algorithms and skills into its consideration. In this way, a transition between a dynamic movement situation and one where the agent is kicking is achieved. Furthermore, it uses the new agent framework developed by the FCPortugal3D team so as to allow these tests to be built upon for future situations with ease

Comportamentos para robôs humanóides simulados

Mestrado em Engenharia de Computadores e TelemáticaThis thesis in inserted in the FC Portugal 3D team, which competes in the humanoid simulation league 3D from RoboCup. The objectives of this thesis are to improve the behaviours already created and to develop tools to support the development and debugging of the robotic agent. With this in mind, the process of optimization was improved to make it more efficient and adapted to include the new heterogeneous models. Executing the optimization process, using the state of the art algorithm CMA-ES, the time of the getup was reduced by half. Afterwards, the agent was put running in sync mode, which allows the simulations to run as fast as the computer in use can process, and not the simulation speed of the competion with cycles of 20ms. In the agent posture, it is now used the information from the gyroscope and the euler angles are calculated to get a better estimative of the robot orientation. On the other hand, the agent architecture was updated and new behaviours were created and optimized to support the new heterogeneous models. In relation to the standard model, some behaviours execute faster because of their physical difference. In the slot behaviours, it is now possible to defined preconditions in each step, so the agent can abort the behaviour when any condition does not comply. This change reduces the time wasted executing all the behaviour in situations in which the success is improbable. In terms of tools, a Agent Monitor Window was created for each agent which can: present in runtime variables from the agent code; interact with the code trough widgets; and if the simulation is in sync mode, defined the simulation cycle time, with the possibility to pause it and execute step by step, which gives a great advantage in terms of analysing the agent execution. The second tool was a behaviour testes for behaviours defined in XML, which allows, in runtime, to change the behaviour to test, edit its content, aggregate different files in sequence and finally the tolls can execute various agents in parallel. The last tools is Log Analyser of the logs generated by the agents and the server, which allows: exporting in different formats, see in form of plots the variables parsed, filtrate the simulation information; and create a server simulation which can be used to analyse, in parallel, the plots of chosen variables and the simulation in a monitor.Esta tese está inserida na equipa FC Portugal 3D, que compete na liga de futebol robótico simulado 3D. Os objetivos da tese são melhorar os comportamentos já existentes e desenvolver ferramentas de suporte ao desenvolvimento e depuração para o agente robótico. Nesse sentido, foi melhorado o processo de optimização de comportamentos de forma a torná-lo mais eficiente e adaptado para incluir os novos modelos heterogéneos disponibilizados. Ao executar o processo de optimização, usando o algoritmo de estado de arte CMA-ES, foi obtido reduções para metade do tempo nos comportamentos de levantar-se. Seguidamente o agente foi colocado a correr em modo síncrono, o que permite que as simulações corram à velocidade de processamento do computador em uso, e não à velocidade da simulação da competição em que cada ciclo demora 20ms. Assim é possível executar simulações e consequentemente inferir conclusões muito mais rapidamente. Passou-se a usar a informação de giroscópio e o cálculo dos ângulos de euler para obter uma melhor estimativa da rotação do robô. Por outro lado, devido ao lançamento de novos tipos de robôs, a arquitectura do agente teve de ser atualizada e novos comportamentos foram criados e optimizados para estes novos modelos. Em relação ao modelo original, alguns comportamentos são executados mais rapidamente e melhor pelos modelos novos, devido às suas alterações físicas. Por fim, nos comportamentos foi dada a possibilidade de definir pré condições em etapa do mesmo, para que possa ser abortado caso as condições não se verifiquem. Esta alteração veio reduzir o tempo desperdiçado a executar a totalidade do comportamento em situações em que não é provável o seu sucesso . Em termos de ferramentas, foi colocada uma Janela de Monitor de Agente para cada agente que, apresenta em tempo de simulação variáveis que o código do agente disponibiliza, interage com código através de widgets de seleção ou preenchimento, e se a simulação estiver a correr em modo síncrono, permite definir o tempo de ciclo da simulação, pausá-la e executar ciclo a ciclo, o que permite vantagens óbvias em termos de análise de execução dos agentes. Seguidamente, foi criada uma ferramenta de teste para comportamentos definidos em XML, que permite, em tempo de execução, alterar o ficheiro a testar, alterar o seu conteúdo, agrupar vários ficheiros em sequências e executar vários agentes em paralelo. Por fim, a última ferramenta é um Analizador de Logs gerados pelos agentes e pelo simulador que permite, entre outras funcionalidades, ver em forma de gráficos variáveis da simulação, exportar para diferentes formatos, filtrar a simulação usando informação da mesma e correr um servidor de forma a ser possível analizar em paralelo, gráficos de variáveis escolhidas e a simulação num visualizador

A robot localization proposal for the RobotAtFactory 4.0: A novel robotics competition within the Industry 4.0 concept

Robotic competitions are an excellent way to promote innovative solutions for the current industries’ challenges and entrepreneurial spirit, acquire technical and transversal skills through active teaching, and promote this area to the public. In other words, since robotics is a multidisciplinary field, its competitions address several knowledge topics, especially in the STEM (Science, Technology, Engineering, and Mathematics) category, that are shared among the students and researchers, driving further technology and science. A new competition encompassed in the Portuguese Robotics Open was created according to the Industry 4.0 concept in the production chain. In this competition, RobotAtFactory 4.0, a shop floor, is used to mimic a fully automated industrial logistics warehouse and the challenges it brings. Autonomous Mobile Robots (AMRs) must be used to operate without supervision and perform the tasks that the warehouse requests. There are different types of boxes which dictate their partial and definitive destinations. In this reasoning, AMRs should identify each and transport them to their destinations. This paper describes an approach to the indoor localization system for the competition based on the Extended Kalman Filter (EKF) and ArUco markers. Different innovation methods for the obtained observations were tested and compared in the EKF. A real robot was designed and assembled to act as a test bed for the localization system’s validation. Thus, the approach was validated in the real scenario using a factory floor with the official specifications provided by the competition organization.The authors are grateful to the Foundation for Science and Technology (FCT, Portugal) for financial support through national funds FCT/MCTES (PIDDAC) to CeDRI (UIDB/ 05757/2020 and UIDP/05757/2020) and SusTEC (LA/P/0007/ 2021). The project that gave rise to these results received the support of a fellowship from “la Caixa” Foundation (ID 100010434). The fellowship code is LCF/BQ/DI20/11780028. The authors also acknowledge the R&D Unit SYSTEC-Base (UIDB/00147/2020), Programmatic (UIDP/00147/2020) and Project Warehouse of the Future (WoF), with reference POCI-01-0247-FEDER-072638, co-funded by FEDER, through COMPETE 2020info:eu-repo/semantics/publishedVersio

RoboCup@Home: Analysis and results of evolving competitions for domestic and service robots

Scientific competitions are becoming more common in many research areas of artificial intelligence and robotics, since they provide a shared testbed for comparing different solutions and enable the exchange of research results. Moreover, they are interesting for general audiences and industries. Currently, many major research areas in artificial intelligence and robotics are organizing multiple-year competitions that are typically associated with scientific conferences. One important aspect of such competitions is that they are organized for many years. This introduces a temporal evolution that is interesting to analyze. However, the problem of evaluating a competition over many years remains unaddressed. We believe that this issue is critical to properly fuel changes over the years and measure the results of these decisions. Therefore, this article focuses on the analysis and the results of evolving competitions. In this article, we present the RoboCup@Home competition, which is the largest worldwide competition for domestic service robots, and evaluate its progress over the past seven years. We show how the definition of a proper scoring system allows for desired functionalities to be related to tasks and how the resulting analysis fuels subsequent changes to achieve general and robust solutions implemented by the teams. Our results show not only the steadily increasing complexity of the tasks that RoboCup@Home robots can solve but also the increased performance for all of the functionalities addressed in the competition. We believe that the methodology used in RoboCup@Home for evaluating competition advances and for stimulating changes can be applied and extended to other robotic competitions as well as to multi-year research projects involving Artificial Intelligence and Robotics

Infer and Adapt: Bipedal Locomotion Reward Learning from Demonstrations via Inverse Reinforcement Learning

Enabling bipedal walking robots to learn how to maneuver over highly uneven, dynamically changing terrains is challenging due to the complexity of robot dynamics and interacted environments. Recent advancements in learning from demonstrations have shown promising results for robot learning in complex environments. While imitation learning of expert policies has been well-explored, the study of learning expert reward functions is largely under-explored in legged locomotion. This paper brings state-of-the-art Inverse Reinforcement Learning (IRL) techniques to solving bipedal locomotion problems over complex terrains. We propose algorithms for learning expert reward functions, and we subsequently analyze the learned functions. Through nonlinear function approximation, we uncover meaningful insights into the expert's locomotion strategies. Furthermore, we empirically demonstrate that training a bipedal locomotion policy with the inferred reward functions enhances its walking performance on unseen terrains, highlighting the adaptability offered by reward learning

Towards adaptive and autonomous humanoid robots: from vision to actions

Although robotics research has seen advances over the last decades robots are still not in widespread use outside industrial applications. Yet a range of proposed scenarios have robots working together, helping and coexisting with humans in daily life. In all these a clear need to deal with a more unstructured, changing environment arises. I herein present a system that aims to overcome the limitations of highly complex robotic systems, in terms of autonomy and adaptation. The main focus of research is to investigate the use of visual feedback for improving reaching and grasping capabilities of complex robots. To facilitate this a combined integration of computer vision and machine learning techniques is employed. From a robot vision point of view the combination of domain knowledge from both imaging processing and machine learning techniques, can expand the capabilities of robots. I present a novel framework called Cartesian Genetic Programming for Image Processing (CGP-IP). CGP-IP can be trained to detect objects in the incoming camera streams and successfully demonstrated on many different problem domains. The approach requires only a few training images (it was tested with 5 to 10 images per experiment) is fast, scalable and robust yet requires very small training sets. Additionally, it can generate human readable programs that can be further customized and tuned. While CGP-IP is a supervised-learning technique, I show an integration on the iCub, that allows for the autonomous learning of object detection and identification. Finally this dissertation includes two proof-of-concepts that integrate the motion and action sides. First, reactive reaching and grasping is shown. It allows the robot to avoid obstacles detected in the visual stream, while reaching for the intended target object. Furthermore the integration enables us to use the robot in non-static environments, i.e. the reaching is adapted on-the- fly from the visual feedback received, e.g. when an obstacle is moved into the trajectory. The second integration highlights the capabilities of these frameworks, by improving the visual detection by performing object manipulation actions

Decision shaping and strategy learning in multi-robot interactions

Recent developments in robot technology have contributed to the advancement of autonomous behaviours in human-robot systems; for example, in following instructions received from an interacting human partner. Nevertheless, increasingly many systems are moving towards more seamless forms of interaction, where factors such as implicit trust and persuasion between humans and robots are brought to the fore. In this context, the problem of attaining, through suitable computational models and algorithms, more complex strategic behaviours that can influence human decisions and actions during an interaction, remains largely open. To address this issue, this thesis introduces the problem of decision shaping in strategic interactions between humans and robots, where a robot seeks to lead, without however forcing, an interacting human partner to a particular state. Our approach to this problem is based on a combination of statistical modeling and synthesis of demonstrated behaviours, which enables robots to efficiently adapt to novel interacting agents. We primarily focus on interactions between autonomous and teleoperated (i.e. human-controlled) NAO humanoid robots, using the adversarial soccer penalty shooting game as an illustrative example. We begin by describing the various challenges that a robot operating in such complex interactive environments is likely to face. Then, we introduce a procedure through which composable strategy templates can be learned from provided human demonstrations of interactive behaviours. We subsequently present our primary contribution to the shaping problem, a Bayesian learning framework that empirically models and predicts the responses of an interacting agent, and computes action strategies that are likely to influence that agent towards a desired goal. We then address the related issue of factors affecting human decisions in these interactive strategic environments, such as the availability of perceptual information for the human operator. Finally, we describe an information processing algorithm, based on the Orient motion capture platform, which serves to facilitate direct (as opposed to teleoperation-mediated) strategic interactions between humans and robots. Our experiments introduce and evaluate a wide range of novel autonomous behaviours, where robots are shown to (learn to) influence a variety of interacting agents, ranging from other simple autonomous agents, to robots controlled by experienced human subjects. These results demonstrate the benefits of strategic reasoning in human-robot interaction, and constitute an important step towards realistic, practical applications, where robots are expected to be not just passive agents, but active, influencing participants