Local Motion Planner for Autonomous Navigation in Vineyards with a RGB-D Camera-Based Algorithm and Deep Learning Synergy

With the advent of agriculture 3.0 and 4.0, researchers are increasingly focusing on the development of innovative smart farming and precision agriculture technologies by introducing automation and robotics into the agricultural processes. Autonomous agricultural field machines have been gaining significant attention from farmers and industries to reduce costs, human workload, and required resources. Nevertheless, achieving sufficient autonomous navigation capabilities requires the simultaneous cooperation of different processes; localization, mapping, and path planning are just some of the steps that aim at providing to the machine the right set of skills to operate in semi-structured and unstructured environments. In this context, this study presents a low-cost local motion planner for autonomous navigation in vineyards based only on an RGB-D camera, low range hardware, and a dual layer control algorithm. The first algorithm exploits the disparity map and its depth representation to generate a proportional control for the robotic platform. Concurrently, a second back-up algorithm, based on representations learning and resilient to illumination variations, can take control of the machine in case of a momentaneous failure of the first block. Moreover, due to the double nature of the system, after initial training of the deep learning model with an initial dataset, the strict synergy between the two algorithms opens the possibility of exploiting new automatically labeled data, coming from the field, to extend the existing model knowledge. The machine learning algorithm has been trained and tested, using transfer learning, with acquired images during different field surveys in the North region of Italy and then optimized for on-device inference with model pruning and quantization. Finally, the overall system has been validated with a customized robot platform in the relevant environment

Optimal allocation of defibrillator drones in mountainous regions

Responding to emergencies in Alpine terrain is quite challenging as air ambulances and mountain rescue services are often confronted with logistics challenges and adverse weather conditions that extend the response times required to provide life-saving support. Among other medical emergencies, sudden cardiac arrest (SCA) is the most time-sensitive event that requires the quick provision of medical treatment including cardiopulmonary resuscitation and electric shocks by automated external defibrillators (AED). An emerging technology called unmanned aerial vehicles (or drones) is regarded to support mountain rescuers in overcoming the time criticality of these emergencies by reducing the time span between SCA and early defibrillation. A drone that is equipped with a portable AED can fly from a base station to the patient's site where a bystander receives it and starts treatment. This paper considers such a response system and proposes an integer linear program to determine the optimal allocation of drone base stations in a given geographical region. In detail, the developed model follows the objectives to minimize the number of used drones and to minimize the average travel times of defibrillator drones responding to SCA patients. In an example of application, under consideration of historical helicopter response times, the authors test the developed model and demonstrate the capability of drones to speed up the delivery of AEDs to SCA patients. Results indicate that time spans between SCA and early defibrillation can be reduced by the optimal allocation of drone base stations in a given geographical region, thus increasing the survival rate of SCA patients

A realistic simulation environment as a teaching aid in educational robotics

The experimental component is an essential method in Engineering education. Sometimes the availability of laboratories and components is compromised, and the COVID-19 pandemic worsened the situation. Resorting to an accurate simulation seems to help this process by allowing students to develop the work, program, test, and validate it. Moreover, it lowers the development time and cost of the prototyping stages of a robotics project. As a multidisciplinary area, robotics requires simulation environments with essential characteristics, such as dynamics, connection to hardware (embedded systems), and other applications. Thus, this paper presents the Simulation environment of SimTwo, emphasizing previous publications with models of sensors, actuators, and simulation scenes. The simulator can be used for free, and the source code is available to the community. Proposed scenes and examples can inspire the development of other simulation scenes to be used in electrical and mechanical Engineering projects. © 2022 IEEE.The authors are grateful to the Foundation for Science and Technology (FCT, Portugal) for financial support through national funds FCT/MCTES (PIDDAC) to CeDRI (UIDB/05757/2020 and UIDP/05757/2020) and SusTEC (LA/P/0007/2021). Thadeu Brito was supported by FCT PhD grant SFRH/BD/08598/2020 and Joao Braun received the ˜ support of a fellowship from ”la Caixa” Foundation (ID 100010434) with code LCF/BQ/DI20/11780028.info:eu-repo/semantics/publishedVersio

RobotAtFactory 4.0: a ROS framework for the SimTwo simulator

Robotics competitions encourage the development of solutions to new challenges that emerge in sync with the rise of Industry 4.0. In this context, robotic simulators are employed to facilitate the development of these solutions by disseminating knowledge in robotics, Education 4.0, and STEM. The RobotAtFactory 4.0 competition arises to promote improvements in industrial challenges related to autonomous robots. The official organization provides the simulation scene of the competition through the open-source SimTwo simulator. This paper aims to integrate the SiwTwo simulator with the Robot Operating System (ROS) middleware by developing a framework. This integration facilitates the design of robotic systems since ROS has a vast repository of packages that address common problems in robotics. Thus, competitors can use this framework to develop their solutions through ROS, allowing the simulated and real systems to be integrated.This work has been supported by FCT - Fundação para a Ciência e Tecnologia within the Project Scope: UIDB/05757/2020. The project that gave rise to these results received the support of a fellowship from ”la Caixa” Foundation (ID 100010434). The fellowship code is LCF/BQ/DI20/11780028.info:eu-repo/semantics/publishedVersio

AdaptPack Studio: an automated intelligent framework for offline factory programming

Purpose This paper aims to propose an automated framework for agile development and simulation of robotic palletizing cells. An automatic offline programming tool, for a variety of robot brands, is also introduced. Design/methodology/approach This framework, named AdaptPack Studio, offers a custom-built library to assemble virtual models of palletizing cells, quick connect these models by drag and drop, and perform offline programming of robots and factory equipment in short steps. Findings Simulation and real tests performed showed an improvement in the design, development and operation of robotic palletizing systems. The AdaptPack Studio software was tested and evaluated in a pure simulation case and in a real-world scenario. Results have shown to be concise and accurate, with minor model displacement inaccuracies because of differences between the virtual and real models. Research limitations/implications An intuitive drag and drop layout modeling accelerates the design and setup of robotic palletizing cells and automatic offline generation of robot programs. Furthermore, A* based algorithms generate collision-free trajectories, discretized both in the robot joints space and in the Cartesian space. As a consequence, industrial solutions are available for production in record time, increasing the competitiveness of companies using this tool. Originality/value The AdaptPack Studio framework includes, on a single package, the possibility to program, simulate and generate the robot code for four different brands of robots. Furthermore, the application is tailored for palletizing applications and specifically includes the components (Building Blocks) of a particular company, which allows a very fast development of new solutions. Furthermore, with the inclusion of the Trajectory Planner, it is possible to automatically develop robot trajectories without collisions. </jats:sec

Robot@factory lite: an educational approach for the competition with simulated and real environment

Teaching based on challenges and competitions is one of the most exciting and promising methods for students. In this paper, a competition of the Portuguese Robotics Open is addressed and a solution is proposed. The Robot@Factory Lite is a new challenge and accepts participants from secondary schools (Rookie) and universities. The concepts of simulation, hardware-in-the-loop and timed finite state machine are presented and validated in the real robot prototype. The aim of this paper is to disseminate the developed solution in order to attract more students to STEM educational program.This work is financed by the ERDF - European Regional Development Fund through the Operational Programme for Competitiveness and Internationalisation- COMPETE 2020 Programme within project (POCI-01-0145-FEDER- 006961), and by National Funds through the FCT - Fundação para a Ciência e a Tecnologia (Portuguese Foundation for Science and Technology) as part of project UID /EEA/50014/2013.info:eu-repo/semantics/publishedVersio

AdaptPack studio translator: translating offline programming to real palletizing robots

Purpose This paper aims to propose a translation library capable of generating robots proprietary code after their offline programming has been performed in a software application, named AdaptPack Studio, running over a robot simulation and offline programming software package. Design/methodology/approach The translation library, named AdaptPack Studio Translator, is capable to generate proprietary code for the Asea Brown Boveri, FANUC, Keller und Knappich Augsburg and Yaskawa Motoman robot brands, after their offline programming has been performed in the AdaptPack Studio application. Findings Simulation and real tests were performed showing an improvement in the creation, operation, modularity and flexibility of new robotic palletizing systems. In particular, it was verified that the time needed to perform these tasks significantly decreased. Practical implications The design and setup of robotics palletizing systems are facilitated by an intuitive offline programming system and by a simple export command to the real robot, independent of its brand. In this way, industrial solutions can be developed faster, in this way, making companies more competitive. Originality/value The effort to build a robotic palletizing system is reduced by an intuitive offline programming system (AdaptPack Studio) and the capability to export command to the real robot using the AdaptPack Studio Translator. As a result, companies have an increase in competitiveness with a fast design framework. Furthermore, and to the best of the author’s knowledge, there is also no scientific publication formalizing and describing how to build the translators for industrial robot simulation and offline programming software packages, being this a pioneer publication in this area. </jats:sec

Robotic Welding Optimization using A* Parallel Path Planning and Advanced Machine Learning

O mundo da robótica está em constante evolução, procurando novas soluções para melhorar as tecnologias atuais e superar os problemas industriais atuais. Um dos principais componentes na robótica, algoritmos de planeamento de trajetórias, não possui flexibilidade quando certas restrições dinâmicas são introduzidas às células robóticas. Este problema é maioritariamente relacionado com a grande quantidade de tempo necessária para gerar uma trajetória, sem colisões, para sistemas muito redundantes. Apesar de todos os benefícios, a utilização de soluções envolvendo paralelização com CPU e GPU ainda não foi devidamente desenvolvida. Além disto, devido à física envolvida na soldadura ser de grande complexidade, a parametrização de um robot consome muito tempo. Na soldadura manual, todos os atributos do operador (entre eles a experiência e a visão) podem compensar as dificuldades em encontrar os parâmetros ideais (para soldadura, posição do robot, velocidade, etc.) para uma determinada peça. Na soldadura com robots, o manipulador e os sensores têm alguns limites, o que faz com que o tempo necessário para a realização da parametrização aumenta significativamente. O objetivo principal deste projeto é otimizar soldadura com robots desenvolvendo um sistema flexível de soldadura, através da utilização de suporte à decisão (baseado em conhecimento) para parametrização de soldaduras numa célula robótica avançada, combinado com programação offline (sem colisões) e sensorização, e de um software capaz de interligar algoritmos de planeamento com ferramentas de computação paralela, para reduzir o tempo necessário para gerar um caminho seguro. Este projeto também fará uma investigação sobre o estado atual da robótica, as soluções existentes para problemas de planeamento de trajetórias, assim como técnicas de aprendizagem computacional e os principais parâmetros na soldadura.The world of robotics is in constant evolution, trying to find new solutions to improve on top of the current technology and to overcome the current industrial pitfalls. To date, one of the key intelligent robotics components, path planning algorithms, lack flexibility when considering dynamic constraints on the surrounding work cell. This is mainly related to a large amount of time required to generate safe collision-free paths for high redundancy systems. Furthermore, and despite the already known benefits, the adoption of CPU/GPU parallel solutions is still lacking in the robotic field. On top of this, welding physics is complex, and therefore the welding parametrization is time-consuming. In manual welding, the "hand", the experience, and the best sensor of all (the eyes) can compensate for the difficulties in finding the right settings (welding parameters, robot posture, speed, ...) for a specific weld seam. In robotic welding, the robotic arm and the sensors are limited, and the parametrization time escalates. The main goal of this project is to optimize robot welding, by developing a flexible welding robotized system, through the introduction of (knowledge-based) decision support for welding parametrization in an advanced robotic work cell, in combination with advanced (collision-free) offline programming and advanced sensing, and improve path planning by developing a software platform capable of interconnecting the path planning algorithms with parallel computing tools, reducing the time needed to generate a safe path. This project will also investigate the current state of robotics and existing solutions for path planning problems, as well as machine learning algorithms and the most important parameters for welding

Locomoção de humanoides robusta e versátil baseada em controlo analítico e física residual

Humanoid robots are made to resemble humans but their locomotion abilities are far from ours in terms of agility and versatility. When humans walk on complex terrains or face external disturbances, they combine a set of strategies, unconsciously and efficiently, to regain stability. This thesis tackles the problem of developing a robust omnidirectional walking framework, which is able to generate versatile and agile locomotion on complex terrains. We designed and developed model-based and model-free walk engines and formulated the controllers using different approaches including classical and optimal control schemes and validated their performance through simulations and experiments. These frameworks have hierarchical structures that are composed of several layers. These layers are composed of several modules that are connected together to fade the complexity and increase the flexibility of the proposed frameworks. Additionally, they can be easily and quickly deployed on different platforms. Besides, we believe that using machine learning on top of analytical approaches is a key to open doors for humanoid robots to step out of laboratories. We proposed a tight coupling between analytical control and deep reinforcement learning. We augmented our analytical controller with reinforcement learning modules to learn how to regulate the walk engine parameters (planners and controllers) adaptively and generate residuals to adjust the robot’s target joint positions (residual physics). The effectiveness of the proposed frameworks was demonstrated and evaluated across a set of challenging simulation scenarios. The robot was able to generalize what it learned in one scenario, by displaying human-like locomotion skills in unforeseen circumstances, even in the presence of noise and external pushes.Os robôs humanoides são feitos para se parecerem com humanos, mas suas habilidades de locomoção estão longe das nossas em termos de agilidade e versatilidade. Quando os humanos caminham em terrenos complexos ou enfrentam distúrbios externos combinam diferentes estratégias, de forma inconsciente e eficiente, para recuperar a estabilidade. Esta tese aborda o problema de desenvolver um sistema robusto para andar de forma omnidirecional, capaz de gerar uma locomoção para robôs humanoides versátil e ágil em terrenos complexos. Projetámos e desenvolvemos motores de locomoção sem modelos e baseados em modelos. Formulámos os controladores usando diferentes abordagens, incluindo esquemas de controlo clássicos e ideais, e validámos o seu desempenho por meio de simulações e experiências reais. Estes frameworks têm estruturas hierárquicas compostas por várias camadas. Essas camadas são compostas por vários módulos que são conectados entre si para diminuir a complexidade e aumentar a flexibilidade dos frameworks propostos. Adicionalmente, o sistema pode ser implementado em diferentes plataformas de forma fácil. Acreditamos que o uso de aprendizagem automática sobre abordagens analíticas é a chave para abrir as portas para robôs humanoides saírem dos laboratórios. Propusemos um forte acoplamento entre controlo analítico e aprendizagem profunda por reforço. Expandimos o nosso controlador analítico com módulos de aprendizagem por reforço para aprender como regular os parâmetros do motor de caminhada (planeadores e controladores) de forma adaptativa e gerar resíduos para ajustar as posições das juntas alvo do robô (física residual). A eficácia das estruturas propostas foi demonstrada e avaliada em um conjunto de cenários de simulação desafiadores. O robô foi capaz de generalizar o que aprendeu em um cenário, exibindo habilidades de locomoção humanas em circunstâncias imprevistas, mesmo na presença de ruído e impulsos externos.Programa Doutoral em Informátic

Robot@Factory Lite: an enhancement for the mobile robot using quadrature encoders

Mestrado de dupla diplomação com a UTFPR - Universidade Tecnológica Federal do ParanáAs competições robóticas estão crescendo a cada ano, trazendo benefícios como soluções para problemas do dia-a-dia, ajudando na educação e aumentando o interesse de crianças e adolescentes em áreas de viés tecnológico. Este trabalho foca-se na competição Robot@Factory Lite apresentando melhorias para o robô utilizado na competição. Após participar da primeira edição da competição, foram constatados alguns defeitos no funcionamento do robô que atrapalhavam o desempenho das equipes, visto que todas as equipes optaram por utilizar o mesmo modelo de robô. Foi desenvolvido um robô utilizando encoders para substituir a movimentação baseada em tempo por uma baseada em distância. O controle de velocidade é feito através de um controlador PID. Um sistema de odometria foi implementado para monitorar a posição do robô. Comunicação via Wi-Fi foi utilizada para substituir o leitor RFID para identificar as caixas além de ser utilizada para enviar informações do robô em tempo real para depuração de erros.Robotic competitions are growing every year, bringing benefits such as solutions to day-to-day problems, helping in education and increasing the interest of children and adolescents in technological areas. This work focuses on the Robot@Factory Lite competition, presenting improvements for the robot used in the competition. After participating in the first edition of the competition, some defects in the functioning of the robot were found that hindered the performance of the teams, since all teams chose to use the same robot model. A robot was developed using encoders to replace time-based motion with distance-based motion. The speed control is done through a PID controller. An odometry system was implemented to monitor the robot’s position. Communication via Wi-Fi was used to replace the RFID reader to identify the boxes in addition to being used to send information from the robot in real time for debugging errors