Revisión de algoritmos de machine learning y deep learning apropiados para la implementación de visión artificial y movimientos autónomos en un brazo robótico.

Este informe presenta el trabajo de grado en la modalidad de proyecto de investigación, y como aporte al macro proyecto de investigación denominado “Diseño e implementación de herramienta robótica inteligente para robots de servicios e industria 4.0”. Las actividades fueron encaminadas a la búsqueda, prueba y análisis de algoritmos de Inteligencia artificial, que permitieran, por medio de visión artificial, la detección de objetos en un espacio de trabajo determinado. Se inicia con una vigilancia tecnológica de los diferentes desarrollos e implementaciones que ha tenido la inteligencia artificial en la visión artificial de robots manipuladores. Luego, se procede a realizar pruebas, en el software MATLAB, de los algoritmos de machine learning y deep learning más destacados en aplicaciones de visión artificial, específicamente, en la detección de objetos. Finalmente, se entrega un análisis de los datos obtenidos en cada uno de los algoritmos probados en la detección de imágenes y una propuesta para su implementación en el macro proyecto de investigación.This report presents the degree work in the form of a research project, and as a contribution to the research macro project called "Design and implementation of an intelligent robotic tool for service robots and industry 4.0". The activities were aimed at the search, test and analysis of artificial intelligence algorithms, which would allow, through artificial vision, the detection of objects in a given workspace. It begins with a technological surveillance of the different developments and implementations that artificial intelligence has had in the artificial vision of manipulative robots. Then, we proceed to carry out tests, in MATLAB software, of the most prominent machine learning and deep learning algorithms in artificial vision applications, specifically, in the detection of objects. Finally, an analysis of the data obtained in each of the algorithms tested in the detection of images and a proposal for its implementation in the main research project is delivered

Robotic Fabric Flattening with Wrinkle Direction Detection

Deformable Object Manipulation (DOM) is an important field of research as it contributes to practical tasks such as automatic cloth handling, cable routing, surgical operation, etc. Perception is considered one of the major challenges in DOM due to the complex dynamics and high degree of freedom of deformable objects. In this paper, we develop a novel image-processing algorithm based on Gabor filters to extract useful features from cloth, and based on this, devise a strategy for cloth flattening tasks. We evaluate the overall framework experimentally, and compare it with three human operators. The results show that our algorithm can determine the direction of wrinkles on the cloth accurately in the simulation as well as the real robot experiments. Besides, the robot executing the flattening tasks using the dewrinkling strategy given by our algorithm achieves satisfying performance compared to other baseline methods. The experiment video is available on https://sites.google.com/view/robotic-fabric-flattening/hom

Learning to Solve Tasks with Exploring Prior Behaviours

Demonstrations are widely used in Deep Reinforcement Learning (DRL) for facilitating solving tasks with sparse rewards. However, the tasks in real-world scenarios can often have varied initial conditions from the demonstration, which would require additional prior behaviours. For example, consider we are given the demonstration for the task of \emph{picking up an object from an open drawer}, but the drawer is closed in the training. Without acquiring the prior behaviours of opening the drawer, the robot is unlikely to solve the task. To address this, in this paper we propose an Intrinsic Rewards Driven Example-based Control \textbf{(IRDEC)}. Our method can endow agents with the ability to explore and acquire the required prior behaviours and then connect to the task-specific behaviours in the demonstration to solve sparse-reward tasks without requiring additional demonstration of the prior behaviours. The performance of our method outperforms other baselines on three navigation tasks and one robotic manipulation task with sparse rewards. Codes are available at https://github.com/Ricky-Zhu/IRDEC

Learning to Solve Tasks with Exploring Prior Behaviour

Postprin

Leveraging Optimal Transport for Enhanced Offline Reinforcement Learning in Surgical Robotic Environments

Most Reinforcement Learning (RL) methods are traditionally studied in an active learning setting, where agents directly interact with their environments, observe action outcomes, and learn through trial and error. However, allowing partially trained agents to interact with real physical systems poses significant challenges, including high costs, safety risks, and the need for constant supervision. Offline RL addresses these cost and safety concerns by leveraging existing datasets and reducing the need for resource-intensive real-time interactions. Nevertheless, a substantial challenge lies in the demand for these datasets to be meticulously annotated with rewards. In this paper, we introduce Optimal Transport Reward (OTR) labelling, an innovative algorithm designed to assign rewards to offline trajectories, using a small number of high-quality expert demonstrations. The core principle of OTR involves employing Optimal Transport (OT) to calculate an optimal alignment between an unlabeled trajectory from the dataset and an expert demonstration. This alignment yields a similarity measure that is effectively interpreted as a reward signal. An offline RL algorithm can then utilize these reward signals to learn a policy. This approach circumvents the need for handcrafted rewards, unlocking the potential to harness vast datasets for policy learning. Leveraging the SurRoL simulation platform tailored for surgical robot learning, we generate datasets and employ them to train policies using the OTR algorithm. By demonstrating the efficacy of OTR in a different domain, we emphasize its versatility and its potential to expedite RL deployment across a wide range of fields.Comment: Preprin