Magnetic-Visual Sensor Fusion-based Dense 3D Reconstruction and Localization for Endoscopic Capsule Robots

Reliable and real-time 3D reconstruction and localization functionality is a crucial prerequisite for the navigation of actively controlled capsule endoscopic robots as an emerging, minimally invasive diagnostic and therapeutic technology for use in the gastrointestinal (GI) tract. In this study, we propose a fully dense, non-rigidly deformable, strictly real-time, intraoperative map fusion approach for actively controlled endoscopic capsule robot applications which combines magnetic and vision-based localization, with non-rigid deformations based frame-to-model map fusion. The performance of the proposed method is demonstrated using four different ex-vivo porcine stomach models. Across different trajectories of varying speed and complexity, and four different endoscopic cameras, the root mean square surface reconstruction errors 1.58 to 2.17 cm.Comment: submitted to IROS 201

Algoritmo de planificación de movimiento para un robot móvil con un sistema de visión artificial inteligente

This study is devoted to the challenges of motion planning for mobile robots with smart machine vision systems. Motion planning for mobile robots in the environment with obstacles is a problem to deal with when creating robots suitable for operation in real-world conditions. The solutions found today are predominantly private, and are highly specialized, which prevents judging of how successful they are in solving the problem of effective motion planning. Solutions with a narrow application field already exist and are being already developed for a long time, however, no major breakthrough has been observed yet. Only a systematic improvement in the characteristics of such systems can be noted. The purpose of this study: develop and investigate a motion planning algorithm for a mobile robot with a smart machine vision system. The research subject for this article is a motion planning algorithm for a mobile robot with a smart machine vision system. This study provides a review of domestic and foreign mobile robots that solve the motion planning problem in a known environment with unknown obstacles. The following navigation methods are considered for mobile robots: local, global, individual. In the course of work and research, a mobile robot prototype has been built, capable of recognizing obstacles of regular geometric shapes, as well as plan and correct the movement path. Environment objects are identified and classified as obstacles by means of digital image processing methods and algorithms. Distance to the obstacle and relative angle are calculated by photogrammetry methods, image quality is improved by linear contrast enhancement and optimal linear filtering using the Wiener-Hopf equation. Virtual tools, related to mobile robot motion algorithm testing, have been reviewed, which led us to selecting Webots software package for prototype testing. Testing results allowed us to make the following conclusions. The mobile robot has successfully identified the obstacle, planned a path in accordance with the obstacle avoidance algorithm, and continued moving to the destination. Conclusions have been drawn regarding the concluded research.Este estudio está dedicado a los desafíos de la planificación del movimiento para robots móviles con sistemas inteligentes de visión artificial. La planificación del movimiento para robots móviles en un entorno con obstáculos es un problema con el que lidiar al crear robots adecuados para operar en condiciones del mundo real. Las soluciones que se encuentran en la actualidad son predominantemente privadas y altamente especializadas, lo que impide juzgar qué tan exitosas son para resolver el problema de la planificación eficaz del movimiento. Ya existen soluciones con un campo de aplicación estrecho y ya se están desarrollando durante mucho tiempo, sin embargo, aún no se han observado avances importantes. Solo se puede observar una mejora sistemática en las características de tales sistemas. El propósito de este estudio: desarrollar e investigar un algoritmo de planificación de movimiento para un robot móvil con un sistema de visión artificial inteligente. El tema de investigación de este artículo es un algoritmo de planificación de movimiento para un robot móvil con un sistema de visión artificial inteligente. Este estudio proporciona una revisión de robots móviles nacionales y extranjeros que resuelven el problema de planificación de movimiento en un entorno conocido con obstáculos desconocidos. Se consideran los siguientes métodos de navegación para robots móviles: local, global, individual. En el transcurso del trabajo e investigación se ha construido un prototipo de robot móvil, capaz de reconocer obstáculos de formas geométricas regulares, así como planificar y corregir la trayectoria del movimiento. Los objetos del entorno se identifican y clasifican como obstáculos mediante métodos y algoritmos de procesamiento de imágenes digitales. La distancia al obstáculo y el ángulo relativo se calculan mediante métodos de fotogrametría, la calidad de la imagen se mejora mediante la mejora del contraste lineal y el filtrado lineal óptimo utilizando la ecuación de Wiener-Hopf. Se han revisado las herramientas virtuales, relacionadas con las pruebas de algoritmos de movimiento de robots móviles, lo que nos llevó a seleccionar el paquete de software Webots para las pruebas de prototipos. Los resultados de las pruebas nos permitieron sacar las siguientes conclusiones. El robot móvil identificó con éxito el obstáculo, planificó una ruta de acuerdo con el algoritmo de evitación de obstáculos y continuó avanzando hacia el destino. Se han extraído conclusiones con respecto a la investigación concluida

Algoritmo de planificación de movimiento para un robot móvil con un sistema de visión artificial inteligente

Este estudio está dedicado a los desafíos de la planificación del movimiento para robots móviles con sistemas inteligentes de visión artificial. La planificación del movimiento para robots móviles en un entorno con obstáculos es un problema con el que lidiar al crear robots adecuados para operar en condiciones del mundo real. Las soluciones que se encuentran en la actualidad son predominantemente privadas y altamente especializadas, lo que impide juzgar qué tan exitosas son para resolver el problema de la planificación eficaz del movimiento. Ya existen soluciones con un campo de aplicación estrecho y ya se están desarrollando durante mucho tiempo, sin embargo, aún no se han observado avances importantes. Solo se puede observar una mejora sistemática en las características de tales sistemas. El propósito de este estudio: desarrollar e investigar un algoritmo de planificación de movimiento para un robot móvil con un sistema de visión artificial inteligente. El tema de investigación de este artículo es un algoritmo de planificación de movimiento para un robot móvil con un sistema de visión artificial inteligente. Este estudio proporciona una revisión de robots móviles nacionales y extranjeros que resuelven el problema de planificación de movimiento en un entorno conocido con obstáculos desconocidos. Se consideran los siguientes métodos de navegación para robots móviles: local, global, individual. En el transcurso del trabajo e investigación se ha construido un prototipo de robot móvil, capaz de reconocer obstáculos de formas geométricas regulares, así como planificar y corregir la trayectoria del movimiento. Los objetos del entorno se identifican y clasifican como obstáculos mediante métodos y algoritmos de procesamiento de imágenes digitales. La distancia al obstáculo y el ángulo relativo se calculan mediante métodos de fotogrametría, la calidad de la imagen se mejora mediante la mejora del contraste lineal y el filtrado lineal óptimo utilizando la ecuación de Wiener-Hopf. Se han revisado las herramientas virtuales, relacionadas con las pruebas de algoritmos de movimiento de robots móviles, lo que nos llevó a seleccionar el paquete de software Webots para las pruebas de prototipos. Los resultados de las pruebas nos permitieron sacar las siguientes conclusiones. El robot móvil identificó con éxito el obstáculo, planificó una ruta de acuerdo con el algoritmo de evitación de obstáculos y continuó avanzando hacia el destino. Se han extraído conclusiones con respecto a la investigación concluida

Visual-based Guidance System for a 6-DOF Robot

The idea of this bachelor’s thesis is to develop a position-based visual servoing system with two webcams that allows an anthropomorphic robot ABB IRB 120 with six degrees of freedom (DOF) to be guided in real time by an operator, by triangulating a specific target moved within the field of view (FOV) of a machine vision system. For this kind of motion control based on visual data input, termed visual servoing control, we need a stereo vision system to acquire the images of the target since three-dimensional information is required to perform object tracking with six DOF. These images are processed by a MATLAB ap-plication running in a remote PC. The current coordinates of the target referred to the left camera reference frame are extracted from the images and sent through an Ethernet connec-tion to the robot controller, which is programmed to receive the vectors and move its tool centre point (TCP) to the demanded position within its workspace.Grado en Ingeniería en Electrónica Industrial y Automátic

Sensing and Automation Technologies for Ornamental Nursery Crop Production: Current Status and Future Prospects

The ornamental crop industry is an important contributor to the economy in the United States. The industry has been facing challenges due to continuously increasing labor and agricultural input costs. Sensing and automation technologies have been introduced to reduce labor requirements and to ensure efficient management operations. This article reviews current sensing and automation technologies used for ornamental nursery crop production and highlights prospective technologies that can be applied for future applications. Applications of sensors, computer vision, artificial intelligence (AI), machine learning (ML), Internet-of-Things (IoT), and robotic technologies are reviewed. Some advanced technologies, including 3D cameras, enhanced deep learning models, edge computing, radio-frequency identification (RFID), and integrated robotics used for other cropping systems, are also discussed as potential prospects. This review concludes that advanced sensing, AI and robotic technologies are critically needed for the nursery crop industry. Adapting these current and future innovative technologies will benefit growers working towards sustainable ornamental nursery crop production

Intelligent 3D seam tracking and adaptable weld process control for robotic TIG welding

Tungsten Inert Gas (TIG) welding is extensively used in aerospace applications, due to its unique ability to produce higher quality welds compared to other shielded arc welding types. However, most TIG welding is performed manually and has not achieved the levels of automation that other welding techniques have. This is mostly attributed to the lack of process knowledge and adaptability to complexities, such as mismatches due to part fit-up. Recent advances in automation have enabled the use of industrial robots for complex tasks that require intelligent decision making, predominantly through sensors. Applications such as TIG welding of aerospace components require tight tolerances and need intelligent decision making capability to accommodate any unexpected variation and to carry out welding of complex geometries. Such decision making procedures must be based on the feedback about the weld profile geometry. In this thesis, a real-time position based closed loop system was developed with a six axis industrial robot (KUKA KR 16) and a laser triangulation based sensor (Micro-Epsilon Scan control 2900-25). [Continues.

TWINBOT: Autonomous Underwater Cooperative Transportation

Underwater Inspection, Maintenance, and Repair operations are nowadays performed using Remotely Operated Vehicles (ROV) deployed from dynamic-positioning vessels, having high daily operational costs. During the last twenty years, the research community has been making an effort to design new Intervention Autonomous Underwater Vehicles (I-AUV), which could, in the near future, replace the ROVs, significantly decreasing these costs. Until now, the experimental work using I-AUVs has been limited to a few single-vehicle interventions, including object search and recovery, valve turning, and hot stab operations. More complex scenarios usually require the cooperation of multiple agents, i.e., the transportation of large and heavy objects. Moreover, using small, autonomous vehicles requires consideration of their limited load capacity and limited manipulation force/torque capabilities. Following the idea of multi-agent systems, in this paper we propose a possible solution: using a group of cooperating I-AUVs, thus sharing the load and optimizing the stress exerted on the manipulators. Specifically, we tackle the problem of transporting a long pipe. The presented ideas are based on a decentralized Task-Priority kinematic control algorithm adapted for the highly limited communication bandwidth available underwater. The aforementioned pipe is transported following a sequence of poses. A path-following algorithm computes the desired velocities for the robots’ end-effectors, and the on-board controllers ensure tracking of these setpoints, taking into account the geometry of the pipe and the vehicles’ limitations. The utilized algorithms and their practical implementation are discussed in detail and validated through extensive simulations and experimental trials performed in a test tank using two 8 DOF I-AUV

Simulation and Planning of a 3D Spray Painting Robotic System

Nesta dissertação é proposto um sistema robótico 3D de pintura com spray. Este sistema inclui uma simulação realista do spray com precisão suficiente para imitar pintura com spray real. Também inclui um algoritmo otimizado para geração de caminhos que é capaz de pintar projetos 3D não triviais. A simulação parte de CAD 3D ou peças digitalizadas em 3D e produz um efeito visual realista que permite analisar qualitativamente o produto pintado. Também é apresentada uma métrica de avaliação que pontua trajetória de pintura baseada na espessura, uniformidade, tempo e desperdício de tinta.In this dissertation a 3D spray painting robotic system is proposed. This system has realistic spray simulation with sufficient accuracy to mimic real spray painting. It also includes an optimized algorithm for path generation that is capable of painting non trivial 3D designs. The simulation has 3D CAD or 3D scanned input pieces and produces a realistic visual effect that allows qualitative analyses of the painted product. It is also presented an evaluation metric that scores the painting trajectory based on thickness, uniformity, time and waste of paint