Design of a strategy to obtain safe paths from collaborative robot teamwork

Documento en PDF a color.figuras, tablasThis doctoral thesis was designed and implemented using a strategy of explorer agents and a management and monitoring system to obtain the shortest and safest paths. The strategy was simulated using Matlab R2016 in 10 test environments. The comparisons were made between the results obtained by considering each robot's work and contrasting it with the results obtained by implementing the cooperative-collaborative strategy. For this purpose, were used two path planning algorithms, they are the A* and the Greedy Best First Search (GBFS). Some changes were made to these classic algorithms to improve their performance to guarantee interactions and comparisons between them, transforming them into Incremental Heuristic (IH) algorithms, which gave rise to a couple of agents with new path planners called IH-A* and IH-GBFS. The cooperative strategy was implemented with IH-A* and IH-GBFS algorithms to obtain the shortest paths. The cooperative process was used 300 times in 100 complete tests (3 times in 10 tests in each of 10 environments), which allowed determining that the strategy decreased the original path (without cooperation) in 79% of the cases. In 20.50% of cases, the author identified that the cooperative process, reduced to less than half the original path. The collaborative strategy was implemented to obtain the safer path, using a communications system that allows the interaction among the explorer agents, the test environment, and the management and monitoring system to generate early warnings and compare the risk between paths. In this work, the risk is due to hidden marks found by the explorer agents; for this reason, it is implemented a potential risk function that allows obtaining the path risk estimated. The path risk estimated metric is the one that facilitates the evaluation and comparison of risk between paths to find safer paths. The AWMRs operates using a kinematic model, a controller, a path planner, and sensors that allow them to navigate through the environment gently and safely. Simultaneously with the explorer agents, the administration and monitoring system as a user interface that facilitates the presentation and consolidation of results were implemented. Subsequently, 16 tests were carried out, implementing the complete cooperative-collaborative strategy in four different environments, which had hidden marks. When analyzing the results, it was determined that the Shortest Safest Estimated Path was found in 62.5% of the tests. A WMR and a square test stage were built. In the test scenario, 240 path tracking tests were carried out (the WMR travelled 24 different paths; the WMR travelled each path ten times). The path data were obtained using odometry with encoders onboard the robot and image processing through an external camera. The author apply a tracking error analysis on the WMR path, travelling a circumference of 3.64 m in length. When comparing the path obtained with the WMR kinematic model with the data obtained using image processing, a Mean Absolute Percentage Error (MAPE) of 2,807% was obtained; and with the odometry data, the MAPE was 1,224%. As a general conclusion, this study has numerically identified the relevance of the implementation of the cooperative-collaborative strategy in robotic teamwork to find shortest and safest paths, a strategy applied in test environments that have obstacles and hidden marks. The cooperative-collaborative strategy can be used in different applications that involve displacement in a dangerous place or environment, such as a minefield or a region at risk of spreading COVID-19.Esta tesis doctoral fue diseñada e implementada utilizando una estrategia de agentes exploradores y un sistema de gestión y seguimiento para obtener caminos más cortos y seguros. La estrategia se simuló utilizando Matlab R2016 en 10 entornos de prueba. Las comparaciones se realizaron entre los resultados obtenidos al considerar el trabajo realizado por cada robot y contrastarlo con los resultados obtenidos al implementar la estrategia cooperativa-colaborativa. Para ello, se utilizaron dos algoritmos de planificación de rutas, que son el A* y el Greedy Best First Search (GBFS). Se realizaron algunos cambios a estos algoritmos clásicos para mejorar su rendimiento para garantizar interacciones y comparaciones entre ellos, transformándolos en algoritmos Heurísticos Incrementales (IH), lo que dio lugar a un par de agentes con nuevos planificadores de rutas denominados IH-A * e IH- GBFS. La estrategia cooperativa se implementó con algoritmos IH-A * e IH-GBFS para obtener los caminos más cortos. El proceso cooperativo se utilizó 300 veces en 100 pruebas completas (3 veces en 10 pruebas en cada uno de los 10 entornos), lo que permitió determinar que la estrategia disminuyó la trayectoria original (sin cooperación) en el 79% de los casos. En el 20,50% de los casos, el autor identificó que el proceso cooperativo, redujo la distancia entre inicio y meta a menos de la mitad del recorrido original. La estrategia colaborativa se implementó para obtener el camino más seguro, utilizando un sistema de comunicaciones que permite la interacción entre los agentes exploradores, el entorno de prueba y el sistema de gestión y monitoreo para generar alertas tempranas y comparar el riesgo entre caminos. En este trabajo, el riesgo se debe a las marcas ocultas encontradas por los agentes exploradores; por ello, se implementa una función de riesgo potencial que permite obtener el riesgo de ruta estimado. La métrica estimada de riesgo de ruta es la que facilita la evaluación y comparación de riesgo entre rutas para encontrar rutas más seguras. Los robots autónomos móviles con ruedas (en inglés AWMR) operan utilizando un modelo cinemático, un controlador, un planificador de rutas y sensores que les permiten navegar por el entorno de manera suave y segura. Simultáneamente con los agentes exploradores, el autor implementó un sistema de administración y monitoreo como interfaz de usuario que facilita la presentación y consolidación de resultados. Posteriormente, se realizaron 16 pruebas, implementando la estrategia cooperativa-colaborativa completa en cuatro entornos diferentes, que tenían marcas ocultas. Al analizar los resultados, se determinó que una ruta estimada más corta y más segura se obtenía en el 62.5% de las pruebas. Se construyeron un WMR y un escenario de prueba cuadrado. En el escenario de prueba, se llevaron a cabo 240 pruebas de seguimiento de ruta (el WMR recorrió 24 rutas diferentes; el WMR recorrió cada ruta diez veces). Los datos de la trayectoria se obtuvieron utilizando odometría con encoders a bordo del robot y procesamiento de imágenes a través de una cámara externa. El autor aplica un análisis de error de seguimiento en la ruta recorrida por el WMR, generando una circunferencia de 3,64 m de longitud. Al comparar la ruta obtenida con el modelo cinemático del WMR con los datos obtenidos usando el procesamiento de imágenesse obtuvo un error de porcentaje absoluto medio (MAPE) de 2.807%; y con los datos de odometría, el MAPE fue de 1,224%. Como conclusión general, este estudio ha identificado numéricamente la relevancia de la implementación de la estrategia cooperativa-colaborativa en el trabajo en equipo robótico para encontrar caminos más cortos y seguros, estrategia aplicada en entornos de prueba que poseen obstáculos y marcas ocultas. La estrategia cooperativa-colaborativa puede ser utilizada en diferentes aplicaciones que involucran el desplazamiento en un lugar o entorno peligroso, como pueden ser un campo minado o una región en riesgo de propagación de COVID-19.DoctoradoDoctor en Ingeniería - Ingeniería Automátic

Metric and appearance based visual SLAM for mobile robots

Simultaneous Localization and Mapping (SLAM) maintains autonomy for mobile robots and it has been studied extensively during the last two decades. It is the process of building the map of an unknown environment and determining the location of the robot using this map concurrently. Different kinds of sensors such as Global Positioning System (GPS), Inertial Measurement Unit (IMU), laser range finder and sonar are used for data acquisition in SLAM. In recent years, passive visual sensors are utilized in visual SLAM (vSLAM) problem because of their increasing ubiquity. This thesis is concerned with the metric and appearance-based vSLAM problems for mobile robots. From the point of view of metric-based vSLAM, a performance improvement technique is developed. Template matching based video stabilization and Harris corner detector are integrated. Extracting Harris corner features from stabilized video consistently increases the accuracy of the localization. Data coming from a video camera and odometry are fused in an Extended Kalman Filter (EKF) to determine the pose of the robot and build the map of the environment. Simulation results validate the performance improvement obtained by the proposed technique. Moreover, a visual perception system is proposed for appearance-based vSLAM and used for under vehicle classification. The proposed system consists of three main parts: monitoring, detection and classification. In the first part a new catadioptric camera system, where a perspective camera points downwards to a convex mirror mounted to the body of a mobile robot, is designed. Thanks to the catadioptric mirror the scenes against the camera optical axis direction can be viewed. In the second part speeded up robust features (SURF) are used to detect the hidden objects that are under vehicles. Fast appearance based mapping algorithm (FAB-MAP) is then exploited for the classification of the means of transportations in the third part. Experimental results show the feasibility of the proposed system. The proposed solution is implemented using a non-holonomic mobile robot. In the implementations the bottom of the tables in the laboratory are considered as the under vehicles. A database that includes di erent under vehicle images is used. All the algorithms are implemented in Microsoft Visual C++ and OpenCV 2.4.4

Visual Odometry and Traversability Analysis for Wheeled Robots in Complex Environments

Durch die technische Entwicklung im Bereich der radbasierten mobilen Roboter (WMRs) erweitern sich deren Anwendungsszenarien. Neben den eher strukturierten industriellen und häuslichen Umgebungen sind nun komplexere städtische Szenarien oder Außenbereiche mögliche Einsatzgebiete. Einer dieser neuen Anwendungsfälle wird in dieser Arbeit beschrieben: ein intelligenter persönlicher Mobilitätsassistent, basierend auf einem elektrischen Rollator. Ein solches System hat mehrere Anforderungen: Es muss sicher, robust, leicht und preiswert sein und sollte in der Lage sein, in Echtzeit zu navigieren, um eine direkte physische Interaktion mit dem Benutzer zu ermöglichen. Da diese Eigenschaften für fast alle Arten von WMRs wünschenswert sind, können alle in dieser Arbeit präsentierten Methoden auch mit anderen Typen von WMRs verwendet werden. Zuerst wird eine visuelle Odometriemethode vorgestellt, welche auf die Arbeit mit einer nach unten gerichteten RGB-D-Kamera ausgelegt ist. Hierzu wird die Umgebung auf die Bodenebene projiziert, um eine 2-dimensionale Repräsentation zu erhalten. Nun wird ein effizientes Bildausrichtungsverfahren verwendet, um die Fahrzeugbewegung aus aufeinander folgenden Bildern zu schätzen. Da das Verfahren für den Einsatz auf einem WMR ausgelegt ist, können weitere Annahmen verwendet werden, um die Genauigkeit der visuellen Odometrie zu verbessern. Für einen nicht-holonomischen WMR mit einem bekannten Fahrzeugmodell, entweder Differentialantrieb, Skid-Lenkung oder Ackermann-Lenkung, können die Bewegungsparameter direkt aus den Bilddaten geschätzt werden. Dies verbessert die Genauigkeit und Robustheit des Verfahrens erheblich. Zusätzlich wird eine Ausreißererkennung vorgestellt, die im Modellraum, d.h. den Bewegungsparametern des kinematischen Models, arbeitet. Üblicherweise wird die Ausreißererkennung im Datenraum, d.h. auf den Bildpunkten, durchgeführt. Mittels der Projektion der Umgebung auf die Bodenebene kann auch eine Höhenkarte der Umgebung erstellt werde. Es wird untersucht, ob diese Karte, in Verbindung mit einem detaillierten Fahrzeugmodell, zur Abschätzung zukünftiger Fahrzeugposen verwendet werden kann. Durch die Verwendung einer gemeinsamen bildbasierten Darstellung der Umgebung und des Fahrzeugs wird eine sehr effiziente und dennoch sehr genaue Posenschätzmethode vorgeschlagen. Da die Befahrbarkeit eines Bereichs durch die Fahrzeugposen und mögliche Kollisionen bestimmt werden kann, wird diese Methode für eine neue echtzeitfähige Pfadplanung verwendet. Aus der Fahrzeugpose werden verschiedene Sicherheitskriterien bestimmt, die als Heuristik für einen A*-ähnlichen Planer verwendet werden. Hierzu werden mithilfe des kinematischen Models mögliche zukünftige Fahrzeugposen ermittelt und für jede dieser Posen ein Befahrbarkeitswert berechnet.Das endgültige System ermöglicht eine sichere und robuste Echtzeit-Navigation auch in schwierigen Innen- und Außenumgebungen.The application of wheeled mobile robots (WMRs) is currently expanding from rather controlled industrial or domestic scenarios into more complex urban or outdoor environments, allowing a variety of new use cases. One of these new use cases is described in this thesis: An intelligent personal mobility assistant, based on an electrical rollator. Such a system comes with several requirements: It must be safe and robust, lightweight, inexpensive and should be able to navigate in real-time in order to allow direct physical interaction with the user. As these properties are desirable for most WMRs, all methods proposed in this thesis can also be used with other WMR platforms.First, a visual odometry method is presented, which is tailored to work with a downward facing RGB-D camera. It projects the environment onto a ground plane image and uses an efficient image alignment method to estimate the vehicle motion from consecutive images. As the method is designed for use on a WMR, further constraints can be employed to improve the accuracy of the visual odometry. For a non-holonomic WMR with a known vehicle model, either differential drive, skid steering or Ackermann, the motion parameters of the corresponding kinematic model, instead of the generic motion parameters, can be estimated directly from the image data. This significantly improves the accuracyand robustness of the method. Additionally, an outlier rejection scheme is presented that operates in model space, i.e. the motion parameters of the kinematic model, instead of data space, i.e. image pixels. Furthermore, the projection of the environment onto the ground plane can also be used to create an elevation map of the environment. It is investigated if this map, in conjunction with a detailed vehicle model, can be used to estimate future vehicle poses. By using a common image-based representation of the environment and the vehicle, a very efficient and still highly accurate pose estimation method is proposed. Since the traversability of an area can be determined by the vehicle poses and potential collisions, the pose estimation method is employed to create a novel real-time path planning method. The detailed vehicle model is extended to also represent the vehicle’s chassis for collision detection. Guided by an A*-like planner, a search graph is constructed by propagating the vehicle using its kinematic model to possible future poses and calculating a traversability score for each of these poses. The final system performs safe and robust real-time navigation even in challenging indoor and outdoor environments

Enhanced vision-based localization and control for navigation of non-holonomic omnidirectional mobile robots in GPS-denied environments

New Zealand’s economy relies on primary production to a great extent, where use of the technological advances can have a significant impact on the productivity. Robotics and automation can play a key role in increasing productivity in primary sector, leading to a boost in national economy. This thesis investigates novel methodologies for design, control, and navigation of a mobile robotic platform, aimed for field service applications, specifically in agricultural environments such as orchards to automate the agricultural tasks. The design process of this robotic platform as a non-holonomic omnidirectional mobile robot, includes an innovative integrated application of CAD, CAM, CAE, and RP for development and manufacturing of the platform. Robot Operating System (ROS) is employed for the optimum embedded software system design and development to enable control, sensing, and navigation of the platform. 3D modelling and simulation of the robotic system is performed through interfacing ROS and Gazebo simulator, aiming for off-line programming, optimal control system design, and system performance analysis. Gazebo simulator provides 3D simulation of the robotic system, sensors, and control interfaces. It also enables simulation of the world environment, allowing the simulated robot to operate in a modelled environment. The model based controller for kinematic control of the non-holonomic omnidirectional platform is tested and validated through experimental results obtained from the simulated and the physical robot. The challenges of the kinematic model based controller including the mathematical and kinematic singularities are discussed and the solution to enable an optimal kinematic model based controller is presented. The kinematic singularity associated with the non-holonomic omnidirectional robots is solved using a novel fuzzy logic based approach. The proposed approach is successfully validated and tested through the simulation and experimental results. Development of a reliable localization system is aimed to enable navigation of the platform in GPS-denied environments such as orchards. For this aim, stereo visual odometry (SVO) is considered as the core of the non-GPS localization system. Challenges of SVO are introduced and the SVO accumulative drift is considered as the main challenge to overcome. SVO drift is identified in form of rotational and translational drift. Sensor fusion is employed to improve the SVO rotational drift through the integration of IMU and SVO. A novel machine learning approach is proposed to improve the SVO translational drift using Neural-Fuzzy system and RBF neural network. The machine learning system is formulated as a drift estimator for each image frame, then correction is applied at that frame to avoid the accumulation of the drift over time. The experimental results and analyses are presented to validate the effectiveness of the methodology in improving the SVO accuracy. An enhanced SVO is aimed through combination of sensor fusion and machine learning methods to improve the SVO rotational and translational drifts. Furthermore, to achieve a robust non-GPS localization system for the platform, sensor fusion of the wheel odometry and the enhanced SVO is performed to increase the accuracy of the overall system, as well as the robustness of the non-GPS localization system. The experimental results and analyses are conducted to support the methodology