Learning Pose Estimation for UAV Autonomous Navigation and Landing Using Visual-Inertial Sensor Data

In this work, we propose a robust network-in-the-loop control system for autonomous navigation and landing of an Unmanned-Aerial-Vehicle (UAV). To estimate the UAV’s absolute pose, we develop a deep neural network (DNN) architecture for visual-inertial odometry, which provides a robust alternative to traditional methods. We first evaluate the accuracy of the estimation by comparing the prediction of our model to traditional visual-inertial approaches on the publicly available EuRoC MAV dataset. The results indicate a clear improvement in the accuracy of the pose estimation up to 25% over the baseline. Finally, we integrate the data-driven estimator in the closed-loop flight control system of Airsim, a simulator available as a plugin for Unreal Engine, and we provide simulation results for autonomous navigation and landing

Vision-based SLAM for the aerial robot ErleCopter

El objetivo principal de este trabajo, es la implementación de distintos tipos de algoritmos SLAM (mapeado y localización simultáneos) de visión monocular en el robot aéreo ErleCopter, empleando la plataforma software ROS (Robotic Operating System). Para ello se han escogido un conjunto de tres algoritmos ampliamente utilizados en el campo de la visión artificial: PTAM, ORB-SLAM y LSD-SLAM. Así se llevará a cabo un estudio del funcionamiento de los mismos en el ErleCopter. Además empleando dichos algoritmos, y fusionando la información extraída por estos con la información de otros sensores presentes en la plataforma robótica, se realizará un EKF (Extended Kalman Filter), de forma que podamos predecir la localización del robot de una manera más exacta en entornos interiores, ante la ausencia de sistemas GPS. Para comprobar el funcionamiento del sistema se empleará la plataforma de simulación robótica Gazebo. Por último se realizarán pruebas con el robot real, de forma que podamos observar y extraer conclusiones del funcionamiento de estos algoritmos sobre el propio ErleCopter.The main objective of this thesis is the implementation of different SLAM (Simultaneous Localization and Mapping) algorithms within the aerial robot ErleCopter, using the software platform ROS (Robotic Operating System). To do so, a bunch of three widely known and used algorithms in the field of the artificial vision have been chosen: PTAM, ORB-SLAM y LSD-SALM. So a study of the performance of such algorithms will be carried out in this way. Besides, working with such algorithms and fusing their information with the one obtained by other sensors existing within the robotic platform, an EKF (Extended Kalman Filter) will be carried out, in order to localize the robot more accurately in indoor environments, given the lack of GPS. To test the performance of the system, the robotic platform Gazebo will be used in this project. Finally tests will be made with the real robot, in order to observe and draw conclusions from the performance of these algorithms within the ErleCopter itself.Máster Universitario en Ingeniería Industrial (M141

Autonomous Tissue Scanning under Free-Form Motion for Intraoperative Tissue Characterisation

In Minimally Invasive Surgery (MIS), tissue scanning with imaging probes is required for subsurface visualisation to characterise the state of the tissue. However, scanning of large tissue surfaces in the presence of deformation is a challenging task for the surgeon. Recently, robot-assisted local tissue scanning has been investigated for motion stabilisation of imaging probes to facilitate the capturing of good quality images and reduce the surgeon's cognitive load. Nonetheless, these approaches require the tissue surface to be static or deform with periodic motion. To eliminate these assumptions, we propose a visual servoing framework for autonomous tissue scanning, able to deal with free-form tissue deformation. The 3D structure of the surgical scene is recovered and a feature-based method is proposed to estimate the motion of the tissue in real-time. A desired scanning trajectory is manually defined on a reference frame and continuously updated using projective geometry to follow the tissue motion and control the movement of the robotic arm. The advantage of the proposed method is that it does not require the learning of the tissue motion prior to scanning and can deal with free-form deformation. We deployed this framework on the da Vinci surgical robot using the da Vinci Research Kit (dVRK) for Ultrasound tissue scanning. Since the framework does not rely on information from the Ultrasound data, it can be easily extended to other probe-based imaging modalities.Comment: 7 pages, 5 figures, ICRA 202

Design and implementation of a domestic disinfection robot based on 2D lidar

In the battle against the Covid-19, the demand for disinfection robots in China and other countries has increased rapidly. Manual disinfection is time-consuming, laborious, and has safety hazards. For large public areas, the deployment of human resources and the effectiveness of disinfection face significant challenges. Using robots for disinfection therefore becomes an ideal choice. At present, most disinfection robots on the market use ultraviolet or disinfectant to disinfect, or both. They are mostly put into service in hospitals, airports, hotels, shopping malls, office buildings, or other places with daily high foot traffic. These robots are often built-in with automatic navigation and intelligent recognition, ensuring day-to-day operations. However, they usually are expensive and need regular maintenance. The sweeping robots and window-cleaning robots have been put into massive use, but the domestic disinfection robots have not gained much attention. The health and safety of a family are also critical in epidemic prevention. This thesis proposes a low-cost, 2D lidar-based domestic disinfection robot and implements it. The robot possesses dry fog disinfection, ultraviolet disinfection, and air cleaning. The thesis is mainly engaged in the following work: The design and implementation of the control board of the robot chassis are elaborated in this thesis. The control board uses STM32F103ZET6 as the MCU. Infrared sensors are used in the robot to prevent from falling over and walk along the wall. The Ultrasonic sensor is installed in the front of the chassis to detect and avoid the path's obstacles. Photoelectric switches are used to record the information when the potential collisions happen in the early phase of mapping. The disinfection robot adopts a centrifugal fan and HEPA filter for air purification. The ceramic atomizer is used to break up the disinfectant's molecular structure to produce the dry fog. The UV germicidal lamp is installed at the bottom of the chassis to disinfect the ground. The robot uses an air pollution sensor to estimate the air quality. Motors are used to drive the chassis to move. The lidar transmits its data to the navigation board directly through the wires and the edge-board contact on the control board. The control board also manages the atmosphere LEDs, horn, press-buttons, battery, LDC, and temperature-humidity sensor. It exchanges data with and executes the command from the navigation board and manages all kinds of peripheral devices. Thus, it is the administrative unit of the disinfection robot. Moreover, the robot is designed in a way that reduces costs while ensuring quality. The control board’s embedded software is realized and analyzed in the thesis. The communication protocol that links the control board and the navigation board is implemented in software. Standard commands, specific commands, error handling, and the data packet format are detailed and processed in software. The software effectively drives and manages the peripheral devices. SLAMWARE CORE is used as the navigation board to complete the system design. System tests like disinfecting, mapping, navigating, and anti-falling were performed to polish and adjust the structure and functionalities of the robot. Raspberry Pi is also used with the control board to explore 2D Simultaneous Localization and Mapping (SLAM) algorithms, such as Hector, Karto, and Cartographer, in Robot Operating System (ROS) for the robot’s further development. The thesis is written from the perspective of engineering practice and proposes a feasible design for a domestic disinfection robot. Hardware, embedded software, and system tests are covered in the thesis

A distributed architecture for unmanned aerial systems based on publish/subscribe messaging and simultaneous localisation and mapping (SLAM) testbed

A dissertation submitted in fulfilment for the degree of Master of Science. School of Computational and Applied Mathematics, University of the Witwatersrand, Johannesburg, South Africa, November 2017The increased capabilities and lower cost of Micro Aerial Vehicles (MAVs) unveil big opportunities for a rapidly growing number of civilian and commercial applications. Some missions require direct control using a receiver in a point-to-point connection, involving one or very few MAVs. An alternative class of mission is remotely controlled, with the control of the drone automated to a certain extent using mission planning software and autopilot systems. For most emerging missions, there is a need for more autonomous, cooperative control of MAVs, as well as more complex data processing from sensors like cameras and laser scanners. In the last decade, this has given rise to an extensive research from both academia and industry. This research direction applies robotics and computer vision concepts to Unmanned Aerial Systems (UASs). However, UASs are often designed for specific hardware and software, thus providing limited integration, interoperability and re-usability across different missions. In addition, there are numerous open issues related to UAS command, control and communication(C3), and multi-MAVs. We argue and elaborate throughout this dissertation that some of the recent standardbased publish/subscribe communication protocols can solve many of these challenges and meet the non-functional requirements of MAV robotics applications. This dissertation assesses the MQTT, DDS and TCPROS protocols in a distributed architecture of a UAS control system and Ground Control Station software. While TCPROS has been the leading robotics communication transport for ROS applications, MQTT and DDS are lightweight enough to be used for data exchange between distributed systems of aerial robots. Furthermore, MQTT and DDS are based on industry standards to foster communication interoperability of “things”. Both protocols have been extensively presented to address many of today’s needs related to networks based on the internet of things (IoT). For example, MQTT has been used to exchange data with space probes, whereas DDS was employed for aerospace defence and applications of smart cities. We designed and implemented a distributed UAS architecture based on each publish/subscribe protocol TCPROS, MQTT and DDS. The proposed communication systems were tested with a vision-based Simultaneous Localisation and Mapping (SLAM) system involving three Parrot AR Drone2 MAVs. Within the context of this study, MQTT and DDS messaging frameworks serve the purpose of abstracting UAS complexity and heterogeneity. Additionally, these protocols are expected to provide low-latency communication and scale up to meet the requirements of real-time remote sensing applications. The most important contribution of this work is the implementation of a complete distributed communication architecture for multi-MAVs. Furthermore, we assess the viability of this architecture and benchmark the performance of the protocols in relation to an autonomous quadcopter navigation testbed composed of a SLAM algorithm, an extended Kalman filter and a PID controller.XL201