Design and Development of an FPGA-based Hardware Accelerator for Corner Feature Extraction and Genetic Algorithm-based SLAM System

Simultaneous Localization and Mapping (SLAM) systems are crucial parts of mobile robots. These systems require a large number of computing units, have significant real-time requirements and are also a vital factor which can determine the stability, operability and power consumption of robots. This thesis aims to improve the calculation speed of a lidar-based SLAM system in domestic scenes, reduce the power consumption of the SLAM algorithm, and reduce the overall cost of the whole platform. Lightweight, low-power and parallel optimization of SLAM algorithms are researched. In the thesis, two SLAM systems are designed and developed with a focus on energy-efficient and fast hardware-level design: a geometric method based on corner extraction and a genetic algorithm-based approach. Finally, an FPGA-based hardware accelerated SLAM is implemented and realized, and compared to a software-based system. As for the front-end SLAM system, a method of using a Corner Feature Extraction (CFE) algorithm on FPGA platforms is first proposed to improve the speed of the feature extraction. Considering building a back-end SLAM system with low power consumption, a SLAM system based on genetic algorithm combined with algorithms such as Extended Kalman Filter (EKF) and FastSLAM to reduce the amount of calculation in the SLAM system is also proposed. Finally, the thesis also proposes and implements an adaptive feature map which can replace a grid point map to reduce the amount of calculation and utilization of hardware resources. In this thesis, the lidar SLAM system with front-end and back-end parts mentioned above is implemented on the Xilinx PYNQ Z2 Platform. The implementation is operated on a mobile robot prototype and evaluated in real scenes. Compared with the implementation on the Raspberry Pi 3B+, the implementation in this thesis can save 86.25% of power consumption. The lidar SLAM system only takes 20 ms for location calculation in each scan which is 5.31 times faster compared with the software implementation with EKF

Feature detection in an indoor environment using Hardware Accelerators for time-efficient Monocular SLAM

In the field of Robotics, Monocular Simultaneous Localization and Mapping (Monocular SLAM) has gained immense popularity, as it replaces large and costly sensors such as laser range finders with a single cheap camera. Additionally, the well-developed area of Computer Vision provides robust image processing algorithms which aid in developing feature detection technique for the implementation of Monocular SLAM. Similarly, in the field of digital electronics and embedded systems, hardware acceleration using FPGAs, has become quite popular. Hardware acceleration is based upon the idea of offloading certain iterative algorithms from the processor and implementing them on a dedicated piece of hardware such as an ASIC or FPGA, to speed up performance in terms of timing and to possibly reduce the net power consumption of the system. Good strides have been taken in developing massively pipelined and resource efficient hardware implementations of several image processing algorithms on FPGAs, which achieve fairly decent speed-up of the processing time. In this thesis, we have developed a very simple algorithm for feature detection in an indoor environment by means of a single camera, based on Canny Edge Detection and Hough Transform algorithms using OpenCV library, and proposed its integration with existing feature initialization technique for a complete Monocular SLAM implementation. Following this, we have developed hardware accelerators for Canny Edge Detection & Hough Transform and we have compared the timing performance of implementation in hardware (using FPGAs) with an implementation in software (using C++ and OpenCV)

Visual-Inertial Odometry on Chip: An Algorithm-and-Hardware Co-design Approach

Autonomous navigation of miniaturized robots (e.g., nano/pico aerial vehicles) is currently a grand challenge for robotics research, due to the need of processing a large amount of sensor data (e.g., camera frames) with limited on-board computational resources. In this paper we focus on the design of a visual-inertial odometry (VIO) system in which the robot estimates its ego-motion (and a landmark-based map) from on- board camera and IMU data. We argue that scaling down VIO to miniaturized platforms (without sacrificing performance) requires a paradigm shift in the design of perception algorithms, and we advocate a co-design approach in which algorithmic and hardware design choices are tightly coupled. Our contribution is four-fold. First, we discuss the VIO co-design problem, in which one tries to attain a desired resource-performance trade-off, by making suitable design choices (in terms of hardware, algorithms, implementation, and parameters). Second, we characterize the design space, by discussing how a relevant set of design choices affects the resource-performance trade-off in VIO. Third, we provide a systematic experiment-driven way to explore the design space, towards a design that meets the desired trade-off. Fourth, we demonstrate the result of the co-design process by providing a VIO implementation on specialized hardware and showing that such implementation has the same accuracy and speed of a desktop implementation, while requiring a fraction of the power.United States. Air Force Office of Scientific Research. Young Investigator Program (FA9550-16-1-0228)National Science Foundation (U.S.) (NSF CAREER 1350685

FPGA based formation control of multiple ubiquitous indoor robots

University of Technology, Sydney. Faculty of Engineering and Information Technology.This thesis explores the feasibility of using Field-Programmable Gate Array (FPGA) technology for formation control of multiple indoor robots in an ubiquitous computing environment. It is anticipated that in the future, computers will become integrated with people’s daily lives. By way of a hub of surrounding sensors, computers and embedded systems, indoor robots will receive commands from users and execute tasks such as home and office chores in a cooperative manner. Important requirements for such scenarios are power efficiency and computation reliability. The focuses of this project are on exploiting the use of the System-on-Programmable Chip technology and ambient intelligence in developing suitable control strategies for the deployment of multiple indoor robots moving in desired geometric patterns. After surveying the current problems associated with computing systems and robotics, this research was determined to design an ubiquitous robotics system using Field-Programmable Gate Array (FPGA) technology, a serial of the Register-Transfer Level (RTL) and gate level hardware for image processing, and control implementation. Work was done to develop novel, FPGA-feasible algorithms for colour identification, object detection, motion tracking, inter-robot distance estimation, trajectory generation and formation turning. These algorithms were integrated on a single FPGA chip to improve energy efficiency and real-time reliability. With the use of infrared sensors and a global high-resolution digital camera for environment sensing, all computation required for data acquisition, image processing, and closed-loop servo control was then performed on an FPGA chip as an external server. Battery-powered miniature mobile robots, Eyebots, were used as a test-bed for experiments. For realization, all the proposed algorithms were implemented and demonstrated via real-life video snapshots as shown on a PC monitor. These live images were captured from the on-board digital camera and then directly output to the monitor from a VGA interface of the FPGA platform. These together serve as the main contributions of this thesis, in both algorithm development and chip design verified by experiments. The digital circuit designs in the chip were simulated using software specifically developed for FPGAs in order to show the timing waveforms of the chip. Experimental results demonstrated the technical feasibility of the proposed architecture for initialization and maintenance of a line formation of three robots. Effectiveness was verified through the percentage usage of the chip capacity and its power consumption. The prototype of this ubiquitous robotic system could be improved for promising applications in home robotics or for concrete finishing in construction automation

Collaborative autonomy in heterogeneous multi-robot systems

As autonomous mobile robots become increasingly connected and widely deployed in different domains, managing multiple robots and their interaction is key to the future of ubiquitous autonomous systems. Indeed, robots are not individual entities anymore. Instead, many robots today are deployed as part of larger fleets or in teams. The benefits of multirobot collaboration, specially in heterogeneous groups, are multiple. Significantly higher degrees of situational awareness and understanding of their environment can be achieved when robots with different operational capabilities are deployed together. Examples of this include the Perseverance rover and the Ingenuity helicopter that NASA has deployed in Mars, or the highly heterogeneous robot teams that explored caves and other complex environments during the last DARPA Sub-T competition. This thesis delves into the wide topic of collaborative autonomy in multi-robot systems, encompassing some of the key elements required for achieving robust collaboration: solving collaborative decision-making problems; securing their operation, management and interaction; providing means for autonomous coordination in space and accurate global or relative state estimation; and achieving collaborative situational awareness through distributed perception and cooperative planning. The thesis covers novel formation control algorithms, and new ways to achieve accurate absolute or relative localization within multi-robot systems. It also explores the potential of distributed ledger technologies as an underlying framework to achieve collaborative decision-making in distributed robotic systems. Throughout the thesis, I introduce novel approaches to utilizing cryptographic elements and blockchain technology for securing the operation of autonomous robots, showing that sensor data and mission instructions can be validated in an end-to-end manner. I then shift the focus to localization and coordination, studying ultra-wideband (UWB) radios and their potential. I show how UWB-based ranging and localization can enable aerial robots to operate in GNSS-denied environments, with a study of the constraints and limitations. I also study the potential of UWB-based relative localization between aerial and ground robots for more accurate positioning in areas where GNSS signals degrade. In terms of coordination, I introduce two new algorithms for formation control that require zero to minimal communication, if enough degree of awareness of neighbor robots is available. These algorithms are validated in simulation and real-world experiments. The thesis concludes with the integration of a new approach to cooperative path planning algorithms and UWB-based relative localization for dense scene reconstruction using lidar and vision sensors in ground and aerial robots

A survey on real-time 3D scene reconstruction with SLAM methods in embedded systems

The 3D reconstruction of simultaneous localization and mapping (SLAM) is an important topic in the field for transport systems such as drones, service robots and mobile AR/VR devices. Compared to a point cloud representation, the 3D reconstruction based on meshes and voxels is particularly useful for high-level functions, like obstacle avoidance or interaction with the physical environment. This article reviews the implementation of a visual-based 3D scene reconstruction pipeline on resource-constrained hardware platforms. Real-time performances, memory management and low power consumption are critical for embedded systems. A conventional SLAM pipeline from sensors to 3D reconstruction is described, including the potential use of deep learning. The implementation of advanced functions with limited resources is detailed. Recent systems propose the embedded implementation of 3D reconstruction methods with different granularities. The trade-off between required accuracy and resource consumption for real-time localization and reconstruction is one of the open research questions identified and discussed in this paper