Real-time on-board obstacle avoidance for UAVs based on embedded stereo vision

In order to improve usability and safety, modern unmanned aerial vehicles (UAVs) are equipped with sensors to monitor the environment, such as laser-scanners and cameras. One important aspect in this monitoring process is to detect obstacles in the flight path in order to avoid collisions. Since a large number of consumer UAVs suffer from tight weight and power constraints, our work focuses on obstacle avoidance based on a lightweight stereo camera setup. We use disparity maps, which are computed from the camera images, to locate obstacles and to automatically steer the UAV around them. For disparity map computation we optimize the well-known semi-global matching (SGM) approach for the deployment on an embedded FPGA. The disparity maps are then converted into simpler representations, the so called U-/V-Maps, which are used for obstacle detection. Obstacle avoidance is based on a reactive approach which finds the shortest path around the obstacles as soon as they have a critical distance to the UAV. One of the fundamental goals of our work was the reduction of development costs by closing the gap between application development and hardware optimization. Hence, we aimed at using high-level synthesis (HLS) for porting our algorithms, which are written in C/C++, to the embedded FPGA. We evaluated our implementation of the disparity estimation on the KITTI Stereo 2015 benchmark. The integrity of the overall realtime reactive obstacle avoidance algorithm has been evaluated by using Hardware-in-the-Loop testing in conjunction with two flight simulators.Comment: Accepted in the International Archives of the Photogrammetry, Remote Sensing and Spatial Information Scienc

Toward a unified PNT, Part 1: Complexity and context: Key challenges of multisensor positioning

The next generation of navigation and positioning systems must provide greater accuracy and reliability in a range of challenging environments to meet the needs of a variety of mission-critical applications. No single navigation technology is robust enough to meet these requirements on its own, so a multisensor solution is required. Known environmental features, such as signs, buildings, terrain height variation, and magnetic anomalies, may or may not be available for positioning. The system could be stationary, carried by a pedestrian, or on any type of land, sea, or air vehicle. Furthermore, for many applications, the environment and host behavior are subject to change. A multi-sensor solution is thus required. The expert knowledge problem is compounded by the fact that different modules in an integrated navigation system are often supplied by different organizations, who may be reluctant to share necessary design information if this is considered to be intellectual property that must be protected

SaPPART White paper. Better use of Global Navigation Satellite Systems for safer and greener transport

Transport and mobility services are crucial to the society that faces important challenges. Up to date, transport facilities and services have been fundamental to economic growth. However, there have significant and unacceptable negative impacts on the environment including pollution, noise and climate change. Therefore, it is paramount that the efficiency of the transport system is improved significantly including lower consumption of energy. A way of achieving this is through the concept of smart transport that exploits Intelligent Transport Systems (ITS) technology. ITS are built on three technology pillars: information, communication and positioning technologies. Of the three technologies, positioning could be argued to be the least familiar amongst transport stakeholders. However, a quick investigation reveals that there are a wide variety of transport and related services often associated with communication technologies that are supported by positioning. Currently, the positioning is provided in the majority of the cases by Global Navigation Satellite System (GNSS), among which the Global Positioning System (GPS) is the pioneer and still the most widely used system. The other current fully operational stand-alone system is Russia’s GLONASS. As these operational systems were not originally and specifically designed for transport applications, the actual capabilities and limitations of the current GNSS are not fully understood by many stakeholders. Therefore, better knowledge of these limitations and their resolution should enable a much more rapid deployment of ITS. This white paper is produced by the members of the COST Action SaPPART with two principal aims. The first is to explain the principles, state-of-the-art performance of GNSS technology and added value in the field of transport. The second aim is to deliver key messages to the stakeholders to facilitate the deployment of GNSS technology and thus contribute to the development of smarter and greener transport systems. The first chapter highlights the important role of positioning in today transport systems and the added value of accurate and reliable positioning for critical systems. The second chapter is about positioning technologies for transport: GNSS and their different aiding and augmentation methods are described, but the other complementary technologies are also introduced. The third and last chapter is about the management of performances inside a positioning-based intelligent transport system, between the positioning system itself and the application-specific part of the system which processes the raw position for delivering its service

Human Motion Trajectory Prediction: A Survey

With growing numbers of intelligent autonomous systems in human environments, the ability of such systems to perceive, understand and anticipate human behavior becomes increasingly important. Specifically, predicting future positions of dynamic agents and planning considering such predictions are key tasks for self-driving vehicles, service robots and advanced surveillance systems. This paper provides a survey of human motion trajectory prediction. We review, analyze and structure a large selection of work from different communities and propose a taxonomy that categorizes existing methods based on the motion modeling approach and level of contextual information used. We provide an overview of the existing datasets and performance metrics. We discuss limitations of the state of the art and outline directions for further research.Comment: Submitted to the International Journal of Robotics Research (IJRR), 37 page

Towards the Implementation of a MPC-based Planner on an Autonomous All-Terrain Vehicle

Planning and control for a wheeled mobile robot are challenging problems when poorly traversable terrains, including dynamic obstacles, are considered. To accomplish a mission, the control system should firstly guarantee the vehicle integrity, for example with respect to possible roll-over/tip-over phenomena. A fundamental contribution to achieve this goal, however, comes from the planner as well. In fact, computing a path that takes into account the terrain traversability, the kinematic and dynamic vehicle constraints, and the presence of dynamic obstacles, is a first and crucial step towards ensuring the vehicle integrity. The present paper addresses some of the aforementioned issues, describing the hardware/software architecture of the planning and control system of an autonomous All-Terrain Mobile Robot and the implementation of a real-time path planner

Continuous Autonomous UAV Inspection for FPSO vessels

This Master's thesis represents the preliminary design study and proposes the unmanned aerial vehicle (UAV) -based inspection framework, comprising several multirotors with automatic charging and deployment for 24/7 integrity inspection tasks. This project has three main topics. First one describes the operational environment and existing regulations that cover use of UAVs. It forms the basis for proposal of the relevant use-case scenarios. Third part comprises two chapters, where design of concept and framework is being based on the previous factors. It shows that before implementation of fully autonomous inspection system, there is a need to cover both regulatory and technical gaps. It can be explained by the fact that there does not exist any autonomous inspection system today. Thus, this project can be seen as a base for future development of the UAV-based inspection system, as it focuses on creation of a general framework

Survey on Recent Advances in Integrated GNSSs Towards Seamless Navigation Using Multi-Sensor Fusion Technology

During the past few decades, the presence of global navigation satellite systems (GNSSs) such as GPS, GLONASS, Beidou and Galileo has facilitated positioning, navigation and timing (PNT) for various outdoor applications. With the rapid increase in the number of orbiting satellites per GNSS, enhancements in the satellite-based augmentation systems (SBASs) such as EGNOS and WAAS, as well as commissioning new GNSS constellations, the PNT capabilities are maximized to reach new frontiers. Additionally, the recent developments in precise point positioning (PPP) and real time kinematic (RTK) algorithms have provided more feasibility to carrier-phase precision positioning solutions up to the third-dimensional localization. With the rapid growth of internet of things (IoT) applications, seamless navigation becomes very crucial for numerous PNT dependent applications especially in sensitive fields such as safety and industrial applications. Throughout the years, GNSSs have maintained sufficiently acceptable performance in PNT, in RTK and PPP applications however GNSS experienced major challenges in some complicated signal environments. In many scenarios, GNSS signal suffers deterioration due to multipath fading and attenuation in densely obscured environments that comprise stout obstructions. Recently, there has been a growing demand e.g. in the autonomous-things domain in adopting reliable systems that accurately estimate position, velocity and time (PVT) observables. Such demand in many applications also facilitates the retrieval of information about the six degrees of freedom (6-DOF - x, y, z, roll, pitch, and heading) movements of the target anchors. Numerous modern applications are regarded as beneficiaries of precise PNT solutions such as the unmanned aerial vehicles (UAV), the automatic guided vehicles (AGV) and the intelligent transportation system (ITS). Hence, multi-sensor fusion technology has become very vital in seamless navigation systems owing to its complementary capabilities to GNSSs. Fusion-based positioning in multi-sensor technology comprises the use of multiple sensors measurements for further refinement in addition to the primary GNSS, which results in high precision and less erroneous localization. Inertial navigation systems (INSs) and their inertial measurement units (IMUs) are the most commonly used technologies for augmenting GNSS in multi-sensor integrated systems. In this article, we survey the most recent literature on multi-sensor GNSS technology for seamless navigation. We provide an overall perspective for the advantages, the challenges and the recent developments of the fusion-based GNSS navigation realm as well as analyze the gap between scientific advances and commercial offerings. INS/GNSS and IMU/GNSS systems have proven to be very reliable in GNSS-denied environments where satellite signal degradation is at its peak, that is why both integrated systems are very abundant in the relevant literature. In addition, the light detection and ranging (LiDAR) systems are widely adopted in the literature for its capability to provide 6-DOF to mobile vehicles and autonomous robots. LiDARs are very accurate systems however they are not suitable for low-cost positioning due to the expensive initial costs. Moreover, several other techniques from the radio frequency (RF) spectrum are utilized as multi-sensor systems such as cellular networks, WiFi, ultra-wideband (UWB) and Bluetooth. The cellular-based systems are very suitable for outdoor navigation applications while WiFi-based, UWB-based and Bluetooth-based systems are efficient in indoor positioning systems (IPS). However, to achieve reliable PVT estimations in multi-sensor GNSS navigation, optimal algorithms should be developed to mitigate the estimation errors resulting from non-line-of-sight (NLOS) GNSS situations. Examples of the most commonly used algorithms for trilateration-based positioning are Kalman filters, weighted least square (WLS), particle filters (PF) and many other hybrid algorithms by mixing one or more algorithms together. In this paper, the reviewed articles under study and comparison are presented by highlighting their motivation, the methodology of implementation, the modelling utilized and the performed experiments. Then they are assessed with respect to the published results focusing on achieved accuracy, robustness and overall implementation cost-benefits as performance metrics. Our summarizing survey assesses the most promising, highly ranked and recent articles that comprise insights into the future of GNSS technology with multi-sensor fusion technique.©2021 The Authors. Published by ION.fi=vertaisarvioimaton|en=nonPeerReviewed