use of fisheye parrot bebop 2 images for 3d modelling using commercial photogrammetric software

Fisheye camera installed on-board mass market UAS are becoming very popular and it is more and more frequent the use of such platforms for photogrammetric purposes. The interest of wide-angles images for 3D modelling is confirmed by the introduction of fisheye models in several commercial software packages. The paper exploits the different mathematical models implemented in the most famous commercial photogrammetric software packages, highlighting the different processing pipelines and analysing the achievable results in terms of checkpoint residuals, as well as the quality of the delivered 3D point clouds. A two-step approach based on the creation of undistorted images has been tested too. An experimental test has been carried out using a Parrot Bebop 2 UAS by performing a flight over an historical complex located near Piacenza (Northern Italy), which is characterized by the simultaneous presence of horizontal, vertical and oblique surfaces. Different flight configurations have been tested to evaluate the potentiality and possible drawbacks of the previously mentioned UAS platform. Results confirmed that the fisheye images acquired with the Parrot Bebop 2 are suitable for 3D modelling, ensuring accuracies of the photogrammetric blocks of the order of the GSD (about 0.05 m normal to the optic axis in case of a flight height equal to 35 m). The generated point clouds have been compared to a reference scan, acquired by means of a MS60 MultiStation, resulting in differences below 0.05 in all directions

Real-Time Hardware-in-the-Loop Laboratory Testing for Multisensor Sense and Avoid Systems

This paper focuses on a hardware-in-the-loop facility aimed at real-time testing of architectures and algorithms of multisensor sense and avoid systems. It was developed within a research project aimed at flight demonstration of autonomous non-cooperative collision avoidance for Unmanned Aircraft Systems. In this framework, an optionally piloted Very Light Aircraft was used as experimental platform. The flight system is based on multiple-sensor data integration and it includes a Ka-band radar, four electro-optical sensors, and two dedicated processing units. The laboratory test system was developed with the primary aim of prototype validation before multi-sensor tracking and collision avoidance flight tests. System concept, hardware/software components, and operating modes are described in the paper. The facility has been built with a modular approach including both flight hardware and simulated systems and can work on the basis of experimentally tested or synthetically generated scenarios. Indeed, hybrid operating modes are also foreseen which enable performance assessment also in the case of alternative sensing architectures and flight scenarios that are hardly reproducible during flight tests. Real-time multisensor tracking results based on flight data are reported, which demonstrate reliability of the laboratory simulation while also showing the effectiveness of radar/electro-optical fusion in a non-cooperative collision avoidance architecture

Radar/electro-optical data fusion for non-cooperative UAS sense and avoid

Abstract This paper focuses on hardware/software implementation and flight results relevant to a multi-sensor obstacle detection and tracking system based on radar/electro-optical (EO) data fusion. The sensing system was installed onboard an optionally piloted very light aircraft (VLA). Test flights with a single intruder plane of the same class were carried out to evaluate the level of achievable situational awareness and the capability to support autonomous collision avoidance. System architecture is presented and special emphasis is given to adopted solutions regarding real time integration of sensors and navigation measurements and high accuracy estimation of sensors alignment. On the basis of Global Positioning System (GPS) navigation data gathered simultaneously with multi-sensor tracking flight experiments, potential of radar/EO fusion is compared with standalone radar tracking. Flight results demonstrate a significant improvement of collision detection performance, mostly due to the change in angular rate estimation accuracy, and confirm data fusion effectiveness for facing EO detection issues. Relative sensors alignment, performance of the navigation unit, and cross-sensor cueing are found to be key factors to fully exploit the potential of multi-sensor architectures

Particle Filtering for Obstacle Tracking in UAS Sense and Avoid Applications

Obstacle detection and tracking is a key function for UAS sense and avoid applications. In fact, obstacles in the flight path must be detected and tracked in an accurate and timely manner in order to execute a collision avoidance maneuver in case of collision threat. The most important parameter for the assessment of a collision risk is the Distance at Closest Point of Approach, that is, the predicted minimum distance between own aircraft and intruder for assigned current position and speed. Since assessed methodologies can cause some loss of accuracy due to nonlinearities, advanced filtering methodologies, such as particle filters, can provide more accurate estimates of the target state in case of nonlinear problems, thus improving system performance in terms of collision risk estimation. The paper focuses on algorithm development and performance evaluation for an obstacle tracking system based on a particle filter. The particle filter algorithm was tested in off-line simulations based on data gathered during flight tests. In particular, radar-based tracking was considered in order to evaluate the impact of particle filtering in a single sensor framework. The analysis shows some accuracy improvements in the estimation of Distance at Closest Point of Approach, thus reducing the delay in collision detection

Technological advances relevant to transport – understanding what drives them

Transport policy makers are increasingly perplexed by the pace of change in their sector and by the increasing influence of external actors. This leads to a variety of responses, including “business as usual”, technological optimism, technological fatalism and technological ignorance. To explore this perplexity and its justification, we examine four areas of technological advance relevant to transport: mobility as a service; unmanned aerial vehicles (drones); automated vehicles; and telehealth. In each case, we identify the principal underlying shifts which are driving these technological advances, concluding that there is considerable overlap: three of the advances rely on ubiquitous sensing and on artificial intelligence and all four rely, to some degree, on connectedness. We then explore these three “drivers”, finding that progress is steadier than may be generally thought. We discuss the implications for our set of transport-related technological developments, concluding that policy makers could approach the future with greater confidence than is currently typical. They could also draw on the concepts of anticipatory governance to support their management of emerging technology and, at the same time, of the influence of external actors

Innovative Tools For Planning, Analysis, and Management of UAV Photogrammetric Surveys

The Unmanned Aerial System (UAV) is widely used in the photogrammetric surveys both for structures and small areas. The geomatics approach, for the several applications where the 3D modeling is required, focuses the attention on the metric quality of the final products of the survey. As widely known, the quality of results derives from the quality of images acquisition phase, which needs an accurate planning phase. Actually, the planning phase is typically managed using dedicated tools, adapted from the traditional aerial-photogrammetric flight plan. Unfortunately, UAV flight has features completely different from the traditional one, hence the use of UAV for photogrammetric applications today requires a growth in the planning knowledge. The basic idea of the present research work is to provide a tool for planning a photogrammetric survey with UAV, called \u201cUnmanned Photogrammetric Office\u201d (U.Ph.O.), that considers the morphology of the object, the effective visibility of its surface, in the respect of the metric precisions. The usual planning tools require the classical parameters of a photogrammetric planning: flight distance from the surface, images overlaps and geometric parameters of the camera. The created \u201cOffice suite\u201d U.Ph.O. allows a realistic planning of a photogrammetric survey, requiring additionally an approximate knowledge of the Digital Surface Model (DSM) and the attitude parameters, potentially changing along the route. The planning products will be the realistic overlapping of the images, the Ground Sample Distance (GSD) and the precision on each pixel taking into account the real geometry. The different tested procedures, the solution proposed to estimates the realistic precisions in the particular case of UAV surveys and the obtained results, are described in this thesis work, with an overview on the recently development of UAV surveys and technologies related to them

Flight test of a radar-based tracking system for UAS sense and avoid

Presented here is an analysis of an extensive flight campaign aimed at characterizing peculiarities, advantages, and limitations of an obstacle detection and tracking system based on a pulse radar. The hardware and software prototypical sensing system was installed onboard an optionally piloted flying laboratory from the very light aircraft (VLA) category. Test flights with a single intruder aircraft of the same class were carried out to demonstrate autonomous noncooperative unmanned aerial system (UAS) collision avoidance capability and to evaluate the level of achievable situational awareness. First, the adopted architecture and the developed tracking algorithm are presented. Subsequently, flight data gathered in various relative flight geometries, covering chasing flights and quasi-frontal encounters, are analyzed in terms of radar performance, including detection range and range and angle measurement accuracies. The analysis describes the impact of ground echoes and navigation uncertainties, system tracking reliability, and achievable accuracy in estimation of relative position and velocity. On the basis of Global Positioning System (GPS) data gathered simultaneously with obstacle detection flight experiments, a detailed error analysis is conducted. Special emphasis is given to the validation of proposed methodology to separate between intruder and ground echoes, which is a critical aspect for light aircraft due to their limited radar cross sections (RCS) and flight altitudes. In conclusion the radar demonstrates its potential to attain adequate situational awareness, however the limits of single sensor tracking are also pointed out. Above all the negative impact of poor angular accuracy on missed detection and false alarm rates is pointed out

Ultralight Radar Sensor for Autonomous Operations by Mini- and Micro-UAS

In recent years the boost in operations by mini- and micro-UAS (Unmanned Aircraft Systems, also known as Remotely Piloted Aircraft Systems - RPAS - or simply drones) and the successful miniaturization of electronic components were experienced. Radar sensors demonstrated to have favorable features for these operations. However, despite their ability to provide meaningful information for navigation, sense-and-avoid, and imaging tasks, currently very few radar sensors are exploited onboard or developed for autonomous operations with mini- and micro-UAS. Exploration of indoor complex, dangerous, and not easily accessible environments represents a possible application for mini-UAS based on radar technology. In this scenario, the objective of the thesis is to develop design strategies and processing approaches for a novel ultralight radar sensor able to provide the miniaturized platform with Simultaneous Localization and Mapping (SLAM) capabilities, mainly but not exclusively indoors. Millimeter-wave Interferometric Synthetic Aperture Radar (mmw InSAR) technology has been identified as a key asset. At the same time, testing of commercial lightweight radar is carried out to assess potentialities towards autonomous navigation, sense-and-avoid, and imaging. The two main research lines can be outlined as follows: - Long-term scenario: Development of very compact and ultralight Synthetic Aperture Radar able to provide mini- or micro-UAS with very accurate 3D awareness in indoor or GPS-denied complex and harsh environments. - Short-term scenario: Assessment of true potentialities of current commercial radar sensors in a UAS-oriented scenario. Within the framework of long-term scenario, after a review of state-of-art SAR sensors, Frequency-Modulated Continuous Wave (FMCW) SAR technology has been selected as preferred candidate. Design procedure tailored to this technology and software simulator for operations have been developed in MATLAB environment. Software simulator accounts for the analysis of ambiguous areas in a three-dimensional environment, different SAR focusing algorithms, and a Ray-Tracing algorithm specifically designed for indoor operations. The simulations provided relevant information on actual feasibility of the sensor, as well as mission design characteristics. Additionally, field tests have been carried out at Fraunhofer Institute FHR with a mmw SAR. Processing approaches developed from simulations proved to be effective when dealing with field tests. A very lightweight FMCW radar sensor manifactured by IMST GmbH has been tested for short-term scenario operations. The codes for data acquisition were developed in Python language both for Windows-based and GNU/Linux-based operative systems. The radar provided information on range and angle of targets in the scene, thus being interesting for radar-aided UAS navigation. Multiple-target tracking and radar odometry algorithms have been developed and tested on actual field data. Radar-only odometry provided to be effective under specific circumstances