A sensor aided H.264 encoder tested on aerial imagery for SFM

Email Print Request Permissions Standard video coding systems currently employed in UAV (Unmanned Aerial Vehicle) and aerial drone applications do not rely on some peculiarities in terms of scene 3D model and correlation among successive frames. In particular, the observed scene is static, i.e. the camera movement is dominant, and it can often be well approximated with a plane. Moreover, camera position and orientation can be obtained from the navigation system. Therefore, correspondent points on two video frames are linked by a simple homography. This paper presents novel results obtained by a low-complexity sensor aided H.264 encoder, recently developed at CIRA and yet tested on simulated data. The proposed encoder employs a new motion estimation scheme which make use of the global motion information provided by the onboard navigation system. The homography is used in order to initialize the block matching algorithm allowing a more robust motion estimation and a smaller search window, and hence reducing the complexity. The tests are made coding real aerial imagery, captured to be used for 3D scene reconstruction. The images are acquired by an high resolution camera mounted on a small drone, flying at low altitude

Robotic 3D Reconstruction Utilising Structure from Motion

Sensing the real-world is a well-established and continual problem in the field of robotics. Investigations into autonomous aerial and underwater vehicles have extended this challenge into sensing, mapping and localising in three dimensions. This thesis seeks to understand and tackle the challenges of recovering 3D information from an environment using vision alone. There is a well-established literature on the principles of doing this, and some impressive demonstrations; but this thesis explores the practicality of doing vision-based 3D reconstruction using multiple, mobile robotic platforms, the emphasis being on producing accurate 3D models. Typically, robotic platforms such as UAVs have a single on-board camera, restricting which method of visual 3D recovery can be employed. This thesis specifically explores Structure from Motion, a monocular 3D reconstruction technique which produces detailed and accurate, although slow to calculate, 3D reconstructions. It examines how well proof-of-concept demonstrations translate onto the kinds of robotic systems that are commonly deployed in the real world, where local processing is limited and network links have restricted capacity. In order to produce accurate 3D models, it is necessary to use high-resolution imagery, and the difficulties of working with this on remote robotic platforms is explored in some detail

Machine Learning-Aided Operations and Communications of Unmanned Aerial Vehicles: A Contemporary Survey

The ongoing amalgamation of UAV and ML techniques is creating a significant synergy and empowering UAVs with unprecedented intelligence and autonomy. This survey aims to provide a timely and comprehensive overview of ML techniques used in UAV operations and communications and identify the potential growth areas and research gaps. We emphasise the four key components of UAV operations and communications to which ML can significantly contribute, namely, perception and feature extraction, feature interpretation and regeneration, trajectory and mission planning, and aerodynamic control and operation. We classify the latest popular ML tools based on their applications to the four components and conduct gap analyses. This survey also takes a step forward by pointing out significant challenges in the upcoming realm of ML-aided automated UAV operations and communications. It is revealed that different ML techniques dominate the applications to the four key modules of UAV operations and communications. While there is an increasing trend of cross-module designs, little effort has been devoted to an end-to-end ML framework, from perception and feature extraction to aerodynamic control and operation. It is also unveiled that the reliability and trust of ML in UAV operations and applications require significant attention before full automation of UAVs and potential cooperation between UAVs and humans come to fruition.Comment: 36 pages, 304 references, 19 Figure

Feature Papers of Drones - Volume II

[EN] The present book is divided into two volumes (Volume I: articles 1–23, and Volume II: articles 24–54) which compile the articles and communications submitted to the Topical Collection ”Feature Papers of Drones” during the years 2020 to 2022 describing novel or new cutting-edge designs, developments, and/or applications of unmanned vehicles (drones). Articles 24–41 are focused on drone applications, but emphasize two types: firstly, those related to agriculture and forestry (articles 24–35) where the number of applications of drones dominates all other possible applications. These articles review the latest research and future directions for precision agriculture, vegetation monitoring, change monitoring, forestry management, and forest fires. Secondly, articles 36–41 addresses the water and marine application of drones for ecological and conservation-related applications with emphasis on the monitoring of water resources and habitat monitoring. Finally, articles 42–54 looks at just a few of the huge variety of potential applications of civil drones from different points of view, including the following: the social acceptance of drone operations in urban areas or their influential factors; 3D reconstruction applications; sensor technologies to either improve the performance of existing applications or to open up new working areas; and machine and deep learning development

UAVs for the Environmental Sciences

This book gives an overview of the usage of UAVs in environmental sciences covering technical basics, data acquisition with different sensors, data processing schemes and illustrating various examples of application

Robotic navigation and inspection of bridge bearings

This thesis focuses on the development of a robotic platform for bridge bearing inspection. The existing literature on this topic highlights an aspiration for increased automation of bridge inspection, due to an increasing amount of ageing infrastructure and costly inspection. Furthermore, bridge bearings are highlighted as being one of the most costly components of the bridge to maintain. However, although autonomous robotic inspection is often stated as an aspiration, the existing literature for robotic bridge inspection often neglects to include the requirement of autonomous navigation. To achieve autonomous inspection, some methods for mapping and localising in the bridge structure are required. This thesis compares existing methods for simultaneous localisation and mapping (SLAM) with localisation-only methods. In addition, a method for using pre-existing data to create maps for localisation is proposed. A robotic platform was developed and these methods for localisation and mapping were then compared in a laboratory environment and then in a real bridge environment. The errors in the bridge environment are greater than in the laboratory environment, but remained within a defined error bound. A combined approach is suggested as an appropriate method for combining the lower errors of a SLAM approach with the advantages of a localisation approach for defining existing goals. Longer-term testing in a real bridge environment is still required. The use of existing inspection data is then extended to the creation of a simulation environment, with the goal of creating a methodology for testing different configurations of bridges or robots in a more realistic environment than laboratory testing, or other existing simulation environments. Finally, the inspection of the structure surrounding the bridge bearing is considered, with a particular focus on the detection and segmentation of cracks in concrete. A deep learning approach is used to segment cracks from an existing dataset and compared to an existing machine learning approach, with the deep-learning approach achieving a higher performance using a pixel-based evaluation. Other evaluation methods were also compared that take the structure of the crack, and other related datasets, into account. The generalisation of the approach for crack segmentation is evaluated by comparing the results of the trained on different datasets. Finally, recommendations for improving the datasets to allow better comparisons in future work is given

Detailed Three-Dimensional Building Façade Reconstruction: A Review on Applications, Data and Technologies

Urban environments are regions of complex and diverse architecture. Their reconstruction and representation as three-dimensional city models have attracted the attention of many researchers and industry specialists, as they increasingly recognise the potential for new applications requiring detailed building models. Nevertheless, despite being investigated for a few decades, the comprehensive reconstruction of buildings remains a challenging task. While there is a considerable body of literature on this topic, including several systematic reviews summarising ways of acquiring and reconstructing coarse building structures, there is a paucity of in-depth research on the detection and reconstruction of façade openings (i.e., windows and doors). In this review, we provide an overview of emerging applications, data acquisition and processing techniques for building façade reconstruction, emphasising building opening detection. The use of traditional technologies from terrestrial and aerial platforms, along with emerging approaches, such as mobile phones and volunteered geography information, is discussed. The current status of approaches for opening detection is then examined in detail, separated into methods for three-dimensional and two-dimensional data. Based on the review, it is clear that a key limitation associated with façade reconstruction is process automation and the need for user intervention. Another limitation is the incompleteness of the data due to occlusion, which can be reduced by data fusion. In addition, the lack of available diverse benchmark datasets and further investigation into deep-learning methods for façade openings extraction present crucial opportunities for future research

Depth Estimation Using 2D RGB Images

Single image depth estimation is an ill-posed problem. That is, it is not mathematically possible to uniquely estimate the 3rd dimension (or depth) from a single 2D image. Hence, additional constraints need to be incorporated in order to regulate the solution space. As a result, in the first part of this dissertation, the idea of constraining the model for more accurate depth estimation by taking advantage of the similarity between the RGB image and the corresponding depth map at the geometric edges of the 3D scene is explored. Although deep learning based methods are very successful in computer vision and handle noise very well, they suffer from poor generalization when the test and train distributions are not close. While, the geometric methods do not have the generalization problem since they benefit from temporal information in an unsupervised manner. They are sensitive to noise, though. At the same time, explicitly modeling of a dynamic scenes as well as flexible objects in traditional computer vision methods is a big challenge. Considering the advantages and disadvantages of each approach, a hybrid method, which benefits from both, is proposed here by extending traditional geometric models’ abilities to handle flexible and dynamic objects in the scene. This is made possible by relaxing geometric computer vision rules from one motion model for some areas of the scene into one for every pixel in the scene. This enables the model to detect even small, flexible, floating debris in a dynamic scene. However, it makes the optimization under-constrained. To change the optimization from under-constrained to over-constrained while maintaining the model’s flexibility, ”moving object detection loss” and ”synchrony loss” are designed. The algorithm is trained in an unsupervised fashion. The primary results are in no way comparable to the current state of the art. Because the training process is so slow, it is difficult to compare it to the current state of the art. Also, the algorithm lacks stability. In addition, the optical flow model is extremely noisy and naive. At the end, some solutions are suggested to address these issues

A COMPARATIVE ANALYSIS OF GLOBAL POSITIONING SYSTEM SCHEMES BASED ON BLOCK CODES

Global Positioning System (GPS) is a satellite based positioning system based on radio ranging technique. The GPS will provide very accurate three-dimensional position, velocity and timing information to users anywhere in the world. GPS can also be used in other applications such as vehicle monitoring for traffic management in urban areas, Geographical Information System (GIS), 4G Communications, marine navigation, search and rescue and military applications. As GPS accuracy is limited by ionospheric effects, this course also covers the basics of ionosphere and its effects on GPS. Navigation is the art of directing a vehicle such as aircraft or a person from one point to another point. Some of the prominent advantages of the GPS are: Land based system problems like ground reflections, electromagnetic interference, reflections from physical systems are avoided in GPS since it is space constellation, Intentional interference like jamming, unintentional interference will not affect GPS since spread spectrum techniques are used in it, System accuracy can be improved to the order of centimeters using differential techniques, Smaller size and reduced cost of the GPS receiver enable it to be used in 3G Communication. In this paper, a literature review of existing GPS schemes based on block codes that mainly targets towards finding out the tolerance to signals from other GPS satellites sharing the same frequency band (multiple access capability), analyzing the tolerance to some level of multipath interference, there are many potential sources of multipath reflection (example man-made or natural object) and finding out the tolerance to reasonable levels of unintentional or intentional interference, jamming or spoofing by signal designed to mimic a GPS signal