Using information content to select keypoints for UAV image matching

Image matching is one of the most important tasks in Unmanned Arial Vehicles (UAV) photogrammetry applications. The number and distribution of extracted keypoints play an essential role in the reliability and accuracy of image matching and orientation results. Conventional detectors generally produce too many redundant keypoints. In this paper, we study the effect of applying various information content criteria to keypoint selection tasks. For this reason, the quality measures of entropy, spatial saliency and texture coefficient are used to select keypoints extracted using SIFT, SURF, MSER and BRISK operators. Experiments are conducted using several synthetic and real UAV image pairs. Results show that the keypoint selection methods perform differently based on the applied detector and scene type, but in most cases, the precision of the matching results is improved by an average of 15%. In general, it can be said that applying proper keypoint selection techniques can improve the accuracy and efficiency of UAV image matching and orientation results. In addition to the evaluation, a new hybrid keypoint selection is proposed that combines all of the information content criteria discussed in this paper. This new screening method was also compared with those of SIFT, which showed 22% to 40% improvement for the bundle adjustment of UAV images

A new multi-criteria tie point filtering approach to increase the accuracy of UAV photogrammetry models

Extracting accurate tie points plays an essential role in the accuracy of image orientation in Unmanned Aerial Vehicle (UAV) photogrammetry. In this study, a Multi-Criteria Decision Making (MCDM) automatic filtering method is presented. Based on the quality features of a photogrammetric model, the proposed method works at the level of sparse point cloud to remove low-quality tie points for refining the orientation results. In the proposed algorithm, different factors that affect the quality of tie points are identified. The quality measures are then aggregated by applying MCDM methods and a competency score for each 3D tie point. These scores are employed in an automatic filtering approach that selects a subset of high-quality points which are then used to repeat the bundle adjustment. To evaluate the proposed algorithm, various internal and external studies were conducted on different datasets. The findings suggest that our method is both effective and reliable. In addition, in comparison to the existing filtering techniques, the proposed strategy increases the accuracy of bundle adjustment and dense point cloud generation by about 40% and 70%, respectively

Automated Two-Step Seamline Detection for Generating Large-Scale Orthophoto Mosaics from Drone Images

To generate an orthophoto mosaic from a collection of aerial images, the original images are first orthorectified individually using a Digital Surface Model (DSM). Then, they are stitched together along some determined seamlines to form the orthophoto mosaic. Determining appropriate seamlines is a critical process, as it affects the visual and geometric quality of the results. The stitching process can usually be done in frame-to-frame or multi-frame modes. Although the latter is more efficient, both still involve a lot of pre-processing, such as creating individual orthophotos, image registration, and overlap extraction. This paper presents a novel coarse-to-fine approach that directly determines the seamline network without such pre-processing. Our method has been specifically applied for UAV photogrammetry projects where, due to the large number of images and the corresponding overlaps, the orthophoto mosaic generation can be very challenging and time-consuming. We established the seamlines simultaneously for all the images through a two-step process. First, a DSM was generated, and a low-resolution grid was overlayed. Then, for each grid point, an optimal image was selected. Then, the grid cells are grouped into polygons based on their corresponding optimal image. Boundaries of these polygons established our seamline network. Thereafter, to generate the orthophoto mosaic, we overlayed a higher/full resolution grid on the top of the DSM, the optimal image of each point of which was quickly identified via our low-resolution polygons. In this approach, not only seamlines were automatically generated, but also were the need for the creation, registration, and overlap extraction of individual orthophotos. Our method was systematically compared with a conventional frame-to-frame (CF) technique from different aspects, including the number of double-mapped areas, discontinuities across the seamlines network, and the amount of processing time. The outcomes revealed a 46% decrease in orthophoto generation time and a notable reduction in the number of double-mapped areas, sawtooth effects, and object discontinuities within the constructed orthophoto mosaic

Image-Based Obstacle Detection Methods for the Safe Navigation of Unmanned Vehicles: A Review

Mobile robots lack a driver or a pilot and, thus, should be able to detect obstacles autonomously. This paper reviews various image-based obstacle detection techniques employed by unmanned vehicles such as Unmanned Surface Vehicles (USVs), Unmanned Aerial Vehicles (UAVs), and Micro Aerial Vehicles (MAVs). More than 110 papers from 23 high-impact computer science journals, which were published over the past 20 years, were reviewed. The techniques were divided into monocular and stereo. The former uses a single camera, while the latter makes use of images taken by two synchronised cameras. Monocular obstacle detection methods are discussed in appearance-based, motion-based, depth-based, and expansion-based categories. Monocular obstacle detection approaches have simple, fast, and straightforward computations. Thus, they are more suited for robots like MAVs and compact UAVs, which usually are small and have limited processing power. On the other hand, stereo-based methods use pair(s) of synchronised cameras to generate a real-time 3D map from the surrounding objects to locate the obstacles. Stereo-based approaches have been classified into Inverse Perspective Mapping (IPM)-based and disparity histogram-based methods. Whether aerial or terrestrial, disparity histogram-based methods suffer from common problems: computational complexity, sensitivity to illumination changes, and the need for accurate camera calibration, especially when implemented on small robots. In addition, until recently, both monocular and stereo methods relied on conventional image processing techniques and, thus, did not meet the requirements of real-time applications. Therefore, deep learning networks have been the centre of focus in recent years to develop fast and reliable obstacle detection solutions. However, we observed that despite significant progress, deep learning techniques also face difficulties in complex and unknown environments where objects of varying types and shapes are present. The review suggests that detecting narrow and small, moving obstacles and fast obstacle detection are the most challenging problem to focus on in future studies

A New Multi-Criteria Tie Point Filtering Approach to Increase the Accuracy of UAV Photogrammetry Models

Extracting accurate tie points plays an essential role in the accuracy of image orientation in Unmanned Aerial Vehicle (UAV) photogrammetry. In this study, a Multi-Criteria Decision Making (MCDM) automatic filtering method is presented. Based on the quality features of a photogrammetric model, the proposed method works at the level of sparse point cloud to remove low-quality tie points for refining the orientation results. In the proposed algorithm, different factors that affect the quality of tie points are identified. The quality measures are then aggregated by applying MCDM methods and a competency score for each 3D tie point. These scores are employed in an automatic filtering approach that selects a subset of high-quality points which are then used to repeat the bundle adjustment. To evaluate the proposed algorithm, various internal and external studies were conducted on different datasets. The findings suggest that our method is both effective and reliable. In addition, in comparison to the existing filtering techniques, the proposed strategy increases the accuracy of bundle adjustment and dense point cloud generation by about 40% and 70%, respectively

Development of an automatic map drawing system for ancient bas-reliefs

In this paper, a new system for automatic drawing of bas-reliefs is presented. The system consists of a fringe projection scanner accompanied by a photometric stereo component. At first, the photometric stereo is used to accurately extract normals of the relief surface. Using a line tracing algorithm the normals are converted into 2D lines. Then, using the fringe projection scanner, the 2D lines are transformed to the 3D space to form a metric map required for cultural heritage applications. To evaluate the system, several tests were carried out using artificial and real bas-reliefs. The results showed that the system is able to draw 98.8% of the object lines correctly

A Two-Step Descriptor-Based Keypoint Filtering Algorithm for Robust Image Matching

Finding robust and correct keypoints in images remains a challenge, especially when repetitive patterns are present. In this article, we propose a universal two-step filtering method to solve the mismatch problem in repetitive patterns. Having applied a mean-shift clustering algorithm to remove obvious mismatches, the proposed confusion reduction (CR) method uses a novel confusion index (CI) in a gridding schema to identify and filter out the remaining confusing keypoints. In both steps, the descriptors’ statistical properties are evaluated using kernel density estimation. Various synthetic and real stereo pairs, along with multiview image blocks, were used to assess the performance of the presented algorithm. The results were also compared with those obtained by several state-of-the-art mismatch removal methods. The experiments showed that, on average, the proposed strategy improves the accuracy of matching by 10% and the accuracy of photogrammetric blocks by 20%–30%