Implementation and assessment of two density-based outlier detection methods over large spatial point clouds

Several technologies provide datasets consisting of a large number of spatial points, commonly referred to as point-clouds. These point datasets provide spatial information regarding the phenomenon that is to be investigated, adding value through knowledge of forms and spatial relationships. Accurate methods for automatic outlier detection is a key step. In this note we use a completely open-source workflow to assess two outlier detection methods, statistical outlier removal (SOR) filter and local outlier factor (LOF) filter. The latter was implemented ex-novo for this work using the Point Cloud Library (PCL) environment. Source code is available in a GitHub repository for inclusion in PCL builds. Two very different spatial point datasets are used for accuracy assessment. One is obtained from dense image matching of a photogrammetric survey (SfM) and the other from floating car data (FCD) coming from a smart-city mobility framework providing a position every second of two public transportation bus tracks. Outliers were simulated in the SfM dataset, and manually detected and selected in the FCD dataset. Simulation in SfM was carried out in order to create a controlled set with two classes of outliers: clustered points (up to 30 points per cluster) and isolated points, in both cases at random distances from the other points. Optimal number of nearest neighbours (KNN) and optimal thresholds of SOR and LOF values were defined using area under the curve (AUC) of the receiver operating characteristic (ROC) curve. Absolute differences from median values of LOF and SOR (defined as LOF2 and SOR2) were also tested as metrics for detecting outliers, and optimal thresholds defined through AUC of ROC curves. Results show a strong dependency on the point distribution in the dataset and in the local density fluctuations. In SfM dataset the LOF2 and SOR2 methods performed best, with an optimal KNN value of 60; LOF2 approach gave a slightly better result if considering clustered outliers (true positive rate: LOF2\u2009=\u200959.7% SOR2\u2009=\u200953%). For FCD, SOR with low KNN values performed better for one of the two bus tracks, and LOF with high KNN values for the other; these differences are due to very different local point density. We conclude that choice of outlier detection algorithm very much depends on characteristic of the dataset\u2019s point distribution, no one-solution-fits-all. Conclusions provide some information of what characteristics of the datasets can help to choose the optimal method and KNN values

Open-source geospatial tools and technologies for urban and environmental studies

AbstractOpen geospatial data and tools are an increasingly important paradigm offering the opportunity to promote the democratization of geographical information, the transparency of governments and institutions, as well as social, economic and environmental opportunities. During the past decade, developments in the area of open geospatial data and open-source geospatial software have greatly improved. Many parts of the research community believe that combining free and open software, open data, as well as open standards, leads to the creation of a sustainable ecosystem to accelerate new discoveries to help solve global cross-disciplinary societal challenges, from climate change mitigation to sustainable cities. The consistent prevalence of open source GIS studies motivated this thematic collection. The contributions are divided into two main categories. In the first category, seven concrete studies on open-source tools and technologies for urban and environmental studies are briefly presented. Each one has been implemented for and applied to a certain use case, and at the same time it may be applied to other use cases due to the reproducibility nature of open source software. The second category presents and discusses the usability of open source geospatial solutions for laser scanning technology and its applications