Learning multi-modal features for dense matching-based confidence estimation

In recent years, the ability to assess the uncertainty of depth estimates in the context of dense stereo matching has received increased attention due to its potential to detect erroneous estimates. Especially, the introduction of deep learning approaches greatly improved general performance, with feature extraction from multiple modalities proving to be highly advantageous due to the unique and different characteristics of each modality. However, most work in the literature focuses on using only mono- or bi- or rarely tri-modal input, not considering the potential effectiveness of modalities, going beyond tri-modality. To further advance the idea of combining different types of features for confidence estimation, in this work, a CNN-based approach is proposed, exploiting uncertainty cues from up to four modalities. For this purpose, a state-of-the-art local-global approach is used as baseline and extended accordingly. Additionally, a novel disparity-based modality named warped difference is presented to support uncertainty estimation at common failure cases of dense stereo matching. The general validity and improved performance of the proposed approach is demonstrated and compared against the bi-modal baseline in an evaluation on three datasets using two common dense stereo matching techniques

Incremental map refinement of building information using lidar point clouds

For autonomous systems, an accurate and precise map of the environment is of importance. Mapping from LiDAR point clouds is one of the promising ways to generate 3D environment models. However, there are many problems caused by inaccurate data, missing areas, low density of points and sensor noise. Also, it is often not possible or accurate enough to generate a map from only one measurement campaign. In this paper, we propose a method to incrementally refine the map by several measurements from different campaigns and represent the map in a hierarchical way with a measure indicating uncertainty and the level of detail for objects. The idea is thus to store all captured information with a tentative semantics and uncertainty - even when it is not yet complete. Hence, occulated areas are presented as well, which can be possibly improved by the supplemental observation from the next measurement campaign. The proposed 3D environment model framework and the incremental update method are evaluated using LiDAR scans obtained from Riegl Mobile Mapping System

Comap: A synthetic dataset for collective multi-agent perception of autonomous driving

Collective perception of connected vehicles can sufficiently increase the safety and reliability of autonomous driving by sharing perception information. However, collecting real experimental data for such scenarios is extremely expensive. Therefore, we built a computational efficient co-simulation synthetic data generator through CARLA and SUMO simulators. The simulated data contain image and point cloud data as well as ground truth for object detection and semantic segmentation tasks. To verify the superior performance gain of collective perception over single-vehicle perception, we conducted experiments of vehicle detection, which is one of the most important perception tasks for autonomous driving, on this data set. A 3D object detector and a Bird's Eye View (BEV) detector are trained and then test with different configurations of the number of cooperative vehicles and vehicle communication ranges. The experiment results showed that collective perception can not only dramatically increase the overall mean detection accuracy but also the localization accuracy of detected bounding boxes. Besides, a vehicle detection comparison experiment showed that the detection performance drop caused by sensor observation noise can be canceled out by redundant information collected by multiple vehicles

State-wide calculation of terrain-visualisations and automatic map generation for archaeological objects

Airborne laser scanning (ALS) became very popular in the last two decades for archaeological prospection. With the state-wide availability of ALS-data in Lower Saxony, Germany, about 48,000 km2;, we needed flexible and scalable approaches to process the data. First, we produced a state-wide digital terrain model (DTM) and some visualisations of it to use it in standard GIS software. Some of these visualisations are available as web maps and used for prospection also by volunteers. In a second approach, we automatically generate maps for all known archaeological objects. This is mainly used for the documentation of the 130,000 known objects in Lower Saxony, but also for object-by-object revision of the database. These Maps will also be presented in the web portal "Denkmalatlas Niedersachsen", an open data imitative of the state Lower Saxony.In the first part of this paper, we show how the state-wide DTM and its visualisations can be calculated using tiles. In the second part, we describe the automatic map generation process. All implementations were done with ArcGIS and its scripting interface ArcPy

CNN-based multi-scale hierarchical land use classification for the verification of geospatial databases

Land use is an important piece of information with many applications. Commonly, land use is stored in geospatial databases in the form of polygons with corresponding land use labels and attributes according to an object catalogue. The object catalogues often have a hierarchical structure, with the level of detail of the semantic information depending on the hierarchy level. In this paper, we extend our prior work for the CNN (Convolutional Neural Network)-based prediction of land use for database objects at multiple semantic levels corresponding to different levels of a hierarchical class catalogue. The main goal is the improvement of the classification accuracy for small database objects, which we observed to be one of the largest problems of the existing method. In order to classify large objects using a CNN of a fixed input size, they are split into tiles that are classified independently before fusing the results to a joint prediction for the object. In this procedure, small objects will only be represented by a single patch, which might even be dominated by the background. To overcome this problem, a multi-scale approach for the classification of small objects is proposed in this paper. Using this approach, such objects are represented by multiple patches at different scales that are presented to the CNN for classification, and the classification results are combined. The new strategy is applied in combination with the earlier tiling-based approach. This method based on an ensemble of the two approaches is tested in two sites located in Germany and improves the classification performance up to +1.8% in overall accuracy and +3.2% in terms of mean F1 score

Determination of parking space and its concurrent usage over time using semantically segmented mobile mapping data

Public space is a scarce good in cities. There are many concurrent usages, which makes an adequate allocation of space both difficult and highly attractive. A lot of space is allocated by parking cars - even if the parking spaces are not occupied by cars all the time. In this work, we analyze space demand and usage by parking cars, in order to evaluate, when this space could be used for other purposes. The analysis is based on 3D point clouds acquired at several times during a day. We propose a processing pipeline to extract car bounding boxes from a given 3D point cloud. For the car extraction we utilize a label transfer technique for transfers from semantically segmented 2D RGB images to 3D point cloud data. This semantically segmented 3D data allows us to identify car instances. Subsequently, we aggregate and analyze information about parking cars. We present an exemplary analysis of the urban area where we extracted 15.000 cars at five different points in time. Based on this aggregated we present analytical results for time dependent parking behavior, parking space availability and utilization

Novel glassy behavior in a ferromagnetic p-spin model

Recent work has suggested the existence of glassy behavior in a ferromagnetic model with a four-spin interaction. Motivated by these findings, we have studied the dynamics of this model using Monte Carlo simulations with particular attention being paid to two-time quantities. We find that the system shares many features in common with glass forming liquids. In particular, the model exhibits: (i) a very long-lived metastable state, (ii) autocorrelation functions that show stretched exponential relaxation, (iii) a non-equilibrium timescale that appears to diverge at a well defined temperature, and (iv) low temperature aging behaviour characteristic of glasses.Comment: 6 pages, 5 figure

Addressing Class Imbalance in Multi-Class Image Classification by Means of Auxiliary Feature Space Restrictions

Learning from imbalanced class distributions generally leads to a classifier that is not able to distinguish classes with few training examples from the other classes. In the context of cultural heritage, addressing this problem becomes important when existing digital online collections consisting of images depicting artifacts and assigned semantic annotations shall be completed automatically; images with known annotations can be used to train a classifier that predicts missing information, where training data is often highly imbalanced. In the present paper, combining a classification loss with an auxiliary clustering loss is proposed to improve the classification performance particularly for underrepresented classes, where additionally different sampling strategies are applied. The proposed auxiliary loss aims to cluster feature vectors with respect to the semantic annotations as well as to visual properties of the images to be classified and thus, is supposed to help the classifier in distinguishing individual classes. We conduct an ablation study on a dataset consisting of images depicting silk fabrics coming along with annotations for different silk-related classification tasks. Experimental results show improvements of up to 10.5% in average F1-score and up to 20.8% in the F1-score averaged over the underrepresented classes in some classification tasks

Uncertainty Representation and Quantification of 3d Building Models

The quality of environmental perception is of great interest for localization tasks in autonomous systems. Maps, generated from the sensed information, are often used as additional spatial references in these applications. The quantification of the map uncertainties gives an insight into how reliable and complete the map is, avoiding the potential systematic deviation in pose estimation. Mapping 3D buildings in urban areas using Light detection and ranging (LiDAR) point clouds is a challenging task as it is often subject to uncertain error sources in the real world such as sensor noise and occlusions, which should be well represented in the 3D models for the downstream localization tasks. In this paper, we propose a method to model 3D building façades in complex urban scenes with uncertainty quantification, where the uncertainties of windows and façades are indicated in a probabilistic fashion. The potential locations of the missing objects (here: windows) are inferred by the available data and layout patterns with the Monte Carlo (MC) sampling approach. The proposed 3D building model and uncertainty measures are evaluated using the real-world LiDAR point clouds collected by Riegl Mobile Mapping System. The experimental results show that our uncertainty representation conveys the quality information of the estimated locations and shapes for the modelled map objects

Building Change Detection in Airborne Laser Scanning and Dense Image Matching Point Clouds Using a Residual Neural Network

National Mapping Agencies (NMAs) acquire nation-wide point cloud data from Airborne Laser Scanning (ALS) sensors as well as using Dense Image Matching (DIM) on aerial images. As these datasets are often captured years apart, they contain implicit information about changes in the real world. While detecting changes within point clouds is not a new topic per se, detecting changes in point clouds from different sensors, which consequently have different point densities, point distributions and characteristics, is still an on-going problem. As such, we approach this task using a residual neural network, which detects building changes using height and class information on a raster level. In the experiments, we show that this approach is capable of detecting building changes automatically and reliably independent of the given point clouds and for various building sizes achieving mean F1-Scores of 80.5% and 79.8% for ALS-ALS and ALS-DIM point clouds on an object-level and F1-Scores of 91.1% and 86.3% on a raster-level, respectively