RGB2LIDAR: Towards Solving Large-Scale Cross-Modal Visual Localization

We study an important, yet largely unexplored problem of large-scale cross-modal visual localization by matching ground RGB images to a geo-referenced aerial LIDAR 3D point cloud (rendered as depth images). Prior works were demonstrated on small datasets and did not lend themselves to scaling up for large-scale applications. To enable large-scale evaluation, we introduce a new dataset containing over 550K pairs (covering 143 km^2 area) of RGB and aerial LIDAR depth images. We propose a novel joint embedding based method that effectively combines the appearance and semantic cues from both modalities to handle drastic cross-modal variations. Experiments on the proposed dataset show that our model achieves a strong result of a median rank of 5 in matching across a large test set of 50K location pairs collected from a 14km^2 area. This represents a significant advancement over prior works in performance and scale. We conclude with qualitative results to highlight the challenging nature of this task and the benefits of the proposed model. Our work provides a foundation for further research in cross-modal visual localization.Comment: ACM Multimedia 202

Map Generation from Large Scale Incomplete and Inaccurate Data Labels

Accurately and globally mapping human infrastructure is an important and challenging task with applications in routing, regulation compliance monitoring, and natural disaster response management etc.. In this paper we present progress in developing an algorithmic pipeline and distributed compute system that automates the process of map creation using high resolution aerial images. Unlike previous studies, most of which use datasets that are available only in a few cities across the world, we utilizes publicly available imagery and map data, both of which cover the contiguous United States (CONUS). We approach the technical challenge of inaccurate and incomplete training data adopting state-of-the-art convolutional neural network architectures such as the U-Net and the CycleGAN to incrementally generate maps with increasingly more accurate and more complete labels of man-made infrastructure such as roads and houses. Since scaling the mapping task to CONUS calls for parallelization, we then adopted an asynchronous distributed stochastic parallel gradient descent training scheme to distribute the computational workload onto a cluster of GPUs with nearly linear speed-up.Comment: This paper is accepted by KDD 202

Visual and geographical data fusion to classify landmarks in geo-tagged images

High level semantic image recognition and classification is a challenging task and currently is a very active research domain. Computers struggle with the high level task of identifying objects and scenes within digital images accurately in unconstrained environments. In this paper, we present experiments that aim to overcome the limitations of computer vision algorithms by combining them with novel contextual based features to describe geo-tagged imagery. We adopt a machine learning based algorithm with the aim of classifying classes of geographical landmarks within digital images. We use community contributed image sets downloaded from Flickr and provide a thorough investigation, the results of which are presented in an evaluation section

Citizens as Sensors for Crisis Event: Sensor Web Enablement for Volunteered Geographic Information

A set of developments within the field of geosensors is to engage citizens to act as sensors, thus providing so-called Volunteered Geographic Information (VGI). There is a long tradition of non specialists contributing to the collection of geo-referenced information. Furthermore thanks to recent convergence of greater access to broadband connections, the availability of Global Positioning Systems at affordable prices, and more participative forms of interaction on the Web (Web 2.0) vast numbers of individuals are able to create and share geographic information. The potential of up to 6 billion human sensors to monitor the state of the environment, validate global models with local knowledge, contribute to crisis situations awareness and provide information that only humans can capture (e.g. emotions and perceptions like fear of crime) is vast and has yet to be fully exploited. However, integrating VGI into Spatial Data Infrastructures (SDI) is a major challenge, as it is often regarded as insufficiently structured, documented or validated according to scientific standards. Early instances of SDIs used to have limited ability to manage and process geosensor-based data (beyond remotely sensed imagery snapshots), which tend to arrive in continuous streams of real-time information. The current works on standards for Sensor Web Enablement (SWE) aim to fill this gap. This paper shows how such SWE standards can be applied to VGI, thus converting it in a timely, cost-effective and valuable source of information for SDIs. By doing so, we extend previous works describing a workflow for VGI integration into SDI and further advance an initial set of VGI Sensing and event detection techniques. In particular, an example of how such VGI Sensing techniques can support crisis information system is provided.JRC.DDG.H.6-Spatial data infrastructure

Unveiling E-bike potential for commuting trips from GPS traces

Common goals of sustainable mobility approaches are to reduce the need for travel, to facilitate modal shifts, to decrease trip distances and to improve energy efficiency in the transportation systems. Among these issues, modal shift plays an important role for the adoption of vehicles with fewer or zero emissions. Nowadays, the electric bike (e-bike) is becoming a valid alternative to cars in urban areas. However, to promote modal shift, a better understanding of the mobility behaviour of e-bike users is required. In this paper, we investigate the mobility habits of e-bikers using GPS data collected in Belgium from 2014 to 2015. By analysing more than 10,000 trips, we provide insights about e-bike trip features such as: distance, duration and speed. In addition, we offer a deep look into which routes are preferred by bike owners in terms of their physical characteristics and how weather influences e-bike usage. Results show that trips with higher travel distances are performed during working days and are correlated with higher average speeds. Usage patterns extracted from our data set also indicate that e-bikes are preferred for commuting (home-work) and business (work related) trips rather than for recreational trips

Spatio-temporal research data infrastructure in the context of autonomous driving

In this paper, we present an implementation of a research data management system that features structured data storage for spatio-temporal experimental data (environmental perception and navigation in the framework of autonomous driving), including metadata management and interfaces for visualization and parallel processing. The demands of the research environment, the design of the system, the organization of the data storage, and computational hardware as well as structures and processes related to data collection, preparation, annotation, and storage are described in detail. We provide examples for the handling of datasets, explaining the required data preparation steps for data storage as well as benefits when using the data in the context of scientific tasks. © 2020 by the authors

Citizen-based sensing of crisis events: sensor web enablement for volunteered geographic information

Thanks to recent convergence of greater access to broadband connections, the availability of Global Positioning Systems in small packages at affordable prices and more participative forms of interaction on the Web (Web 2.0), vast numbers of individuals became able to create and share Volunteered Geographic Information (VGI). The potential of up to six billion persons to monitor the state of the environment, validate global models with local knowledge, contribute to crisis situations awareness, and provide information that only humans can capture is vast and has yet to be fully exploited. Integrating VGI into Spatial Data Infrastructures (SDI) is a major challenge, as it is often regarded as insufficiently structured, documented, or validated according to scientific standards. Early instances of SDIs used to have limited ability to manage and process geosensor-based data (beyond remotely sensed imagery), which tend to arrive in continuous streams of real-time information. The current works on standards for Sensor Web Enablement fill this gap. This paper shows how such standards can be applied to VGI, thus converting it in a timely, cost-effective and valuable source of information for SDIs. By doing so, we extend previous efforts describing a workflow for VGI integration into SDI and further advance an initial set of VGI Sensing and event detection techniques. Examples of how such VGI Sensing techniques can support crisis information system are provided. The presented approach serves central building blocks for a Digital Earth’s nervous system, which is required to develop the next generation of (geospatial) information infrastructures

A WEB-BASED INTERACTIVE TOOL FOR MULTI-RESOLUTION 3D MODELS OF A MAYA ARCHAEOLOGICAL SITE

Continuous technological advances in surveying, computing and digital-content delivery are strongly contributing to a change in the way Cultural Heritage is “perceived”: new tools and methodologies for documentation, reconstruction and research are being created to assist not only scholars, but also to reach more potential users (e.g. students and tourists) willing to access more detailed information about art history and archaeology. 3D computer-simulated models, sometimes set in virtual landscapes, offer for example the chance to explore possible hypothetical reconstructions, while on-line GIS resources can help interactive analyses of relationships and change over space and time. While for some research purposes a traditional 2D approach may suffice, this is not the case for more complex analyses concerning spatial and temporal features of architecture, like for example the relationship of architecture and landscape, visibility studies etc. The project aims therefore at creating a tool, called “QueryArch3D” tool, which enables the web-based visualisation and queries of an interactive, multi-resolution 3D model in the framework of Cultural Heritage. More specifically, a complete Maya archaeological site, located in Copan (Honduras), has been chosen as case study to test and demonstrate the platform’s capabilities. Much of the site has been surveyed and modelled at different levels of detail (LoD) and the geometric model has been semantically segmented and integrated with attribute data gathered from several external data sources. The paper describes the characteristics of the research work, along with its implementation issues and the initial results of the developed prototype

Beyond Geo-localization: Fine-grained Orientation of Street-view Images by Cross-view Matching with Satellite Imagery

Street-view imagery provides us with novel experiences to explore different places remotely. Carefully calibrated street-view images (e.g. Google Street View) can be used for different downstream tasks, e.g. navigation, map features extraction. As personal high-quality cameras have become much more affordable and portable, an enormous amount of crowdsourced street-view images are uploaded to the internet, but commonly with missing or noisy sensor information. To prepare this hidden treasure for "ready-to-use" status, determining missing location information and camera orientation angles are two equally important tasks. Recent methods have achieved high performance on geo-localization of street-view images by cross-view matching with a pool of geo-referenced satellite imagery. However, most of the existing works focus more on geo-localization than estimating the image orientation. In this work, we re-state the importance of finding fine-grained orientation for street-view images, formally define the problem and provide a set of evaluation metrics to assess the quality of the orientation estimation. We propose two methods to improve the granularity of the orientation estimation, achieving 82.4% and 72.3% accuracy for images with estimated angle errors below 2 degrees for CVUSA and CVACT datasets, corresponding to 34.9% and 28.2% absolute improvement compared to previous works. Integrating fine-grained orientation estimation in training also improves the performance on geo-localization, giving top 1 recall 95.5%/85.5% and 86.8%/80.4% for orientation known/unknown tests on the two datasets.Comment: This paper has been accepted by ACM Multimedia 2022. The version contains additional supplementary material