Data trustworthiness and user reputation as indicators of VGI quality

ABSTRACTVolunteered geographic information (VGI) has entered a phase where there are both a substantial amount of crowdsourced information available and a big interest in using it by organizations. But the issue of deciding the quality of VGI without resorting to a comparison with authoritative data remains an open challenge. This article first formulates the problem of quality assessment of VGI data. Then presents a model to measure trustworthiness of information and reputation of contributors by analyzing geometric, qualitative, and semantic aspects of edits over time. An implementation of the model is running on a small data-set for a preliminary empirical validation. The results indicate that the computed trustworthiness provides a valid approximation of VGI quality

Evaluation of a volunteered geographical information trust measure in the case of OpenStreetMap

Dissertation submitted in partial fulfillment of the requirements for the Degree of Master of Science in Geospatial Technologies.The presence of Volunteered Geographical Information is attracting research because its high availability and diversity make it an interesting source of information. For many organisations it is important that quality of geographical information is of a certain level. Recent developments in studies related to VGI direct towards the estimation of its quality through the notion of trust as a proxy. For this thesis is investigated which factors have an important influence on trust and a simple approach was used to come up with an indication of trust levels for geographical features. The indicators were selected based on a literature review and on a dataset extracted from the open mapping project OpenStreetMap. Numbers of users, versions and confirmations were counted or calculated and involved as positive indicators, while numbers of various corrections were treated as indicators having a negative influence on the development of trust in information. Analysis of the dataset and thinking about how to incorporate what in the trust measure showed for example how ideas about time decay could be different. Importance of tags was determined based on a method adopted from documentation studies and applied on the dataset. It allowed for generating lists of tags to be described when publishing information about particular features. This was of importance for assessing information completeness in measuring the quality of the data. The results of the trust measure have been compared to those if the quality measure and an evaluation of this comparison shows significant signs of support for the hypothesis that VGI data quality can be estimated based on a trust model that incorporates data provenance. On the other hand there is also a significant number of features of which both measures show opposite indications of quality. Various single assumptions, simplifications and the relatively small size of the dataset restricted the possibilities for obtaining more accurate results. Confirmation or denial of the ideas that resulted from this research can be made by enlarging the dataset and experimenting with different methods. Automating all the data processing would be necessary

VGI Edit History Reveals Data Trustworthiness and User Reputation

Ponencias, comunicaciones y pósters presentados en el 17th AGILE Conference on Geographic Information Science "Connecting a Digital Europe through Location and Place", celebrado en la Universitat Jaume I del 3 al 6 de junio de 2014.Volunteered Geographic Information (VGI) is an approach to crowdsource information about geospatial features around us. People around the world are engaged with typing in their observations about the world (like locations of shops, cafeterias), or to semi-automatically gather them with mobile devices (like hiking paths or roads). In this process people might make mistakes, for instance assign misleading tags to features or provide over simplistic boundaries for features. In this paper we study what kinds of things might contribute to assess trustworthiness of data, and reputation of contributors for VGI. We present a model for analysing the different factors, and a method for automatically creating the trust and reputation scores

Reputation evaluation of georeferenced data for crowd-sensed applications

Volunteered Geographic Information (VGI) is a process where individuals, supported by enabling technologies, behave like physicalsensorstoharvestgeoreferencedcontentintheirsurroundings. Thevalueofthis, typicallyheterogeneous, contenthasbeen recognized by both researchers and organizations. However, in order to be fruitfully used in various VGI-based types of application reliability and quality of particular VGI content (i.e., Points of Interest) have to be assessed. This evaluation can be based on reputation scores that summarize users’ experiences with the specific content. Following this direction, our contribution provides, primarily, a new comprehensive model and a multi-layer architecture for reputation evaluation aimed to assess quality of VGI content. Secondly, we demonstrate the relevance of adopting such a framework through an applicative scenario for recommending touristic itineraries

Quality Assessment of the Canadian OpenStreetMap Road Networks

Volunteered geographic information (VGI) has been applied in many fields such as participatory planning, humanitarian relief and crisis management because of its cost-effectiveness. However, coverage and accuracy of VGI cannot be guaranteed. OpenStreetMap (OSM) is a popular VGI platform that allows users to create or edit maps using GPS-enabled devices or aerial imageries. The issue of geospatial data quality in OSM has become a trending research topic because of the large size of the dataset and the multiple channels of data access. The objective of this study is to examine the overall reliability of the Canadian OSM data. A systematic review is first presented to provide details on the quality evaluation process of OSM. A case study of London, Ontario is followed as an experimental analysis of completeness, positional accuracy and attribute accuracy of the OSM street networks. Next, a national study of the Canadian OSM data assesses the overall semantic accuracy and lineage in addition to the quality measures mentioned above. Results of the quality evaluation are compared with associated OSM provenance metadata to examine potential correlations. The Canadian OSM road networks were found to have comparable accuracy with the tested commercial database (DMTI). Although statistical analysis suggests that there are no significant relations between OSM accuracy and its editing history, the study presents the complex processes behind OSM contributions possibly influenced by data import and remote mapping. The findings of this thesis can potentially guide cartographic product selection for interested parties and offer a better understanding of future quality improvement in OSM

Visualisation of trust and quality information for geospatial dataset selection and use:Drawing trust presentation comparisons with B2C e-Commerce

The evaluation of geospatial data quality and trustworthiness presents a major challenge to geospatial data users when making a dataset selection decision. Part of the problem arises from the inconsistent and patchy nature of data quality information, which makes intercomparison very difficult. Over recent years, the production and availability of geospatial data has significantly increased, facilitated by the recent explosion of Web-based catalogues, portals, standards and services, and by initiatives such as INSPIRE and GEOSS. Despite this significant growth in availability of geospatial data and the fact that geospatial datasets can, in many respects, be considered commercial products that are available for purchase online, consumer trust has to date received relatively little attention in the GIS domain. In this paper, we discuss how concepts of trust, trust models, and trust indicators (largely derived from B2C e-Commerce) apply to the GIS domain and to geospatial data selection and use. Our research aim is to support data users in more efficient and effective geospatial dataset selection on the basis of quality, trustworthiness and fitness for purpose. To achieve this, we propose a GEO label – a decision support mechanism that visually summarises availability of key geospatial data informational aspects. We also present a Web service that was developed to support generation of dynamic GEO label representations for datasets by combining producer metadata (from standard catalogues or other published locations) with structured user feedback

Modeling aggregated expertise of user contributions to assess the credibility of OpenStreetMap features + Erratum

The emergence of volunteered geographic information (VGI) during the past decade has fueled a wide range of research and applications. The assessment of VGI quality and fitness-of-use is still a challenge because of the non-standardized and crowdsourced data collection process, as well as the unknown skill and motivation of the contributors. However, the frequent approach of assessing VGI quality against external data sources using ISO quality standard measures is problematic because of a frequent lack of available external (reference) data, and because for certain types of features, VGI might be more up-to-date than the reference data. Therefore, a VGI-intrinsic measure of quality is highly desirable. This study proposes such an intrinsic measure of quality by developing the concept of aggregated expertise based on the characteristics of a feature's contributors. The article further operationalizes this concept and examines its feasibility through a case study using OpenStreetMap (OSM). The comparison of model OSM feature quality with information from a field survey demonstrates the successful implementation of this novel approach

Enhancing Data Classification Quality of Volunteered Geographic Information

Geographic data is one of the fundamental components of any Geographic Information System (GIS). Nowadays, the utility of GIS becomes part of everyday life activities, such as searching for a destination, planning a trip, looking for weather information, etc. Without a reliable data source, systems will not provide guaranteed services. In the past, geographic data was collected and processed exclusively by experts and professionals. However, the ubiquity of advanced technology results in the evolution of Volunteered Geographic Information (VGI), when the geographic data is collected and produced by the general public. These changes influence the availability of geographic data, when common people can work together to collect geographic data and produce maps. This particular trend is known as collaborative mapping. In collaborative mapping, the general public shares an online platform to collect, manipulate, and update information about geographic features. OpenStreetMap (OSM) is a prominent example of a collaborative mapping project, which aims to produce a free world map editable and accessible by anyone. During the last decade, VGI has expanded based on the power of crowdsourcing. The involvement of the public in data collection raises great concern about the resulting data quality. There exist various perspectives of geographic data quality this dissertation focuses particularly on the quality of data classification (i.e., thematic accuracy). In professional data collection, data is classified based on quantitative and/or qualitative ob- servations. According to a pre-defined classification model, which is usually constructed by experts, data is assigned to appropriate classes. In contrast, in most collaborative mapping projects data classification is mainly based on individualsa cognition. Through online platforms, contributors collect information about geographic features and trans- form their perceptions into classified entities. In VGI projects, the contributors mostly have limited experience in geography and cartography. Therefore, the acquired data may have a questionable classification quality. This dissertation investigates the challenges of data classification in VGI-based mapping projects (i.e., collaborative mapping projects). In particular, it lists the challenges relevant to the evolution of VGI as well as to the characteristics of geographic data. Furthermore, this work proposes a guiding approach to enhance the data classification quality in such projects. The proposed approach is based on the following premises (i) the availability of large amounts of data, which fosters applying machine learning techniques to extract useful knowledge, (ii) utilization of the extracted knowledge to guide contributors to appropriate data classification, (iii) the humanitarian spirit of contributors to provide precise data, when they are supported by a guidance system, and (iv) the power of crowdsourcing in data collection as well as in ensuring the data quality. This cumulative dissertation consists of five peer-reviewed publications in international conference proceedings and international journals. The publications divide the disser- tation into three parts the first part presents a comprehensive literature review about the relevant previous work of VGI quality assurance procedures (Chapter 2), the second part studies the foundations of the approach (Chapters 3-4), and the third part discusses the proposed approach and provides a validation example for implementing the approach (Chapters 5-6). Furthermore, Chapter 1 presents an overview about the research ques- tions and the adapted research methodology, while Chapter 7 concludes the findings and summarizes the contributions. The proposed approach is validated through empirical studies and an implemented web application. The findings reveal the feasibility of the proposed approach. The output shows that applying the proposed approach results in enhanced data classification quality. Furthermore, the research highlights the demands for intuitive data collection and data interpretation approaches adequate to VGI-based mapping projects. An interaction data collection approach is required to guide the contributors toward enhanced data quality, while an intuitive data interpretation approach is needed to derive more precise information from rich VGI resources

A framework of quality assessment methods for crowdsourced geographic information : a systematic literature review

Collaboration is the foundation to strengthen disaster preparedness and for effective emergency response actions at all levels. Some studies have highlighted that remote volunteers, i.e., volunteers supported by Web 2.0 technologies, possess the potential to strengthen humanitarian relief organizations by offering information regarding disaster-affected people and infrastructure. Although studies have explored various aspects of this topic, none of those provided an overview of the state-of-the-art of researches on the collaboration among humanitarian organizations and communities of remote volunteers. With the aim of overcoming this gap, a systematic literature review was conducted on the existing research works. Therefore, the main contribution of this work lies in examining the state of research in this field and in identifying potential research gaps. The results show that most of the research works addresses the general domain of disaster management, whereas only few of them address the domain of humanitarian logistics. Collaboration among Humanitarian Relief Organizations and Volunteer Technical Communities: Identifying Research Opportunities and Challenges through a Systematic Literature Review (PDF Download Available). Available from: https://www.researchgate.net/publication/315790817_Collaboration_among_Humanitarian_Relief_Organizations_and_Volunteer_Technical_Communities_Identifying_Research_Opportunities_and_Challenges_through_a_Systematic_Literature_Review [accessed May 26, 2017]

Evaluating Reputation in VGI-enabled Applications

ABSTRACT Volunteered Geographic Information (VGI) is an approach to crowdsource information about geospatial objects around us, as implemented in Open Street Map, Google Map Maker and WikiMapia projects. The value of this content has been recognized by both researchers and organizations for acquiring free, timely and detailed spatial data versus standard spatial data warehouses where objects are created by professionals with variable updating time. However, evaluating its quality and handling its heterogeneity remain challenging concerns. For instance, VGI data sources have been compared to authoritative geospatial ones on specific regions/areas in order to determine an average overall quality level. In user-oriented VGI-based applications, it can be more relevant to assess the quality of particular contents, like specific Points of Interest. In this case, evaluation can be performed indirectly by reputation scores associated with the specific content. This paper focuses on this last aspect. Our contribution primarily provides a comprehensive model and architecture for reputation evaluation aimed to assess quality of VGI content. On the other hand, we also focus on applications by discussing two motivating scenarios for reputation-enhanced VGI data in the context of geospatial decision support systems and in recommending tourist itineraries