Understanding MapSwipe: Analysing Data Quality of Crowdsourced Classifications on Human Settlements

Geodata is missing to populate maps for usage of local communities. Efforts for filling gaps (automatically) by deriving data on human settlements using aerial or satellite imagery is of current concern (Esch et al., 2013; Pesaresi et al., 2013; Voigt et al., 2007). Among semi-automated methods and pre-processed data products, crowdsourcing is another tool which can help to collect information on human settlements and complement existing data, yet it’s accuracy is debated (Goodchild and Li, 2012; Haklay, 2010; Senaratne et al., 2016). Here the quality of data produced by volunteers using the MapSwipe app was investigated. Three different intrinsic parameters of crowdsourced data and their impact on data quality were examined: agreement, user characteristics and spatial characteristics. Additionally, a novel mechanism based on machine learning techniques was presented to aggregate data provided from multiple users. The results have shown that a random forest based aggregation of crowdsourced classifications from MapSwipe can produce high quality data in comparison to state-of-the-art products derived from satellite imagery. High agreement serves as an indicator for correct classifications. Intrinsic user characteristics can be utilized to identify consistently incorrect classifications. Classifications that are spatial outliers show a higher error rate. The findings pronounce that the integration of machine learning techniques into existing crowdsourcing workflows can become a key point for the future development of crowdsourcing applications