Engineering Crowdsourced Stream Processing Systems

A crowdsourced stream processing system (CSP) is a system that incorporates crowdsourced tasks in the processing of a data stream. This can be seen as enabling crowdsourcing work to be applied on a sample of large-scale data at high speed, or equivalently, enabling stream processing to employ human intelligence. It also leads to a substantial expansion of the capabilities of data processing systems. Engineering a CSP system requires the combination of human and machine computation elements. From a general systems theory perspective, this means taking into account inherited as well as emerging properties from both these elements. In this paper, we position CSP systems within a broader taxonomy, outline a series of design principles and evaluation metrics, present an extensible framework for their design, and describe several design patterns. We showcase the capabilities of CSP systems by performing a case study that applies our proposed framework to the design and analysis of a real system (AIDR) that classifies social media messages during time-critical crisis events. Results show that compared to a pure stream processing system, AIDR can achieve a higher data classification accuracy, while compared to a pure crowdsourcing solution, the system makes better use of human workers by requiring much less manual work effort

empathi: An ontology for Emergency Managing and Planning about Hazard Crisis

In the domain of emergency management during hazard crises, having sufficient situational awareness information is critical. It requires capturing and integrating information from sources such as satellite images, local sensors and social media content generated by local people. A bold obstacle to capturing, representing and integrating such heterogeneous and diverse information is lack of a proper ontology which properly conceptualizes this domain, aggregates and unifies datasets. Thus, in this paper, we introduce empathi ontology which conceptualizes the core concepts concerning with the domain of emergency managing and planning of hazard crises. Although empathi has a coarse-grained view, it considers the necessary concepts and relations being essential in this domain. This ontology is available at https://w3id.org/empathi/

Crowdsourcing as a tool for urban emergency management: lessons from the literature and typology

Recently, citizen involvement has been increasingly used in urban disaster prevention and management, taking advantage of new ubiquitous and collaborative technologies. This scenario has created a unique opportunity to leverage the work of crowds of volunteers. As a result, crowdsourcing approaches for disaster prevention and management have been proposed and evaluated. However, the articulation of citizens, tasks, and outcomes as a continuous flow of knowledge generation reveals a complex ecosystem that requires coordination efforts to manage interdependencies in crowd work. To tackle this challenging problem, this paper extends to the context of urban emergency management the results of a previous study that investigates how crowd work is managed in crowdsourcing platforms applied to urban planning. The goal is to understand how crowdsourcing techniques and quality control dimensions used in urban planning could be used to support urban emergency management, especially in the context of mining-related dam outages. Through a systematic literature review, our study makes a comparison between crowdsourcing tools designed for urban planning and urban emergency management and proposes a five-dimension typology of quality in crowdsourcing, which can be leveraged for optimizing urban planning and emergency management processes

Crowdsourced validation and updating of dynamic features in OpenStreetMap an analysis of shelter mapping after the 2015 Nepal earthquake

The paper presents results from a validation process of OpenStreetMap (OSM) rapid mapping activities using crowdsourcing technology in the aftermath of the Gorkha earthquake 2015 in Nepal. We present a framework and tool to iteratively validate and update OSM objects. Two main objectives are addressed: first, analyzing the accuracy of the volunteered geographic information (VGI) generated by the OSM community; second, investigating the spatio-temporal dynamics of spontaneous shelter camps in Kathmandu. Results from three independent validation iterations show that only 10 % of the OSM objects are false positives (no shelter camps). Unexpectedly, previous mapping experience only had a minor influence on mapping accuracy. The results further show that it is critical to monitor the temporal dynamics. Out of 4,893 identified shelter camps, 54% were already empty/closed six days after the first mapping. So far, updating geographical features during humanitarian crisis is not properly addressed by the existing crowdsourcing approaches

Being specific about geographic information crowdsourcing : a typology and analysis of the Missing Maps project in South Kivu

Recent development in disaster management and humanitarian aid is shaped by the rise of new information sources such as social media or volunteered geographic information. As these show great potential, making sense out of the new geographical datasets is a field of important scientific research. Therefore, this paper attempts to develop a typology of geographical information crowdsourcing. Furthermore, we use this typology to frame existing crowdsourcing projects and to further point out the potential of different kinds of crowdsourcing for disaster management and humanitarian aid. In order to exemplify its practical usage and value, we apply the typology to analyze the crowdsourcing methods utilized by the members of the Missing Maps project developed in South Kiv

Quality of Information in Mobile Crowdsensing: Survey and Research Challenges

Smartphones have become the most pervasive devices in people's lives, and are clearly transforming the way we live and perceive technology. Today's smartphones benefit from almost ubiquitous Internet connectivity and come equipped with a plethora of inexpensive yet powerful embedded sensors, such as accelerometer, gyroscope, microphone, and camera. This unique combination has enabled revolutionary applications based on the mobile crowdsensing paradigm, such as real-time road traffic monitoring, air and noise pollution, crime control, and wildlife monitoring, just to name a few. Differently from prior sensing paradigms, humans are now the primary actors of the sensing process, since they become fundamental in retrieving reliable and up-to-date information about the event being monitored. As humans may behave unreliably or maliciously, assessing and guaranteeing Quality of Information (QoI) becomes more important than ever. In this paper, we provide a new framework for defining and enforcing the QoI in mobile crowdsensing, and analyze in depth the current state-of-the-art on the topic. We also outline novel research challenges, along with possible directions of future work.Comment: To appear in ACM Transactions on Sensor Networks (TOSN

Identifying success factors in crowdsourced geographic information use in government

Crowdsourcing geographic information in government is focusing on projects that are engaging people who are not government officials and employees in collecting, editing and sharing information with governmental bodies. This type of projects emerged in the past decade, due to technological and societal changes - such as the increased use of smartphones, combined with growing levels of education and technical abilities to use them by citizens. They also flourished due to the need for updated data in relatively quick time when financial resources are low. They range from recording the experience of feeling an earthquake to recording the location of businesses during the summer time. 50 cases of projects in which crowdsourced geographic information was used by governmental bodies across the world are analysed. About 60% of the cases were examined in 2014 and in 2017, to allow for comparison and identification of success and failure. The analysis looked at different aspects and their relationship to success: the drivers to start a project; scope and aims; stakeholders and relationships; inputs into the project; technical and organisational aspect; and problems encountered. The main key factors of the case studies were analysed with the use of Qualitative Comparative Analysis (QCA) which is an analytical method that combines quantitative and qualitative tools in sociological research. From the analysis, we can conclude that there is no “magic bullet” or a perfect methodology for a successful crowdsourcing in government project. Unless the organisation has reached maturity in the area of crowdsourcing, identifying a champion and starting a project that will not address authoritative datasets directly is a good way to ensure early success and start the process of organisational learning on how to run such projects. Governmental support and trust is undisputed. If the choice is to use new technologies, this should be accompanied by an investment of appropriate resources within the organisation to ensure that the investment bear fruits. Alternatively, using an existing technology that was successful elsewhere and investing in training and capacity building is another path for success. We also identified the importance of intermediary Non-Governmental Organizations (NGOs) with the experience and knowledge in working with crowdsourcing within a partnership. These organizations have the knowledge and skills to implement projects at the boundary between government and the crowd, and therefore can offer the experience to ensure better implementation. Changes and improvement of public services, or a focus on environmental monitoring can be a good basis for a project. Capturing base mapping is a good point to start, too. The recommendation of the report address organisational issues, resources, and legal aspects