Engineering Crowdsourced Stream Processing Systems

A crowdsourced stream processing system (CSP) is a system that incorporates crowdsourced tasks in the processing of a data stream. This can be seen as enabling crowdsourcing work to be applied on a sample of large-scale data at high speed, or equivalently, enabling stream processing to employ human intelligence. It also leads to a substantial expansion of the capabilities of data processing systems. Engineering a CSP system requires the combination of human and machine computation elements. From a general systems theory perspective, this means taking into account inherited as well as emerging properties from both these elements. In this paper, we position CSP systems within a broader taxonomy, outline a series of design principles and evaluation metrics, present an extensible framework for their design, and describe several design patterns. We showcase the capabilities of CSP systems by performing a case study that applies our proposed framework to the design and analysis of a real system (AIDR) that classifies social media messages during time-critical crisis events. Results show that compared to a pure stream processing system, AIDR can achieve a higher data classification accuracy, while compared to a pure crowdsourcing solution, the system makes better use of human workers by requiring much less manual work effort

An Abstract Formal Basis for Digital Crowds

Crowdsourcing, together with its related approaches, has become very popular in recent years. All crowdsourcing processes involve the participation of a digital crowd, a large number of people that access a single Internet platform or shared service. In this paper we explore the possibility of applying formal methods, typically used for the verification of software and hardware systems, in analysing the behaviour of a digital crowd. More precisely, we provide a formal description language for specifying digital crowds. We represent digital crowds in which the agents do not directly communicate with each other. We further show how this specification can provide the basis for sophisticated formal methods, in particular formal verification.Comment: 32 pages, 4 figure

Towards a classification framework for social machines

The state of the art in human interaction with computational systems blurs the line between computations performed by machine logic and algorithms, and those that result from input by humans, arising from their own psychological processes and life experience. Current socio-technical systems, known as ‘social machines’ exploit the large-scale interaction of humans with machines. Interactions that are motivated by numerous goals and purposes including financial gain, charitable aid, and simply for fun. In this paper we explore the landscape of social machines, both past and present, with the aim of defining an initial classificatory framework. Through a number of knowledge elicitation and refinement exercises we have identified the polyarchical relationship between infrastructure, social machines, and large-scale social initiatives. Our initial framework describes classification constructs in the areas of contributions, participants, and motivation. We present an initial characterization of some of the most popular social machines, as demonstration of the use of the identified constructs. We believe that it is important to undertake an analysis of the behaviour and phenomenology of social machines, and of their growth and evolution over time. Our future work will seek to elicit additional opinions, classifications and validation from a wider audience, to produce a comprehensive framework for the description, analysis and comparison of social machines

Business Processes for the Crowd Computer

open7noKucherbaev, Pavel; Tranquillini, Stefano; Daniel, Florian; Casati, Fabio; Marchese, Maurizio; Brambilla, Marco; Fraternali, PieroKucherbaev, Pavel; Tranquillini, Stefano; Daniel, Florian; Casati, Fabio; Marchese, Maurizio; Brambilla, Marco; Fraternali, Pier

Crime applications and social machines: crowdsourcing sensitive data

The authors explore some issues with the United Kingdom (U.K.) crime reporting and recording systems which currently produce Open Crime Data. The availability of Open Crime Data seems to create a potential data ecosystem which would encourage crowdsourcing, or the creation of social machines, in order to counter some of these issues. While such solutions are enticing, we suggest that in fact the theoretical solution brings to light fairly compelling problems, which highlight some limitations of crowdsourcing as a means of addressing Berners-Lee’s “social constraint.” The authors present a thought experiment – a Gendankenexperiment - in order to explore the implications, both good and bad, of a social machine in such a sensitive space and suggest a Web Science perspective to pick apart the ramifications of this thought experiment as a theoretical approach to the characterisation of social machine