Optimization in Knowledge-Intensive Crowdsourcing

We present SmartCrowd, a framework for optimizing collaborative knowledge-intensive crowdsourcing. SmartCrowd distinguishes itself by accounting for human factors in the process of assigning tasks to workers. Human factors designate workers' expertise in different skills, their expected minimum wage, and their availability. In SmartCrowd, we formulate task assignment as an optimization problem, and rely on pre-indexing workers and maintaining the indexes adaptively, in such a way that the task assignment process gets optimized both qualitatively, and computation time-wise. We present rigorous theoretical analyses of the optimization problem and propose optimal and approximation algorithms. We finally perform extensive performance and quality experiments using real and synthetic data to demonstrate that adaptive indexing in SmartCrowd is necessary to achieve efficient high quality task assignment.Comment: 12 page

Engineering Crowdsourced Stream Processing Systems

A crowdsourced stream processing system (CSP) is a system that incorporates crowdsourced tasks in the processing of a data stream. This can be seen as enabling crowdsourcing work to be applied on a sample of large-scale data at high speed, or equivalently, enabling stream processing to employ human intelligence. It also leads to a substantial expansion of the capabilities of data processing systems. Engineering a CSP system requires the combination of human and machine computation elements. From a general systems theory perspective, this means taking into account inherited as well as emerging properties from both these elements. In this paper, we position CSP systems within a broader taxonomy, outline a series of design principles and evaluation metrics, present an extensible framework for their design, and describe several design patterns. We showcase the capabilities of CSP systems by performing a case study that applies our proposed framework to the design and analysis of a real system (AIDR) that classifies social media messages during time-critical crisis events. Results show that compared to a pure stream processing system, AIDR can achieve a higher data classification accuracy, while compared to a pure crowdsourcing solution, the system makes better use of human workers by requiring much less manual work effort

Innovation Labs: Leveraging Openness for Radical Innovation?

A growing range of public, private and civic organisations, from Unicef through Nesta to NHS, now run units known as “innovation labs”. The hopeful assumption they share is that labs, by building on openness among other features, can generate promising solutions to grand challenges of systemic nature. Despite their seeming proliferation and popularisation, the underlying innovation principles embodied by labs have, however, received scant academic attention. This is a missed opportunity, because innovation labs appear to leverage openness for radical innovation in an unusual fashion. Indeed, in this exploratory paper we draw on original interview data and online self-descriptions to illustrate that, beyond convening “uncommon partners” across organisational boundaries, labs apply the principle of openness throughout the innovation process, including the experimentation and development phases. While the emergence of labs clearly forms part of a broader trend towards openness, we show how it transcends established conceptualisations of open innovation (Chesbrough et al., 2006), open science (David, 1998) or open government (Janssen et al., 2012)

Crowds, not Drones: Modeling Human Factors in Interactive Crowdsourcing

International audienceIn this vision paper, we propose SmartCrowd, an intelligent and adaptive crowdsourcing framework. Contrary to existing crowdsourcing systems, where the process of hiring workers (crowd), learning their skills, and evaluating the accuracy of tasks they perform are fragmented, siloed, and often ad-hoc, SmartCrowd foresees a paradigm shift in that process, considering unpredictability of human nature, namely human factors. SmartCrowd offers opportunities in making crowdsourcing intelligent through iterative interaction with the workers, and adaptively learning and improving the underlying processes. Both existing (majority of which do not require longer engagement from volatile and mostly non-recurrent workers) and next generation crowdsourcing applications (which require longer engagement from the crowd) stand to benefit from SmartCrowd. We outline the opportunities in SmartCrowd, and discuss the challenges and directions, that can potentially revolutionize the existing crowdsourcing landscape

Crowds, not Drones: Modeling Human Factors in Interactive Crowdsourcing

International audienceIn this vision paper, we propose SmartCrowd, an intelligent and adaptive crowdsourcing framework. Contrary to existing crowdsourcing systems, where the process of hiring workers (crowd), learning their skills, and evaluating the accuracy of tasks they perform are fragmented, siloed, and often ad-hoc, SmartCrowd foresees a paradigm shift in that process, considering unpredictability of human nature, namely human factors. SmartCrowd offers opportunities in making crowdsourcing intelligent through iterative interaction with the workers, and adaptively learning and improving the underlying processes. Both existing (majority of which do not require longer engagement from volatile and mostly non-recurrent workers) and next generation crowdsourcing applications (which require longer engagement from the crowd) stand to benefit from SmartCrowd. We outline the opportunities in SmartCrowd, and discuss the challenges and directions, that can potentially revolutionize the existing crowdsourcing landscape

Semantic Indexing and Retrieval based on Formal Concept Analysis

Semantic indexing and retrieval has become an important research area, as the available amount of information on the Web is growing more and more. In this paper, we introduce an original approach to semantic indexing and retrieval based on Formal Concept Analysis. The concept lattice is used as a semantic index and we propose an original algorithm for traversing the lattice and answering user queries. This framework has been used and evaluated on song datasets

Capturing the City’s Heritage On-the-Go: Design Requirements for Mobile Crowdsourced Cultural Heritage

Intangible Cultural Heritage is at a continuous risk of extinction. Where historical artefacts engine the machinery of intercontinental mass-tourism, socio-technical changes are reshaping the anthropomorphic landscapes everywhere on the globe, at an unprecedented rate. There is an increasing urge to tap into the hidden semantics and the anecdotes surrounding people, memories and places. The vast cultural knowledge made of testimony, oral history and traditions constitutes a rich cultural ontology tying together human beings, times, and situations. Altogether, these complex, multidimensional features make the task of data-mapping of intangible cultural heritage a problem of sustainability and preservation. This paper addresses a suggested route for conceiving, designing and appraising a digital framework intended to support the conservation of the intangible experience, from a user and a collective-centred perspective. The framework is designed to help capture the intangible cultural value of all places exhibiting cultural-historical significance, supported by an extensive analysis of the literature. We present a set of design recommendations for designing mobile apps that are intended to converge crowdsourcing to Intangible Cultural Heritage