51 research outputs found
Optimization in Knowledge-Intensive Crowdsourcing
We present SmartCrowd, a framework for optimizing collaborative
knowledge-intensive crowdsourcing. SmartCrowd distinguishes itself by
accounting for human factors in the process of assigning tasks to workers.
Human factors designate workers' expertise in different skills, their expected
minimum wage, and their availability. In SmartCrowd, we formulate task
assignment as an optimization problem, and rely on pre-indexing workers and
maintaining the indexes adaptively, in such a way that the task assignment
process gets optimized both qualitatively, and computation time-wise. We
present rigorous theoretical analyses of the optimization problem and propose
optimal and approximation algorithms. We finally perform extensive performance
and quality experiments using real and synthetic data to demonstrate that
adaptive indexing in SmartCrowd is necessary to achieve efficient high quality
task assignment.Comment: 12 page
Crowds, not Drones: Modeling Human Factors in Interactive Crowdsourcing
International audienceIn this vision paper, we propose SmartCrowd, an intelligent and adaptive crowdsourcing framework. Contrary to existing crowdsourcing systems, where the process of hiring workers (crowd), learning their skills, and evaluating the accuracy of tasks they perform are fragmented, siloed, and often ad-hoc, SmartCrowd foresees a paradigm shift in that process, considering unpredictability of human nature, namely human factors. SmartCrowd offers opportunities in making crowdsourcing intelligent through iterative interaction with the workers, and adaptively learning and improving the underlying processes. Both existing (majority of which do not require longer engagement from volatile and mostly non-recurrent workers) and next generation crowdsourcing applications (which require longer engagement from the crowd) stand to benefit from SmartCrowd. We outline the opportunities in SmartCrowd, and discuss the challenges and directions, that can potentially revolutionize the existing crowdsourcing landscape
Crowds, not Drones: Modeling Human Factors in Interactive Crowdsourcing
International audienceIn this vision paper, we propose SmartCrowd, an intelligent and adaptive crowdsourcing framework. Contrary to existing crowdsourcing systems, where the process of hiring workers (crowd), learning their skills, and evaluating the accuracy of tasks they perform are fragmented, siloed, and often ad-hoc, SmartCrowd foresees a paradigm shift in that process, considering unpredictability of human nature, namely human factors. SmartCrowd offers opportunities in making crowdsourcing intelligent through iterative interaction with the workers, and adaptively learning and improving the underlying processes. Both existing (majority of which do not require longer engagement from volatile and mostly non-recurrent workers) and next generation crowdsourcing applications (which require longer engagement from the crowd) stand to benefit from SmartCrowd. We outline the opportunities in SmartCrowd, and discuss the challenges and directions, that can potentially revolutionize the existing crowdsourcing landscape
- …