12,351 research outputs found
Analytic Methods for Optimizing Realtime Crowdsourcing
Realtime crowdsourcing research has demonstrated that it is possible to
recruit paid crowds within seconds by managing a small, fast-reacting worker
pool. Realtime crowds enable crowd-powered systems that respond at interactive
speeds: for example, cameras, robots and instant opinion polls. So far, these
techniques have mainly been proof-of-concept prototypes: research has not yet
attempted to understand how they might work at large scale or optimize their
cost/performance trade-offs. In this paper, we use queueing theory to analyze
the retainer model for realtime crowdsourcing, in particular its expected wait
time and cost to requesters. We provide an algorithm that allows requesters to
minimize their cost subject to performance requirements. We then propose and
analyze three techniques to improve performance: push notifications, shared
retainer pools, and precruitment, which involves recalling retainer workers
before a task actually arrives. An experimental validation finds that
precruited workers begin a task 500 milliseconds after it is posted, delivering
results below the one-second cognitive threshold for an end-user to stay in
flow.Comment: Presented at Collective Intelligence conference, 201
- …