2,146 research outputs found
Context-Aware Hierarchical Online Learning for Performance Maximization in Mobile Crowdsourcing
In mobile crowdsourcing (MCS), mobile users accomplish outsourced human
intelligence tasks. MCS requires an appropriate task assignment strategy, since
different workers may have different performance in terms of acceptance rate
and quality. Task assignment is challenging, since a worker's performance (i)
may fluctuate, depending on both the worker's current personal context and the
task context, (ii) is not known a priori, but has to be learned over time.
Moreover, learning context-specific worker performance requires access to
context information, which may not be available at a central entity due to
communication overhead or privacy concerns. Additionally, evaluating worker
performance might require costly quality assessments. In this paper, we propose
a context-aware hierarchical online learning algorithm addressing the problem
of performance maximization in MCS. In our algorithm, a local controller (LC)
in the mobile device of a worker regularly observes the worker's context,
her/his decisions to accept or decline tasks and the quality in completing
tasks. Based on these observations, the LC regularly estimates the worker's
context-specific performance. The mobile crowdsourcing platform (MCSP) then
selects workers based on performance estimates received from the LCs. This
hierarchical approach enables the LCs to learn context-specific worker
performance and it enables the MCSP to select suitable workers. In addition,
our algorithm preserves worker context locally, and it keeps the number of
required quality assessments low. We prove that our algorithm converges to the
optimal task assignment strategy. Moreover, the algorithm outperforms simpler
task assignment strategies in experiments based on synthetic and real data.Comment: 18 pages, 10 figure
Bandit-Based Task Assignment for Heterogeneous Crowdsourcing
We consider a task assignment problem in crowdsourcing, which is aimed at
collecting as many reliable labels as possible within a limited budget. A
challenge in this scenario is how to cope with the diversity of tasks and the
task-dependent reliability of workers, e.g., a worker may be good at
recognizing the name of sports teams, but not be familiar with cosmetics
brands. We refer to this practical setting as heterogeneous crowdsourcing. In
this paper, we propose a contextual bandit formulation for task assignment in
heterogeneous crowdsourcing, which is able to deal with the
exploration-exploitation trade-off in worker selection. We also theoretically
investigate the regret bounds for the proposed method, and demonstrate its
practical usefulness experimentally
- …