Anonymous privacy-preserving task matching in crowdsourcing

With the development of sharing economy, crowdsourcing as a distributed computing paradigm has become increasingly pervasive. As one of indispensable services for most crowdsourcing applications, task matching has also been extensively explored. However, privacy issues are usually ignored during the task matching and few existing privacy-preserving crowdsourcing mechanisms can simultaneously protect both task privacy and worker privacy. This paper systematically analyzes the privacy leaks and potential threats in the task matching and proposes a single-keyword task matching scheme for the multirequester/multiworker crowdsourcing with efficient worker revocation. The proposed scheme not only protects data confidentiality and identity anonymity against the crowd-server, but also achieves query traceability against dishonest or revoked workers. Detailed privacy analysis and thorough performance evaluation show that the proposed scheme is secure and feasible

SybMatch: Sybil detection for privacy-preserving task matching in crowdsourcing

The past decade has witnessed the rise of crowdsourcing, and privacy in crowdsourcing has also gained rising concern in the meantime. In this paper, we focus on the privacy leaks and sybil attacks during the task matching, and propose a privacy-preserving task matching scheme, called SybMatch. The SybMatch scheme can simultaneously protect the privacy of publishers and subscribers against semi-honest crowdsourcing service provider, and meanwhile support the sybil detection against greedy subscribers and efficient user revocation. Detailed security analysis and thorough performance evaluation show that the SybMatch scheme is secure and efficient

SMAP: A Novel Heterogeneous Information Framework for Scenario-based Optimal Model Assignment

The increasing maturity of big data applications has led to a proliferation of models targeting the same objectives within the same scenarios and datasets. However, selecting the most suitable model that considers model's features while taking specific requirements and constraints into account still poses a significant challenge. Existing methods have focused on worker-task assignments based on crowdsourcing, they neglect the scenario-dataset-model assignment problem. To address this challenge, a new problem named the Scenario-based Optimal Model Assignment (SOMA) problem is introduced and a novel framework entitled Scenario and Model Associative percepts (SMAP) is developed. SMAP is a heterogeneous information framework that can integrate various types of information to intelligently select a suitable dataset and allocate the optimal model for a specific scenario. To comprehensively evaluate models, a new score function that utilizes multi-head attention mechanisms is proposed. Moreover, a novel memory mechanism named the mnemonic center is developed to store the matched heterogeneous information and prevent duplicate matching. Six popular traffic scenarios are selected as study cases and extensive experiments are conducted on a dataset to verify the effectiveness and efficiency of SMAP and the score function

Making Task Recommendations in Crowdsourcing Contests

Crowdsourcing contests have emerged as an innovative way for firms to solve business problems by acquiring ideas from participants external to the firm. To facilitate such contests a number of crowdsourcing platforms have emerged in recent years. A crowdsourcing platform provides a two-sided marketplace with one set of members (seekers) posting tasks, and another set of members (solvers) working on these tasks and submitting solutions. As crowdsourcing platforms attract more seekers and solvers, the number of tasks that are open at any time can become quite large. Consequently, solvers search only a limited number of tasks before deciding which one(s) to participate in, often examining only those tasks that appear on the first couple of pages of the task listings. This kind of search behavior has potentially detrimental implications for all parties involved: (i) solvers typically end up participating in tasks they are less likely to win relative some other tasks, (ii) seekers receive solutions of poorer quality compared to a situation where solvers are able to find tasks that they are more likely to win, and (iii) when seekers are not satisfied with the outcome, they may decide to leave the platform; therefore, the platform could lose revenues in the short term and market share in the long term. To counteract these concerns, platforms can provide recommendations to solvers in order to reduce their search costs for identifying the most preferable tasks. This research proposes a methodology to develop a system that can recommend tasks to solvers who wish to participate in crowdsourcing contests. A unique aspect of this environment is that it involves competition among solvers. The proposed approach explicitly models the competition that a solver would face in each open task. The approach makes recommendations based on the probability of the solver winning an open task. A multinomial logit model has been developed to estimate these winning probabilities. We have validated our approach using data from a real crowdsourcing platform

A survey of task-oriented crowdsourcing

Since the advent of artificial intelligence, researchers have been trying to create machines that emulate human behaviour. Back in the 1960s however, Licklider (IRE Trans Hum Factors Electron 4-11, 1960) believed that machines and computers were just part of a scale in which computers were on one side and humans on the other (human computation). After almost a decade of active research into human computation and crowdsourcing, this paper presents a survey of crowdsourcing human computation systems, with the focus being on solving micro-tasks and complex tasks. An analysis of the current state of the art is performed from a technical standpoint, which includes a systematized description of the terminologies used by crowdsourcing platforms and the relationships between each term. Furthermore, the similarities between task-oriented crowdsourcing platforms are described and presented in a process diagram according to a proposed classification. Using this analysis as a stepping stone, this paper concludes with a discussion of challenges and possible future research directions.This work is part-funded by ERDF-European Regional Development Fund through the COMPETE Programme (Operational Programme for Competitiveness) and by National Funds through the FCT-Fundacao para a Ciencia e a Tecnologia (Portuguese Foundation for Science and Technology) within the Ph.D. Grant SFRH/BD/70302/2010 and by the Projects AAL4ALL (QREN11495), World Search (QREN 13852) and FCOMP-01-0124-FEDER-028980 (PTDC/EEI-SII/1386/2012). The authors also thank Jane Boardman for her assistance proof reading the document.info:eu-repo/semantics/publishedVersio

A Labeling Task Design for Supporting Algorithmic Needs: Facilitating Worker Diversity and Reducing AI Bias

Studies on supervised machine learning (ML) recommend involving workers from various backgrounds in training dataset labeling to reduce algorithmic bias. Moreover, sophisticated tasks for categorizing objects in images are necessary to improve ML performance, further complicating micro-tasks. This study aims to develop a task design incorporating the fair participation of people, regardless of their specific backgrounds or task's difficulty. By collaborating with 75 labelers from diverse backgrounds for 3 months, we analyzed workers' log-data and relevant narratives to identify the task's hurdles and helpers. The findings revealed that workers' decision-making tendencies varied depending on their backgrounds. We found that the community that positively helps workers and the machine's feedback perceived by workers could make people easily engaged in works. Hence, ML's bias could be expectedly mitigated. Based on these findings, we suggest an extended human-in-the-loop approach that connects labelers, machines, and communities rather than isolating individual workers.Comment: 45 pages, 4 figure