Fairness and Transparency in Crowdsourcing

International audienceDespite the success of crowdsourcing, the question of ethics has not yet been addressed in its entirety. Existing efforts have studied fairness in worker compensation and in helping requesters detect malevolent workers. In this paper, we propose fairness axioms that generalize existing work and pave the way to studying fairness for task assignment, task completion, and worker compensation. Transparency on the other hand, has been addressed with the development of plug-ins and forums to track workers' performance and rate requesters. Similarly to fairness, we define transparency axioms and advocate the need to address it in a holistic manner by providing declarative specifications. We also discuss how fairness and transparency could be enforced and evaluated in a crowdsourcing platform

Crowdsourcing Strategies for Text Creation Tasks

International audienceWe examine deployment strategies for text translation and text summarization tasks. We formalize a deployment strategy along three dimensions: work structure, workforce organization , and work style. Work structure can be either simultaneous or sequential, workforce organization independent or collaborative, and work style either crowd-only or hybrid. We use Amazon Mechanical Turk to evaluate the cost, latency, and quality of various deployment strategies. We asses our strategies for different scenarios: short/long text, presence/absence of an outline, and popular/unpopular topics. Our findings serve as a basis to automate the deployment of text creation tasks

Task Assignment Strategies for Crowd Worker Ability Improvement

International audienceWorkers are the most important resource in crowdsourcing. However, only investing in worker-centric needs, such as skill improvement, often conflicts with short-term platform-centric needs, such as task throughput. This paper studies learning strategies in task assignment in crowdsourcing and their impact on platform-centric needs. We formalize learning potential of individual tasks and collaborative tasks, and devise an iterative task assignment and completion approach that implements strategies grounded in learning theories. We conduct experiments to compare several learning strategies in terms of skill improvement, and in terms of task throughput and contribution quality. We discuss how our findings open new research directions in learning and collaboration. CCS Concepts: • Human-centered computing → Empirical studies in collaborative and social computing

User group analytics: hypothesis generation and exploratory analysis of user data

International audienc

Customizing Travel Packages with Interactive Composite Items

International audienc

GroupTravel: Customizing Travel Packages for Groups

International audienceWe present GroupTravel, a framework that generates cus-tomized travel packages (TPs) for a group of individuals. GroupTravel implements different consensus functions proposed in group recommendation to reach agreement among members. Given a group whose members provide a travel query, GroupTravel returns k Composite Items (CIs) of Points Of Interest (POIs) that are valid, representative, cohesive and personalized. Validity is achieved by satisfying the query expressed by the group. Representativity ensures good coverage of a city. Cohesiveness reflects geographic proximity of POIs forming a CI. Personalization is achieved by choosing POIs that best match the travel preferences of group members. Additionally, group members can interact with generated TPs to customize them. With extensive synthetic experiments and user studies, we examine the benefit of personalization and the impact of different group consensus on user satisfaction. We also show that providing the ability to interact with TPs and reflecting that in the consensus yields better TPs

GroupTravel: Customizing Travel Packages for Groups

International audienceWe present GroupTravel, a framework that generates cus-tomized travel packages (TPs) for a group of individuals. GroupTravel implements different consensus functions proposed in group recommendation to reach agreement among members. Given a group whose members provide a travel query, GroupTravel returns k Composite Items (CIs) of Points Of Interest (POIs) that are valid, representative, cohesive and personalized. Validity is achieved by satisfying the query expressed by the group. Representativity ensures good coverage of a city. Cohesiveness reflects geographic proximity of POIs forming a CI. Personalization is achieved by choosing POIs that best match the travel preferences of group members. Additionally, group members can interact with generated TPs to customize them. With extensive synthetic experiments and user studies, we examine the benefit of personalization and the impact of different group consensus on user satisfaction. We also show that providing the ability to interact with TPs and reflecting that in the consensus yields better TPs

Interactive Generation and Customization of Travel Packages for Individuals and Groups

International audienc

Task Composition in Crowdsourcing

International audienceCrowdsourcing has gained popularity in a variety of domains as an increasing number of jobs are " taskified " and completed independently by a set of workers. A central process in crowdsourcing is the mechanism through which workers find tasks. On popular platforms such as Amazon Mechanical Turk, tasks can be sorted by dimensions such as creation date or reward amount. Research efforts on task assignment have focused on adopting a requester-centric approach whereby tasks are proposed to workers in order to maximize overall task throughput, result quality and cost. In this paper, we advocate the need to complement that with a worker-centric approach to task assignment, and examine the problem of producing, for each worker, a personalized summary of tasks that preserves overall task throughput. We formalize task composition for workers as an optimization problem that finds a representative set of k valid and relevant Composite Tasks (CTs). Validity enforces that a composite task complies with the task arrival rate and satisfies the worker's expected wage. Relevance imposes that tasks match the worker's qualifications. We show empirically that workers' experience is greatly improved due to task homogeneity in each CT and to the adequation of CTs with workers' skills. As a result task throughput is improved