Crowdsourcing Strategies for Text Creation Tasks

Abstract

International audienceWe examine deployment strategies for text translation and text summarization tasks. We formalize a deployment strategy along three dimensions: work structure, workforce organization , and work style. Work structure can be either simultaneous or sequential, workforce organization independent or collaborative, and work style either crowd-only or hybrid. We use Amazon Mechanical Turk to evaluate the cost, latency, and quality of various deployment strategies. We asses our strategies for different scenarios: short/long text, presence/absence of an outline, and popular/unpopular topics. Our findings serve as a basis to automate the deployment of text creation tasks