The Four Pillars of Crowdsourcing: A Reference Model

Crowdsourcing is an emerging business model where tasks are accomplished by the general public; the crowd. Crowdsourcing has been used in a variety of disciplines, including information systems development, marketing and operationalization. It has been shown to be a successful model in recommendation systems, multimedia design and evaluation, database design, and search engine evaluation. Despite the increasing academic and industrial interest in crowdsourcing,there is still a high degree of diversity in the interpretation and the application of the concept. This paper analyses the literature and deduces a taxonomy of crowdsourcing. The taxonomy is meant to represent the different configurations of crowdsourcing in its main four pillars: the crowdsourcer, the crowd, the crowdsourced task and the crowdsourcing platform. Our outcome will help researchers and developers as a reference model to concretely and precisely state their particular interpretation and configuration of crowdsourcing

Control de calidad en sistemas crowdsourcing: un mapeo sistemático

The web 2.0 have been easing the implementation of aneffective model denominated crowdsourcing to find solutions forgiven situations. This model pretend to take the advantages,capacities and proficiency of a group of people (from few ones tothousands of peoples) to solve some tasks. The crowdsourcing isnot a new concept, but thanks to information and communicationstechnologies (ICT) it is evolving in the same way as the new ICThave been develop and applied. Currently, the crowdsourcing isusing in data acquiring process and information generation,establishing a new perspective in the knowledge managementfield. The present work shows the results of a systematic mappingdid to literature review, with the purpose to recognize or learnabout several proposed mechanisms to ensure quality resultswithin crowdsourcing systems and research opportunities aroundthis topic.En el ámbito de la Web 2.0 se ha facilitado la implementación de un modelo efectivo en la solución de problemas denominado crowdsourcing que pretende aprovechar las capacidades y competencias de un grupo grande de individuos para resolver una determinada tarea.  El crowdsourcing no es un concepto nuevo, pero gracias a las nuevas tecnologías de la información y la comunicación está evolucionando a medida que se emplea en nuevas investigaciones y aplicaciones. Actualmente, el crowdsourcing es utilizado, entre otros, en procesos de adquisición y tratamiento de información estableciendo un nuevo enfoque en el campo de la gestión del conocimiento. Este artículo presenta los resultados de un mapeo sistemático realizado a la literatura con el propósito de conocer los diferentes mecanismos propuestos para asegurar la calidad de los resultados en los sistemas crowdsourcing y las oportunidades de investigación se encuentran en torno a este tema.

Control de calidad en sistemas crowdsourcing: un mapeo sistemático

The web 2.0 have been easing the implementation of aneffective model denominated crowdsourcing to find solutions forgiven situations. This model pretend to take the advantages,capacities and proficiency of a group of people (from few ones tothousands of peoples) to solve some tasks. The crowdsourcing isnot a new concept, but thanks to information and communicationstechnologies (ICT) it is evolving in the same way as the new ICThave been develop and applied. Currently, the crowdsourcing isusing in data acquiring process and information generation,establishing a new perspective in the knowledge managementfield. The present work shows the results of a systematic mappingdid to literature review, with the purpose to recognize or learnabout several proposed mechanisms to ensure quality resultswithin crowdsourcing systems and research opportunities aroundthis topic.En el ámbito de la Web 2.0 se ha facilitado la implementación de un modelo efectivo en la solución de problemas denominado crowdsourcing que pretende aprovechar las capacidades y competencias de un grupo grande de individuos para resolver una determinada tarea.  El crowdsourcing no es un concepto nuevo, pero gracias a las nuevas tecnologías de la información y la comunicación está evolucionando a medida que se emplea en nuevas investigaciones y aplicaciones. Actualmente, el crowdsourcing es utilizado, entre otros, en procesos de adquisición y tratamiento de información estableciendo un nuevo enfoque en el campo de la gestión del conocimiento. Este artículo presenta los resultados de un mapeo sistemático realizado a la literatura con el propósito de conocer los diferentes mecanismos propuestos para asegurar la calidad de los resultados en los sistemas crowdsourcing y las oportunidades de investigación se encuentran en torno a este tema.

Crowdsourcing as part of producing content for a critical reading comprehension game

Abstract. The purpose of this thesis was to examine how crowdsourcing can be used to create and validate data on a topic, misleading graphs, that are difficult for people to interpret. In crowdsourcing tasks, the worker is shown a graph that is intentionally designed to be misleading, from which the worker is supposed to create four headline options that are used as content of a critical reading comprehension game. To ensure the quality of the headlines, they are validated using crowdsourcing and two expert evaluators. As a result of the thesis, a graphical user interface was created from which crowdsourcing projects could be managed. The major challenge of crowdsourcing is quality control when unknown people from different backgrounds perform tasks on a different basis. The tasks were formed around a tricky topic, in which case it is difficult to keep the amount of usable data high in relation to the total amount of gathered data. The topics of the graphs and the task interface were intentionally designed to be simple so as not to take too much focus from the context of the misleading graph. The results show that there is a lot of variation in the quality of the responses although an effort was made to select the best among the workers. It was noticeable that misleading graphs or assignments were often misinterpreted in the task of creating headlines. A small part of the responses was completely in accordance with the assignment. In the task of validating headlines, the worker’s task was to choose one of the three options, which was used to determine how well the headline formed in the previous task corresponded to the assignment. The results show that it was too easy for the worker to click and move on to the next task without proper consideration

Playing Planning Poker in Crowds: Human Computation of Software Effort Estimates

Reliable cost effective effort estimation remains a considerable challenge for software projects. Recent work has demonstrated that the popular Planning Poker practice can produce reliable estimates when undertaken within a software team of knowledgeable domain experts. However, the process depends on the availability of experts and can be time-consuming to perform, making it impractical for large scale or open source projects that may curate many thousands of outstanding tasks. This paper reports on a full study to investigate the feasibility of using crowd workers supplied with limited information about a task to provide comparably accurate estimates using Planning Poker. We describe the design of a Crowd Planning Poker (CPP) process implemented on Amazon Mechanical Turk and the results of a substantial set of trials, involving more than 5000 crowd workers and 39 diverse software tasks. Our results show that a carefully organised and selected crowd of workers can produce effort estimates that are of similar accuracy to those of a single expert