Quality Control in Crowdsourcing: A Survey of Quality Attributes, Assessment Techniques and Assurance Actions

Crowdsourcing enables one to leverage on the intelligence and wisdom of potentially large groups of individuals toward solving problems. Common problems approached with crowdsourcing are labeling images, translating or transcribing text, providing opinions or ideas, and similar - all tasks that computers are not good at or where they may even fail altogether. The introduction of humans into computations and/or everyday work, however, also poses critical, novel challenges in terms of quality control, as the crowd is typically composed of people with unknown and very diverse abilities, skills, interests, personal objectives and technological resources. This survey studies quality in the context of crowdsourcing along several dimensions, so as to define and characterize it and to understand the current state of the art. Specifically, this survey derives a quality model for crowdsourcing tasks, identifies the methods and techniques that can be used to assess the attributes of the model, and the actions and strategies that help prevent and mitigate quality problems. An analysis of how these features are supported by the state of the art further identifies open issues and informs an outlook on hot future research directions.Comment: 40 pages main paper, 5 pages appendi

Internal Crowdsourcing in Companies

This open access book examines the implications of internal crowdsourcing (IC) in companies. Presenting an employee-oriented, cross-sector reference model for good IC practice, it discusses the core theoretical foundations, and offers guidelines for process-management and blueprints for the implementation of IC. Furthermore, it examines solutions for employee training and competence development based on crowdsourcing. As such, the book will appeal to scholars of management science, work studies, organizational and participation research and to readers interested in inclusive approaches for cooperative change management and the IT implications for IC platforms

Management of «Systematic Innovation»: A kind of quest for the Holy Grail!

In this paper, authors propose a contribution for improving the open innovation processes. It shows the necessity to get an efficient methodology for open innovation in order to build a computer aided tool for inventive design in Process Systems Engineering (PSE). The proposed methodology will be evocated to be fully used in the context of the “revolutionary” concepts around the so-called factory for the future, also called integrated digital factory, innovative factory… As a result the main contribution of this paper is to propose a software prototype for an Open Computer Aided Innovation 2.0. By definition this open innovation relies on collaboration. This collaboration should enable a community, with a very broad spectrum of skills, to share data, information, knowledge and ideas. As a consequence, a first sub objective is to create a methodological framework that takes advantages of collaboration and collective intelligence (with its capacity to join intelligence and knowledge). Furthermore, the raise of the digital company and more particularly the breakthroughs in information technologies is a powerful enabler to extend and improve the potential of collective intelligence. The second sub objective is to propose a problem resolution process to impel creativity of expert but also to develop, validate and select innovative solutions. After dealing with the importance of Process Innovation and Problem solving investigation in PSE, the proposed approach originally based on an extension of the TRIZ theory (Russian acronym for Theory of Inventive Problem Solving), has been improved by using approach such as case-based reasoning, in order to tackle and revisit problems encountered in the PSE. A case study on biomass is used to illustrate the capabilities of the methodology and the tool

Data Analytics on Online Labor Markets: Opportunities and Challenges

The data-driven economy has led to a significant shortage of data scientists. To address this shortage, this study explores the prospects of outsourcing data analysis tasks to freelancers available on online labor markets (OLMs) by identifying the essential factors for this endeavor. Specifically, we explore the skills required from freelancers, collect information about the skills present on major OLMs, and identify the main hurdles for out-/crowd-sourcing data analysis. Adopting a sequential mixed-method approach, we interviewed 20 data scientists and subsequently surveyed 80 respondents from OLMs. Besides confirming the need for expected skills such as technical/mathematical capabilities, it also identifies less known ones such as domain understanding, an eye for aesthetic data visualization, good communication skills, and a natural understanding of the possibilities/limitations of data analysis in general. Finally, it elucidates obstacles for crowdsourcing like the communication overhead, knowledge gaps, quality assurance, and data confidentiality, which need to be mitigated

Data Analytics on Online Labor Markets: Opportunities and Challenges

The data-driven economy has led to a significant shortage of data scientists. To address this shortage, this study explores the prospects of outsourcing data analysis tasks to freelancers available on online labor markets (OLMs) by identifying the essential factors for this endeavor. Specifically, we explore the skills required from freelancers, collect information about the skills present on major OLMs, and identify the main hurdles for out-/crowd-sourcing data analysis. Adopting a sequential mixed-method approach, we interviewed 20 data scientists and subsequently surveyed 80 respondents from OLMs. Besides confirming the need for expected skills such as technical/mathematical capabilities, it also identifies less known ones such as domain understanding, an eye for aesthetic data visualization, good communication skills, and a natural understanding of the possibilities/limitations of data analysis in general. Finally, it elucidates obstacles for crowdsourcing like the communication overhead, knowledge gaps, quality assurance, and data confidentiality, which need to be mitigated

Control de calidad en sistemas crowdsourcing: un mapeo sistemático

The web 2.0 have been easing the implementation of aneffective model denominated crowdsourcing to find solutions forgiven situations. This model pretend to take the advantages,capacities and proficiency of a group of people (from few ones tothousands of peoples) to solve some tasks. The crowdsourcing isnot a new concept, but thanks to information and communicationstechnologies (ICT) it is evolving in the same way as the new ICThave been develop and applied. Currently, the crowdsourcing isusing in data acquiring process and information generation,establishing a new perspective in the knowledge managementfield. The present work shows the results of a systematic mappingdid to literature review, with the purpose to recognize or learnabout several proposed mechanisms to ensure quality resultswithin crowdsourcing systems and research opportunities aroundthis topic.En el ámbito de la Web 2.0 se ha facilitado la implementación de un modelo efectivo en la solución de problemas denominado crowdsourcing que pretende aprovechar las capacidades y competencias de un grupo grande de individuos para resolver una determinada tarea.  El crowdsourcing no es un concepto nuevo, pero gracias a las nuevas tecnologías de la información y la comunicación está evolucionando a medida que se emplea en nuevas investigaciones y aplicaciones. Actualmente, el crowdsourcing es utilizado, entre otros, en procesos de adquisición y tratamiento de información estableciendo un nuevo enfoque en el campo de la gestión del conocimiento. Este artículo presenta los resultados de un mapeo sistemático realizado a la literatura con el propósito de conocer los diferentes mecanismos propuestos para asegurar la calidad de los resultados en los sistemas crowdsourcing y las oportunidades de investigación se encuentran en torno a este tema.

Control de calidad en sistemas crowdsourcing: un mapeo sistemático

The web 2.0 have been easing the implementation of aneffective model denominated crowdsourcing to find solutions forgiven situations. This model pretend to take the advantages,capacities and proficiency of a group of people (from few ones tothousands of peoples) to solve some tasks. The crowdsourcing isnot a new concept, but thanks to information and communicationstechnologies (ICT) it is evolving in the same way as the new ICThave been develop and applied. Currently, the crowdsourcing isusing in data acquiring process and information generation,establishing a new perspective in the knowledge managementfield. The present work shows the results of a systematic mappingdid to literature review, with the purpose to recognize or learnabout several proposed mechanisms to ensure quality resultswithin crowdsourcing systems and research opportunities aroundthis topic.En el ámbito de la Web 2.0 se ha facilitado la implementación de un modelo efectivo en la solución de problemas denominado crowdsourcing que pretende aprovechar las capacidades y competencias de un grupo grande de individuos para resolver una determinada tarea.  El crowdsourcing no es un concepto nuevo, pero gracias a las nuevas tecnologías de la información y la comunicación está evolucionando a medida que se emplea en nuevas investigaciones y aplicaciones. Actualmente, el crowdsourcing es utilizado, entre otros, en procesos de adquisición y tratamiento de información estableciendo un nuevo enfoque en el campo de la gestión del conocimiento. Este artículo presenta los resultados de un mapeo sistemático realizado a la literatura con el propósito de conocer los diferentes mecanismos propuestos para asegurar la calidad de los resultados en los sistemas crowdsourcing y las oportunidades de investigación se encuentran en torno a este tema.

“The Second Vice is Lying, the First is Running into Debt.” Antecedents and Mitigating Practices of Social Debt: an Exploratory Study in Distributed Software Development Teams

Although much is known about the concept of technical debt in software development, less is known about its social counterpart, also known as social debt. Social debt refers to future consequences of decisions related to people and their interactions. Omissions in social interactions or reduction of communication can foster social debt – and in turn result in negative outcomes in the long run. In this paper, we explore what factors drive and mitigate social debt in distributed agile software development teams. Utilizing an exploratory case study approach, we derive insights from two case organizations. We present antecedents and mitigating factors of social debt related to communication, collaboration, and coordination