The Crowd on the Assembly Line: Designing Tasks for a Better Crowdsourcing Experience

Leveraging crowd potentials through low paid crowdsourcing micro-tasks has attracted great attention in the last decade as it proves to be a powerful new paradigm to get large amounts of work done quickly. A main challenge for crowdsourcers has been to design tasks that trigger optimum outputs from the crowd while providing crowdsourcees with an experience that would attract them to the platform in the future. Drawing mainly from expectancy theory and the motivation through design of work model, we develop and test a theoretical framework to explore the impact of extrinsic reward valence and perceived task characteristics on perceived output measures in crowdsourcing contexts. We specifically focus on the impact of three crowdsourcing task dimensions: autonomy, skill use, and meaningfulness. Our findings provide support for our model and suggest ways to improve task design, use extrinsic rewards, and provide an enhanced crowdsourcing experience for participants

The Four Pillars of Crowdsourcing: A Reference Model

Crowdsourcing is an emerging business model where tasks are accomplished by the general public; the crowd. Crowdsourcing has been used in a variety of disciplines, including information systems development, marketing and operationalization. It has been shown to be a successful model in recommendation systems, multimedia design and evaluation, database design, and search engine evaluation. Despite the increasing academic and industrial interest in crowdsourcing,there is still a high degree of diversity in the interpretation and the application of the concept. This paper analyses the literature and deduces a taxonomy of crowdsourcing. The taxonomy is meant to represent the different configurations of crowdsourcing in its main four pillars: the crowdsourcer, the crowd, the crowdsourced task and the crowdsourcing platform. Our outcome will help researchers and developers as a reference model to concretely and precisely state their particular interpretation and configuration of crowdsourcing

Considering Human Aspects on Strategies for Designing and Managing Distributed Human Computation

A human computation system can be viewed as a distributed system in which the processors are humans, called workers. Such systems harness the cognitive power of a group of workers connected to the Internet to execute relatively simple tasks, whose solutions, once grouped, solve a problem that systems equipped with only machines could not solve satisfactorily. Examples of such systems are Amazon Mechanical Turk and the Zooniverse platform. A human computation application comprises a group of tasks, each of them can be performed by one worker. Tasks might have dependencies among each other. In this study, we propose a theoretical framework to analyze such type of application from a distributed systems point of view. Our framework is established on three dimensions that represent different perspectives in which human computation applications can be approached: quality-of-service requirements, design and management strategies, and human aspects. By using this framework, we review human computation in the perspective of programmers seeking to improve the design of human computation applications and managers seeking to increase the effectiveness of human computation infrastructures in running such applications. In doing so, besides integrating and organizing what has been done in this direction, we also put into perspective the fact that the human aspects of the workers in such systems introduce new challenges in terms of, for example, task assignment, dependency management, and fault prevention and tolerance. We discuss how they are related to distributed systems and other areas of knowledge.Comment: 3 figures, 1 tabl

Workers’ Task Choice in Crowdsourcing and Human Computation Markets

In human computation systems, humans and computers work together to solve hard problems. Many of these systems use crowdsourcing marketplaces to recruit workers. However, the task selection process of crowdsourcing workers is still unclear. We therefore outline this process and propose a structural model showing the criteria that workers use to choose tasks. The model is based on the person-job fit theory, which includes the measures demands-abilities fit and needs-supplies fit, in order to explain the work intention. We adapt the needs-supplies fit to the specific requirements of crowdsourcing markets by adding concepts for payment fit, enjoyment fit, and time fit. We further assume that the task presentation can have an effect on work intention. In this research-in-progress paper, we present our measures and experimental design as well as our newly developed method for participants’ recruitment. Our work could have strong implications for organizations using crowdsourcing marketplaces

Quality Control in Crowdsourcing: A Survey of Quality Attributes, Assessment Techniques and Assurance Actions

Crowdsourcing enables one to leverage on the intelligence and wisdom of potentially large groups of individuals toward solving problems. Common problems approached with crowdsourcing are labeling images, translating or transcribing text, providing opinions or ideas, and similar - all tasks that computers are not good at or where they may even fail altogether. The introduction of humans into computations and/or everyday work, however, also poses critical, novel challenges in terms of quality control, as the crowd is typically composed of people with unknown and very diverse abilities, skills, interests, personal objectives and technological resources. This survey studies quality in the context of crowdsourcing along several dimensions, so as to define and characterize it and to understand the current state of the art. Specifically, this survey derives a quality model for crowdsourcing tasks, identifies the methods and techniques that can be used to assess the attributes of the model, and the actions and strategies that help prevent and mitigate quality problems. An analysis of how these features are supported by the state of the art further identifies open issues and informs an outlook on hot future research directions.Comment: 40 pages main paper, 5 pages appendi

Designing for Collective Intelligence and Community Resilience on Social Networks

The popularity and ubiquity of social networks has enabled a new form of decentralised online collaboration: groups of users gathering around a central theme and working together to solve problems, complete tasks and develop social connections. Groups that display such ‘organic collaboration’ have been shown to solve tasks quicker and more accurately than other methods of crowdsourcing. They can also enable community action and resilience in response to different events, from casual requests to emergency response and crisis management. However, engaging such groups through formal agencies risks disconnect and disengagement by destabilising motivational structures. This paper explores case studies of this henomenon, reviews models of motivation that can help design systems to harness these groups and proposes a framework for lightweight engagement using existing platforms and social networks

Innovation is created by humans, not by systems: an exploration of user involvement in living labs: user motivation versus lead user criteria

The past few years companies have become more interested in involving users during the production process of their products. On the other hand, a group of users started to innovate on their own. Users also became interested in becoming part of the production processes themselves. Certain users experience certain needs earlier than others and they enjoy finding solutions for these needs. They are called Lead Users (von Hippel, 2005). Living Labs are one possibility for users to realize this interest to innovate. iLab.o, the Living Lab division of iMinds, has been organizing Living Lab research since 2009. To get a better view on the motivations of this panel, we analyzed the behavior of the involved users from September 2009 to December 2013. We tried to detect Lead Users, but it is not obvious to define people as Lead Users because of the different used definitions. Instead, we divided this panel into three types of users based on the intensity of their involvement: passive, sleeping and active users. A small group of users is extremely active and are been defined as “alpha users”. Based on interviews with these alpha users in November and December 2013, a better view on their motivations to keep participating in Living Lab research was made. In this paper we focus on the participation of these different user types in one research phase type within Living Lab research, more specifically co-creation sessions. By means of a comparative case study, we tried to get a better understanding of the behavior of the different user types. It became clear that in order to keep the panel involved it is important to focus on community building

Understanding the Turnover Intention of Crowd Workers of Microtask Crowdsourcing Platform

Microtask crowdsourcing is a relatively new work form enabled by information technologies. For both practitioners and academics, understanding the turnover intention of the users, requesters and crowd workers respectively, of microtask crowdsourcing is very important. However, compared with the relatively large literature on requester, studies focusing on worker crowd workers are limited. Therefore, in this study, we investigate the crowd workers’ intentions to leave the microtask crowdsourcing. The research goal is to analyze the motivations of crowd workers systematically and identify those factors that influenced their turnover intention. Based on perceived value and justice perspectives, a research model is developed. The proposed hypotheses will be tested using data from Amazon Mechanical Turk

Not so Shore Anymore: The New Imperatives When Sourcing in the Age of Open

Software outsourcing has been the subject of much research in the past 25 years, largely because of potential cost savings envisaged through lower labour costs, ‘follow-the-sun’ development, access to skilled developers, and proximity to new markets. In recent years, the success of the open source phe-nomenon has inspired a number of new forms of sourcing that combine the potential of global sourcing with the elusive and much sought-after possibility of increased innovation. Three of these new forms of sourcing are opensourcing, innersourcing and crowdsourcing. Based on a comparative analysis of a number of case studies of these forms of sourcing, we illustrate how they differ in both significant and subtle ways from outsourcing. We conclude that these emerging sourcing approaches call for conceptual development and refocusing. Specifically, to understand software sourcing in the age of open, the important concept is no longer ‘shoring,’ but rather five identified imperatives (governance sharedness, unknownness, intrinsicness, innovativeness and co-opetitiveness) and their implications for the development situation at hand

Motivasi Pengguna Dalam Menggunakan Metode Crowdsourcing Pada Pembuatan Perangkat Lunak

Perkembangan metode pada pengembangan perangkat lunak telah meningkat pada akhir-akhir ini, dengan meningkatnya teknologi dan kebutuhan pasar, metode crowdsourcing telah berkembang dan mendapat tingkat popularitas yang tinggi dikalangan masyarakat. Metode crowdsourcing lebih condong mengandalkan kekuatan orang banyak sebagai kemampuan utama dalam produksinya. Meskipun begitu, sejak crowdsourcing menjadi kekuatan utama baru dan merambah ke dunia pembuatan perangkat lunak, kualitas pada perangkat lunak menjadi dipertanyakan. Crowdsourcing memiliki perbedaan dengan alur pembuatan perangkat lunak secara tradisional seperti Software Life Development Cycle maupun Waterfall Model, selain itu metode crowdsourcing mengandalkan kekuatan keramaian pada saat pembuatanya. Beberapa studi dan jurnal sebelumnya beranggapan bahwa motivasi merupakan kunci utama kesuksesan ketika metode crowdsourcing digunakan untuk memproduksi sebuah produk. Pada studi ini diajukan model yang dikombinasikan dari dua teori utama untuk menjawab pertanyaan tentang motivasi penggunaan crowdsourcing untuk pembuatan software yaitu teori self-determination, dan IS success model untuk lebih mengerti tentang hubunganya intensitas pengguna dengan kepuasan pada pengguna pada kasus pengembangan perangkat lunak dengan metode crowdsourcing ==================================================================== Software Development has increased emerging new methods in its development, with the advancement of digitalization, technology and global networking, Crowdsourcing has been developed and gaining popularity among the people. Unlike the outsourcing, crowdsourcing is more emphasis on the power of crowds as major power production. This study will discuss crowdsourcing activity that focused on software development. Software engineering is a process which software is written a complex process without compromising the quality of the software. However, since crowdsourcing software engineering relies on its robust method to produce a software and entirely different from traditional software engineering, their quality are questionable. A major issue in of crowdsourcing is how to attract and to sustain for development. Motivation is a matter that should be investigated further by the researchers for better crowdsourcing development to bring right crowds to the table so it can sustain the crowdsourcing activity. This study discusses more a several factors motivation that can be an impact, an influence to the development of crowdsourcing in software development. To improve these study findings, this study also combines two major theories about self-determination and IS Success Model to investigate further about motivation the users joined crowdsourcing on software development and to understand the impact of user satisfaction in case of crowdsourcing on software developmen