Theoretical Underpinnings and Practical Challenges of Crowdsourcing as a Mechanism for Academic Study

Researchers in a variety of fields are increasingly adopting crowdsourcing as a reliable instrument for performing tasks that are either complex for humans and computer algorithms. As a result, new forms of collective intelligence have emerged from the study of massive crowd-machine interactions in scientific work settings as a field for which there is no known theory or model able to explain how it really works. Such type of crowd work uses an open participation model that keeps the scientific activity (including datasets, methods, guidelines, and analysis results) widely available and mostly independent from institutions, which distinguishes crowd science from other crowd-assisted types of participation. In this paper, we build on the practical challenges of crowd-AI supported research and propose a conceptual framework for addressing the socio-technical aspects of crowd science from a CSCW viewpoint. Our study reinforces a manifested lack of systematic and empirical research of the symbiotic relation of AI with human computation and crowd computing in scientific endeavors

Crowdsourcing and Scholarly Culture: Understanding Expertise in an Age of Popularism

The increasing volume of digital material available to the humanities creates clear potential for crowdsourcing. However, tasks in the digital humanities typically do not satisfy the standard requirement for decomposition into microtasks each of which must require little expertise on behalf of the worker and little context of the broader task. Instead, humanities tasks require scholarly knowledge to perform and even where sub-tasks can be extracted, these often involve broader context of the document or corpus from which they are extracted. That is the tasks are macrotasks, resisting simple decomposition. Building on a case study from musicology, the In Concert project, we will explore both the barriers to crowdsourcing in the creation of digital corpora and also examples where elements of automatic processing or less-expert work are possible in a broader matrix that also includes expert microtasks and macrotasks. Crucially we will see that the macrotask–microtask distinction is nuanced: it is often possible to create a partial decomposition into less-expert microtasks with residual expert macrotasks, and crucially do this in ways that preserve scholarly values

Unleashing the Potential of Crowd Work: The Need for a Post-Taylorism Crowdsourcing Model

Paid crowdsourcing connects task requesters to a globalized, skilled workforce that is available 24/7. In doing so, this new labor model promises not only to complete work faster and more efficiently than any previous approach but also to harness the best of our collective capacities. Nevertheless, for almost a decade now, crowdsourcing has been limited to addressing rather straightforward and simple tasks. Large-scale innovation, creativity, and wicked problem solving are still largely out of the crowd’s reach. In this opinion paper, we argue that existing crowdsourcing practices bear significant resemblance to the management paradigm of Taylorism. Although criticized and often abandoned by modern organizations, Taylorism principles are prevalent in many crowdsourcing platforms, which employ practices such as the forceful decomposition of all tasks regardless of their knowledge nature and the disallowing of worker interactions, which diminish worker motivation and performance. We argue that a shift toward post-Taylorism is necessary to enable the crowd address at scale the complex problems that form the backbone of today’s knowledge economy. Drawing from recent literature, we highlight four design rules that can help make this shift, namely, endorsing social crowd networks, encouraging teamwork, scaffolding ownership of one’s work within the crowd, and leveraging algorithm-guided worker self-coordination.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/171075/1/Lykourentzou et al. 2021.pdfDescription of Lykourentzou et al. 2021.pdf : Final ArticleSEL

An Examination of the Work Practices of Crowdfarms

Crowdsourcing is a new value creation business model. Annual revenue of the Chinese market alone is hundreds of millions of dollars, yet few studies have focused on the practices of the Chinese crowdsourcing workforce, and those that do mainly focus on solo crowdworkers. We have extended our study of solo crowdworker practices to include crowdfarms, a relatively new entry to the gig economy: small companies that carry out crowdwork as a key part of their business. We report here on interviews of people who work in 53 crowdfarms. We describe how crowdfarms procure jobs, carry out macrotasks and microtasks, manage their reputation, and employ different management practices to motivate crowdworkers and customers

Who wants to cooperate-and why? Attitude and perception of crowd workers in online labor markets

Existing literature and studies predominantly focus on how crowdsource workers individually complete tasks and projects. Our study examines crowdsource workers' willingness to work collaboratively. We report results from a survey of 122 workers on a leading online labor platform (Upwork) to examine crowd workers' behavioral preferences for collaboration and explore several antecedents of cooperative behaviors. We then test if actual cooperative behavior matches with workers' behavioral preferences through an incentivized social dilemma experiment. We find that respondents cooperate at a higher rate (85%) than reported in previous comparable studies (between 50-75%). This high rate of cooperation is likely explained by an ingroup bias. Using a sequential mediation model we demonstrate the importance of a sense of shared expectations and accountability for cooperation. We contribute to a better understanding of the potential for collaborative work in crowdsourcing by accessing if and what social factors and collective culture exist among crowd workers. We discuss the implications of our results for platform designers by highlighting the importance of platform features that promote shared expectations and improve accountability. Overall, contrary to existing literature and predictions, our results suggest that crowd workers display traits that are more consistent with belonging to a coherent group with a shared collective culture, rather than being anonymous actors in a transaction-based market

Crowdsourcing in China: Exploring the Work Experience of Solo Crowdworkers and Crowdfarm Workers

Recent research highlights the potential of crowdsourcing in China. Yet very few studies explore the workplace context and experiences of Chinese crowdworkers. Those that do, focus mainly on the work experiences of solo crowdworkers but do not deal with issues pertaining to the substantial amount of people working in ‘crowdfarms’. This article addresses this gap as one of its primary concerns. Drawing on a study that involves 48 participants, our research explores, compares and contrasts the work experiences of solo crowdworkers to those of crowdfarm workers. Our findings illustrate that the work experiences and context of the solo workers and crowdfarm workers are substantially different, with regards to their motivations, the ways they engage with crowdsourcing, the tasks they work on, and the crowdsourcing platforms they utilize. Overall, our study contributes to furthering the understandings on the work experiences of crowdworkers in China

Crowdsourcing Controls: A Review and Research Agenda for Crowdsourcing Controls Used for Macro-tasks

Crowdsourcing—the employment of ad hoc online labor to perform various tasks—has become a popular outsourcing vehicle. Our current approach to crowdsourcing—focusing on micro-tasks—fails to leverage the potential of crowds to tackle more complex problems. To leverage crowds to tackle more complex macro tasks requires a better comprehension of crowdsourcing controls. Crowdsourcing controls are mechanisms used to align crowd workers’ actions with predefined standards to achieve a set of goals and objectives. Unfortunately, we know very little about the topic of crowdsourcing controls directed at accomplishing complex macro tasks. To address issues associated with crowdsourcing controls formacro-tasks, this chapter has several objectives. First, it presents and discusses the literature on control theory. Second, this chapter presents a scoping literature review of crowdsourcing controls. Finally, the chapter identifies gaps and puts forth a research agenda to address these shortcomings. The research agenda focuses on understanding how to employ the controls needed to perform macro-tasking in crowds and the implications for crowdsourcing system designers.National Science Foundation grant CHS-1617820Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/150493/1/Robert 2019 Preprint Chapter 3.pdfDescription of Robert 2019 Preprint Chapter 3.pdf : PrePrint Versio

Draft: Crowdsourcing in cultural heritage: a practical guide to designing and running successful projects

Have you ever wanted to recruit hundreds of members of the public to assist with the task of making cultural heritage collections findable online? Or to connect with passionate volunteers who'll share their discoveries with you? Crowdsourcing in cultural heritage is a broad term for projects that ask the public to help with tasks that contribute to a shared, significant goal or research interest related to cultural heritage collections or knowledge. As participants receive no financial reward, the activities and/or goals should be inherently rewarding for those volunteering their time. This definition is partly descriptive and partly proscriptive, and this chapter is largely concerned with explaining/describing how to meet the standards it implies. [A draft (not quite pre-print) version of my chapter for the Routledge International Handbook of Research Methods in Digital Humanities, edited by Kristen Schuster, Stuart Dunn, 2021. ISBN 9781138363021