A survey of task-oriented crowdsourcing

Since the advent of artificial intelligence, researchers have been trying to create machines that emulate human behaviour. Back in the 1960s however, Licklider (IRE Trans Hum Factors Electron 4-11, 1960) believed that machines and computers were just part of a scale in which computers were on one side and humans on the other (human computation). After almost a decade of active research into human computation and crowdsourcing, this paper presents a survey of crowdsourcing human computation systems, with the focus being on solving micro-tasks and complex tasks. An analysis of the current state of the art is performed from a technical standpoint, which includes a systematized description of the terminologies used by crowdsourcing platforms and the relationships between each term. Furthermore, the similarities between task-oriented crowdsourcing platforms are described and presented in a process diagram according to a proposed classification. Using this analysis as a stepping stone, this paper concludes with a discussion of challenges and possible future research directions.This work is part-funded by ERDF-European Regional Development Fund through the COMPETE Programme (Operational Programme for Competitiveness) and by National Funds through the FCT-Fundacao para a Ciencia e a Tecnologia (Portuguese Foundation for Science and Technology) within the Ph.D. Grant SFRH/BD/70302/2010 and by the Projects AAL4ALL (QREN11495), World Search (QREN 13852) and FCOMP-01-0124-FEDER-028980 (PTDC/EEI-SII/1386/2012). The authors also thank Jane Boardman for her assistance proof reading the document.info:eu-repo/semantics/publishedVersio

Privacy in crowdsourcing:a systematic review

The advent of crowdsourcing has brought with it multiple privacy challenges. For example, essential monitoring activities, while necessary and unavoidable, also potentially compromise contributor privacy. We conducted an extensive literature review of the research related to the privacy aspects of crowdsourcing. Our investigation revealed interesting gender differences and also differences in terms of individual perceptions. We conclude by suggesting a number of future research directions.</p

Towards an integrated crowdsourcing definition

Crowdsourcing is a relatively recent concept that encompasses many practices. This diversity leads to the blurring of the limits of crowdsourcing that may be identified virtually with any type of internet-based collaborative activity, such as co-creation or user innovation. Varying definitions of crowdsourcing exist, and therefore some authors present certain specific examples of crowdsourcing as paradigmatic, while others present the same examples as the opposite. In this article, existing definitions of crowdsourcing are analysed to extract common elements and to establish the basic characteristics of any crowdsourcing initiative. Based on these existing definitions, an exhaustive and consistent definition for crowdsourcing is presented and contrasted in 11 cases.Estelles Arolas, E.; González-Ladrón-De-Guevara, F. (2012). Towards an integrated crowdsourcing definition. Journal of Information Science. 32(2):189-200. doi:10.1177/0165551512437638S189200322Vukovic, M., & Bartolini, C. (2010). Towards a Research Agenda for Enterprise Crowdsourcing. Leveraging Applications of Formal Methods, Verification, and Validation, 425-434. doi:10.1007/978-3-642-16558-0_36Brabham, D. C. (2008). Crowdsourcing as a Model for Problem Solving. Convergence: The International Journal of Research into New Media Technologies, 14(1), 75-90. doi:10.1177/1354856507084420Vukovic, M. (2009). Crowdsourcing for Enterprises. 2009 Congress on Services - I. doi:10.1109/services-i.2009.56Doan, A., Ramakrishnan, R., & Halevy, A. Y. (2011). Crowdsourcing systems on the World-Wide Web. Communications of the ACM, 54(4), 86. doi:10.1145/1924421.1924442Brabham, D. C. (2008). Moving the crowd at iStockphoto: The composition of the crowd and motivations for participation in a crowdsourcing application. First Monday, 13(6). doi:10.5210/fm.v13i6.2159Huberman, B. A., Romero, D. M., & Wu, F. (2009). Crowdsourcing, attention and productivity. Journal of Information Science, 35(6), 758-765. doi:10.1177/0165551509346786Andriole, S. J. (2010). Business impact of Web 2.0 technologies. Communications of the ACM, 53(12), 67. doi:10.1145/1859204.1859225Denyer, D., Tranfield, D., & van Aken, J. E. (2008). Developing Design Propositions through Research Synthesis. Organization Studies, 29(3), 393-413. doi:10.1177/0170840607088020Egger, M., Smith, G. D., & Altman, D. G. (Eds.). (2001). Systematic Reviews in Health Care. doi:10.1002/9780470693926Tatarkiewicz, W. (1980). A History of Six Ideas. doi:10.1007/978-94-009-8805-7Cosma, G., & Joy, M. (2008). Towards a Definition of Source-Code Plagiarism. IEEE Transactions on Education, 51(2), 195-200. doi:10.1109/te.2007.906776Brabham, D. C. (2009). Crowdsourcing the Public Participation Process for Planning Projects. Planning Theory, 8(3), 242-262. doi:10.1177/1473095209104824Alonso, O., & Lease, M. (2011). Crowdsourcing 101. Proceedings of the fourth ACM international conference on Web search and data mining - WSDM ’11. doi:10.1145/1935826.1935831Bederson, B. B., & Quinn, A. J. (2011). Web workers unite! addressing challenges of online laborers. Proceedings of the 2011 annual conference extended abstracts on Human factors in computing systems - CHI EA ’11. doi:10.1145/1979742.1979606Grier, D. A. (2011). Not for All Markets. Computer, 44(5), 6-8. doi:10.1109/mc.2011.155Heer, J., & Bostock, M. (2010). Crowdsourcing graphical perception. Proceedings of the 28th international conference on Human factors in computing systems - CHI ’10. doi:10.1145/1753326.1753357Heymann, P., & Garcia-Molina, H. (2011). Turkalytics. Proceedings of the 20th international conference on World wide web - WWW ’11. doi:10.1145/1963405.1963473Kazai, G. (2011). In Search of Quality in Crowdsourcing for Search Engine Evaluation. Advances in Information Retrieval, 165-176. doi:10.1007/978-3-642-20161-5_17La Vecchia, G., & Cisternino, A. (2010). Collaborative Workforce, Business Process Crowdsourcing as an Alternative of BPO. Lecture Notes in Computer Science, 425-430. doi:10.1007/978-3-642-16985-4_40Liu, E., & Porter, T. (2010). Culture and KM in China. VINE, 40(3/4), 326-333. doi:10.1108/03055721011071449Oliveira, F., Ramos, I., & Santos, L. (2010). Definition of a Crowdsourcing Innovation Service for the European SMEs. Lecture Notes in Computer Science, 412-416. doi:10.1007/978-3-642-16985-4_37Porta, M., House, B., Buckley, L., & Blitz, A. (2008). Value 2.0: eight new rules for creating and capturing value from innovative technologies. Strategy & Leadership, 36(4), 10-18. doi:10.1108/10878570810888713Ribiere, V. M., & Tuggle, F. D. (Doug). (2010). Fostering innovation with KM 2.0. VINE, 40(1), 90-101. doi:10.1108/03055721011024955Sloane, P. (2011). The brave new world of open innovation. Strategic Direction, 27(5), 3-4. doi:10.1108/02580541111125725Wexler, M. N. (2011). Reconfiguring the sociology of the crowd: exploring crowdsourcing. International Journal of Sociology and Social Policy, 31(1/2), 6-20. doi:10.1108/01443331111104779Whitla, P. (2009). Crowdsourcing and Its Application in Marketing Activities. Contemporary Management Research, 5(1). doi:10.7903/cmr.1145Yang, J., Adamic, L. A., & Ackerman, M. S. (2008). Crowdsourcing and knowledge sharing. Proceedings of the 9th ACM conference on Electronic commerce - EC ’08. doi:10.1145/1386790.1386829Brabham, D. C. (2010). MOVING THE CROWD AT THREADLESS. Information, Communication & Society, 13(8), 1122-1145. doi:10.1080/13691181003624090Giudice, K. D. (2010). Crowdsourcing credibility: The impact of audience feedback on Web page credibility. Proceedings of the American Society for Information Science and Technology, 47(1), 1-9. doi:10.1002/meet.14504701099Stewart, O., Huerta, J. M., & Sader, M. (2009). Designing crowdsourcing community for the enterprise. Proceedings of the ACM SIGKDD Workshop on Human Computation - HCOMP ’09. doi:10.1145/1600150.1600168Maslow, A. H. (1943). A theory of human motivation. Psychological Review, 50(4), 370-396. doi:10.1037/h0054346Veal, A. J. (Ed.). (2002). Leisure and tourism policy and planning. doi:10.1079/9780851995465.0000Dahlander, L., & Gann, D. M. (2010). How open is innovation? Research Policy, 39(6), 699-709. doi:10.1016/j.respol.2010.01.01

Crowdcloud: A Crowdsourced System for Cloud Infrastructure

The widespread adoption of truly portable, smart devices and Do-It-Yourself computing platforms by the general public has enabled the rise of new network and system paradigms. This abundance of wellconnected, well-equipped, affordable devices, when combined with crowdsourcing methods, enables the development of systems with the aid of the crowd. In this work, we introduce the paradigm of Crowdsourced Systems, systems whose constituent infrastructure, or a significant part of it, is pooled from the general public by following crowdsourcing methodologies. We discuss the particular distinctive characteristics they carry and also provide their “canonical” architecture. We exemplify the paradigm by also introducing Crowdcloud, a crowdsourced cloud infrastructure where crowd members can act both as cloud service providers and cloud service clients. We discuss its characteristic properties and also provide its functional architecture. The concepts introduced in this work underpin recent advances in the areas of mobile edge/fog computing and co-designed/cocreated systems

Crowd Intelligence in Requirements Engineering: Current Status and Future Directions

Software systems are the joint creative products of multiple stakeholders, including both designers and users, based on their perception, knowledge and personal preferences of the application context. The rapid rise in the use of Internet, mobile and social media applications make it even more possible to provide channels to link a large pool of highly diversified and physically distributed designers and end users, the crowd. Converging the knowledge of designers and end users in requirements engineering process is essential for the success of software systems. In this paper, we report the findings of a survey of the literature on crowd-based requirements engineering research. It helps us understand the current research achievements, the areas of concentration, and how requirements related activities can be enhanced by crowd intelligence. Based on the survey, we propose a general research map and suggest the possible future roles of crowd intelligence in requirements engineering

Crowdfunding in the Cultural Industries

publishedVersio

Crowdsourcing: A new tool for policy-making?

Crowdsourcing is rapidly evolving and applied in situations where ideas, labour, opinion or expertise of large groups of people are used. Crowdsourcing is now used in various policy-making initiatives; however, this use has usually focused on open collaboration platforms and specific stages of the policy process, such as agenda-setting and policy evaluations. Other forms of crowdsourcing have been neglected in policy-making, with a few exceptions. This article examines crowdsourcing as a tool for policy-making, and explores the nuances of the technology and its use and implications for different stages of the policy process. The article addresses questions surrounding the role of crowdsourcing and whether it can be considered as a policy tool or as a technological enabler and investigates the current trends and future directions of crowdsourcing. Keywords: Crowdsourcing, Public Policy, Policy Instrument, Policy Tool, Policy Process, Policy Cycle, Open Collaboration, Virtual Labour Markets, Tournaments, Competition