Competing or aiming to be average?: Normification as a means of engaging digital volunteers

Engagement, motivation and active contribution by digital volunteers are key requirements for crowdsourcing and citizen science projects. Many systems use competitive elements, for example point scoring and leaderboards, to achieve these ends. However, while competition may motivate some people, it can have a neutral or demotivating effect on others. In this paper we explore theories of personal and social norms and investigate normification as an alternative approach to engagement, to be used alongside or instead of competitive strategies. We provide a systematic review of existing crowdsourcing and citizen science literature and categorise the ways that theories of norms have been incorporated to date. We then present qualitative interview data from a pro-environmental crowdsourcing study, Close the Door, which reveals normalising attitudes in certain participants. We assess how this links with competitive behaviour and participant performance. Based on our findings and analysis of norm theories, we consider the implications for designers wishing to use normification as an engagement strategy in crowdsourcing and citizen science systems

Engineering Crowdsourced Stream Processing Systems

A crowdsourced stream processing system (CSP) is a system that incorporates crowdsourced tasks in the processing of a data stream. This can be seen as enabling crowdsourcing work to be applied on a sample of large-scale data at high speed, or equivalently, enabling stream processing to employ human intelligence. It also leads to a substantial expansion of the capabilities of data processing systems. Engineering a CSP system requires the combination of human and machine computation elements. From a general systems theory perspective, this means taking into account inherited as well as emerging properties from both these elements. In this paper, we position CSP systems within a broader taxonomy, outline a series of design principles and evaluation metrics, present an extensible framework for their design, and describe several design patterns. We showcase the capabilities of CSP systems by performing a case study that applies our proposed framework to the design and analysis of a real system (AIDR) that classifies social media messages during time-critical crisis events. Results show that compared to a pure stream processing system, AIDR can achieve a higher data classification accuracy, while compared to a pure crowdsourcing solution, the system makes better use of human workers by requiring much less manual work effort

Towards understanding the process of tournament crowdsourcing:the value co-creation perspective

In the contemporary business environment, firms are increasingly moving from creating business value internally to co-creating business value with external stakeholders. Value cocreation refers to the process where a focal firm involves external stakeholders in its previously in-house performed business processes and interacts intensively with each other to create a stream of value. Tournament crowdsourcing, as an application of crowdsourcing, has become an emerging approach for firms to engage with external crowds in pursuit of business value. In the existing Information Systems literature, scholars’ understanding of valueco-creation and crowdsourcing is still at an explorative stage. The process of value co-creation and crowdsourcing have not been extensively studied. In this research, we adopt an interpretive approach and employ multiple-case designs to investigate the process of tournament crowdsourcing through the lens of value co-creation. The findings of this research contribute to the literature on crowdsourcing by 1) introducing the process framework which examines value-generating phases and value propositions from both the perspective of the focal entity and the crowd, 2) revealing the dynamic involvement of the crowd, the process from value creation to value co-creation, and the dynamic value stream, 3) identifying the combined usage of multiple systems and mechanisms for tournament crowdsourcing by contemporary platforms, and potential conflicts related to the governance of the platform,and 4) identifying phases and associated activities relevant to finding the right crowd members from the perspective of the focal entity during the process of tournament crowdsourcing. The findings of this research also contribute to the literature on value cocreation by 1) introducing a thorough definition of value co-creation, 2) conceptually and empirically enriching the most salient components in value co-creation, and 3) bringing in new insights into the value co-creation phenomenon by examining the context of tournament crowdsourcing. In practical terms, the findings of this research may inspire practitioners of generating better understanding about their roles in facilitating value co-creation, the strategic usage of systems and mechanisms, being aware of potential conflicts and finding the right crowd members when conducting tournament crowdsourcing initiatives

Human Beyond the Machine: Challenges and Opportunities of Microtask Crowdsourcing

In the 21st century, where automated systems and artificial intelligence are replacing arduous manual labor by supporting data-intensive tasks, many problems still require human intelligence. Over the last decade, by tapping into human intelligence through microtasks, crowdsourcing has found remarkable applications in a wide range of domains. In this article, the authors discuss the growth of crowdsourcing systems since the term was coined by columnist Jeff Howe in 2006. They shed light on the evolution of crowdsourced microtasks in recent times. Next, they discuss a main challenge that hinders the quality of crowdsourced results: the prevalence of malicious behavior. They reflect on crowdsourcing's advantages and disadvantages. Finally, they leave the reader with interesting avenues for future research

Crowdsourcing defense acquisition programs

MBA Professional Report. Crowdsourcing solutions have the potential to meet the Army's modernization goals. With the rise of improved Internet access and online resources, crowdsourcing has been increasing in popularity since 2006. The benefits of crowdsourcing have been visible in commercial industry and can apply to Department of Defense (DOD) Acquisition Programs. This report identifies the overall use of crowdsourcing, looks at cases in the DOD and in industry, and analyzes strengths and weaknesses. Our findings consist of crowdsourcing strategies that can benefit the DOD and include prize competitions, open dialogue, and open-data collaboration. Integrating the crowd-force with defense contractors through online collaboration platforms can speed up the time required to find solutions and reduce program costs. Barriers include senior-level leaderships' reluctance to change, risks associated with opening up the DOD to crowdsourcing, and the DOD's unwillingness to adapt to new ways of innovation. Recommendations include that Congress pass laws directing the use of open innovation, crowdsourcing, and implementing directives across federal agencies. The best area for the DOD to implement crowdsourcing focuses on design, forecasting, and software. Lessons learned allow for better use of crowdsourcing in new modernization goals and efforts in reducing costs and fielding equipment.http://archive.org/details/crowdsourcingdef1094556900Major, United States ArmyApproved for public release; distribution is unlimited

Designing Organizations for Dynamic Capabilities

How can organizations put dynamic capabilities into practice? We focus on the power of organizational design, showing how managers can harness new organizational forms to build a capacity for sensing, shaping and seizing opportunities. Fast-moving environments favor open organization and self-organizing processes that quickly convert individual capabilities into actionable collective intellect. We argue that self-organizing processes do not organize themselves but require managers to design and execute them. We examine new design principles – such as polyarchy, social proofs, and new forms of open organization – that allow organizations to build dynamic capabilities for sustained innovation in dynamic environments

Mining and quality assessment of mashup model patterns with the crowd: A feasibility study

Pattern mining, that is, the automated discovery of patterns from data, is a mathematically complex and computationally demanding problem that is generally not manageable by humans. In this article, we focus on small datasets and study whether it is possible to mine patterns with the help of the crowd by means of a set of controlled experiments on a common crowdsourcing platform. We specifically concentrate on mining model patterns from a dataset of real mashup models taken from Yahoo! Pipes and cover the entire pattern mining process, including pattern identification and quality assessment. The results of our experiments show that a sensible design of crowdsourcing tasks indeed may enable the crowd to identify patterns from small datasets (40 models). The results, however, also show that the design of tasks for the assessment of the quality of patterns to decide which patterns to retain for further processing and use is much harder (our experiments fail to elicit assessments from the crowd that are similar to those by an expert). The problem is relevant in general to model-driven development (e.g., UML, business processes, scientific workflows), in that reusable model patterns encode valuable modeling and domain knowledge, such as best practices, organizational conventions, or technical choices, that modelers can benefit from when designing their own models