A data-driven game theoretic strategy for developers in software crowdsourcing: a case study

Crowdsourcing has the advantages of being cost-effective and saving time, which is a typical embodiment of collective wisdom and community workers’ collaborative development. However, this development paradigm of software crowdsourcing has not been used widely. A very important reason is that requesters have limited knowledge about crowd workers’ professional skills and qualities. Another reason is that the crowd workers in the competition cannot get the appropriate reward, which affects their motivation. To solve this problem, this paper proposes a method of maximizing reward based on the crowdsourcing ability of workers, they can choose tasks according to their own abilities to obtain appropriate bonuses. Our method includes two steps: Firstly, it puts forward a method to evaluate the crowd workers’ ability, then it analyzes the intensity of competition for tasks at Topcoder.com—an open community crowdsourcing platform—on the basis of the workers’ crowdsourcing ability; secondly, it follows dynamic programming ideas and builds game models under complete information in different cases, offering a strategy of reward maximization for workers by solving a mixed-strategy Nash equilibrium. This paper employs crowdsourcing data from Topcoder.com to carry out experiments. The experimental results show that the distribution of workers’ crowdsourcing ability is uneven, and to some extent it can show the activity degree of crowdsourcing tasks. Meanwhile, according to the strategy of reward maximization, a crowd worker can get the theoretically maximum reward

Making Task Recommendations in Crowdsourcing Contests

Crowdsourcing contests have emerged as an innovative way for firms to solve business problems by acquiring ideas from participants external to the firm. To facilitate such contests a number of crowdsourcing platforms have emerged in recent years. A crowdsourcing platform provides a two-sided marketplace with one set of members (seekers) posting tasks, and another set of members (solvers) working on these tasks and submitting solutions. As crowdsourcing platforms attract more seekers and solvers, the number of tasks that are open at any time can become quite large. Consequently, solvers search only a limited number of tasks before deciding which one(s) to participate in, often examining only those tasks that appear on the first couple of pages of the task listings. This kind of search behavior has potentially detrimental implications for all parties involved: (i) solvers typically end up participating in tasks they are less likely to win relative some other tasks, (ii) seekers receive solutions of poorer quality compared to a situation where solvers are able to find tasks that they are more likely to win, and (iii) when seekers are not satisfied with the outcome, they may decide to leave the platform; therefore, the platform could lose revenues in the short term and market share in the long term. To counteract these concerns, platforms can provide recommendations to solvers in order to reduce their search costs for identifying the most preferable tasks. This research proposes a methodology to develop a system that can recommend tasks to solvers who wish to participate in crowdsourcing contests. A unique aspect of this environment is that it involves competition among solvers. The proposed approach explicitly models the competition that a solver would face in each open task. The approach makes recommendations based on the probability of the solver winning an open task. A multinomial logit model has been developed to estimate these winning probabilities. We have validated our approach using data from a real crowdsourcing platform

Recomendation systems and crowdsourcing: a good wedding for enabling innovation? Results from technology affordances and costraints theory

Recommendation Systems have come a long way since their first appearance in the e-commerce platforms.Since then, evolved Recommendation Systems have been successfully integrated in social networks. Now its time to test their usability and replicate their success in exciting new areas of web -enabled phenomena. One of these is crowdsourcing. Research in the IS field is investigating the need, benefits and challenges of linking the two phenomena. At the moment, empirical works have only highlighted the need to implement these techniques for tasks assignment in crowdsourcing distributed work platforms and the derived benefits for contributors and firms. We review the variety of the tasks that can be crowdsourced through these platforms and theoretically evaluate the efficiency of using RS to recommend a task in creative crowdsourcing platforms. Adopting a Technology Affordances and Constraints Theory, an emerging perspective in the Information Systems (IS) literature to understand technology use and consequences, we anticipate the tensions that this implementation can generate

Behavioral Mechanism Design: Optimal Contests for Simple Agents

Incentives are more likely to elicit desired outcomes when they are designed based on accurate models of agents' strategic behavior. A growing literature, however, suggests that people do not quite behave like standard economic agents in a variety of environments, both online and offline. What consequences might such differences have for the optimal design of mechanisms in these environments? In this paper, we explore this question in the context of optimal contest design for simple agents---agents who strategically reason about whether or not to participate in a system, but not about the input they provide to it. Specifically, consider a contest where $n$ potential contestants with types $(q_i,c_i)$ each choose between participating and producing a submission of quality $q_i$ at cost $c_i$, versus not participating at all, to maximize their utilities. How should a principal distribute a total prize $V$ amongst the $n$ ranks to maximize some increasing function of the qualities of elicited submissions in a contest with such simple agents? We first solve the optimal contest design problem for settings with homogenous participation costs $c_i = c$. Here, the optimal contest is always a simple contest, awarding equal prizes to the top $j^*$ contestants for a suitable choice of $j^*$. (In comparable models with strategic effort choices, the optimal contest is either a winner-take-all contest or awards possibly unequal prizes, depending on the curvature of agents' effort cost functions.) We next address the general case with heterogeneous costs where agents' types are inherently two-dimensional, significantly complicating equilibrium analysis. Our main result here is that the winner-take-all contest is a 3-approximation of the optimal contest when the principal's objective is to maximize the quality of the best elicited contribution.Comment: This is the full version of a paper in the ACM Conference on Economics and Computation (ACM-EC), 201

Understanding Crowdsourcing Contest Fitness Strategic Decision Factors and Performance: An Expectation-Confirmation Theory Perspective

Contest-based intermediary crowdsourcing represents a powerful new business model for generating ideas or solutions by engaging the crowd through an online competition. Prior research has examined motivating factors such as increased monetary reward or demotivating factors such as project requirement ambiguity. However, problematic issues related to crowd contest fitness have received little attention, particularly with regard to crowd strategic decision-making and contest outcomes that are critical for success of crowdsourcing platforms as well as implementation of crowdsourcing models in organizations. Using Expectation-Confirmation Theory (ECT), we take a different approach that focuses on contest level outcomes by developing a model to explain contest duration and performance. We postulate these contest outcomes are a function of managing crowdsourcing participant contest-fitness expectations and disconfirmation, particularly during the bidding process. Our empirical results show that contest fitness expectations and disconfirmation have an overall positive effect on contest performance. This study contributes to theory by demonstrating the adaptability of ECT literature to the online crowdsourcing domain at the level of the project contest. For practice, important insights regarding strategic decision making and understanding how crowd contest-fitness are observed for enhancing outcomes related to platform viability and successful organizational implementation

Heterogeneity Based Solvers’ Segmentation In Crowdsourcing

Multiple facets of factors were examined to be drivers for crowdsourcing intention. However, there is limited research that has studied whether this factors-intention link is uniform for all solvers or not in detail. In fact, the present studies have identified three different segments that are internally consistent and stable. The comparison between the results of two different solutions, single-class and prediction-oriented-segmentation, confirms the existence of unobserved solver segments. The three established segments are “Self-leading solvers”, “External-driving solvers” and “Dual-driving solvers”. These results point the way for factors-based segmentation in intention initiatives and reflect the importance of a multidimensional conceptualization of factors, comprising motivation, perceived sponsor’s and platform’s support components. The paper expands and deepens the application of the heterogeneity theory in the study of crowdsourcing usage behavior and offers implications for organizers to recognize the solvers more clearly and get directions for more valid strategies

Towards A Better Design of Online Marketplaces

Online markets are staggering in volume and variety. These online marketplaces are transforming lifestyles, expanding the boundaries of conventional businesses, and reshaping labor force structures. To fully realize their potential, online marketplaces must be designed carefully. However, this is a significant challenge. This dissertation studies individual behavior and interactions in online marketplaces, and examines how to enhance efficiency and outcomes of these online marketplaces by providing actionable operational policy recommendations. An important question in the context of open-ended innovative service marketplaces is how to manage information when specifying design problems to achieve better outcomes. Chapter 1 investigates this problem in the context of online crowdsourcing contests where innovation seekers source innovative products (designs) from a crowd of competing solvers (designers). We propose and empirically test a theoretical model featuring different types of information in the problem specification (conceptual objectives, execution guidelines), and the corresponding impact on design processes and submission qualities. We find that, to maximize the best solution quality in crowdsourced design problems, seekers should always provide more execution guidelines, and only a moderate number of conceptual objectives. Building on the same research setting, Chapter 2 looks into another important yet challenging problem---how the innovation seeker should provide interim performance feedback to the solvers in online service marketplaces where seekers and solvers can interact dynamically. In particular, we study whether and when the seeker should provide such interim performance feedback. We empirically examine these research questions using a dataset from a crowdsourcing platform. We develop and estimate a dynamic structural model to understand contestants’ behavior, compare alternative feedback policies using counter-factual simulations, and find providing feedback throughout the contest may not be optimal. The late feedback policy, i.e., providing feedback only in the second half of the contest, leads to a better overall contest outcome. Moving to a wider application, Chapter 3 leverages consumer clickstream information in e-commerce marketplaces to help market organizers improve demand estimation and pricing decisions. These decisions can be challenging, as e-commerce marketplaces offer an astonishing variety of product choices and face extremely diversified consumer decision journeys. We provide a novel solution to these challenges by combining econometric and machine learning (Graphical Lasso) approaches, leveraging customer clickstream information to learn the product correlation network, and creating high-dimensional choice models that easily scale and allow for flexible substitution patterns. Our model offers better in- and out-of-sample demand forecasts and enhanced pricing recommendations in various synthetic datasets and in a real-world empirical setting.PHDBusiness AdministrationUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/163283/1/jiangzh_1.pd

The Role of Problem Specification in Crowdsourcing Contests for Design Problems: A Theoretical and Empirical Analysis

This paper studies the role of seekers' problem specification in crowdsourcing contests for design problems. Platforms hosting design contests offer detailed guidance for seekers to specify their design problems when launching a contest. Yet, problem specification in such crowdsourcing contests is something the theoretical and empirical literature has largely overlooked. We aim to fill this gap by offering an empirically-validated model to generate insights for the provision of information at contest launch. We develop a game-theoretic model featuring different types of information (categorized as “conceptual objectives” or “execution guidelines”) conveyed in problem specifications, and assess their impact on design processes. Real-world data is used to empirically test hypotheses generated from the model, and a quasi-natural experiment provides further empirical evidence for our predictions and recommendations. We show theoretically and verify empirically that, with more conceptual objectives disclosed in the problem specification, the number of participants in a contest decreases, but the trial effort provision by each participant does not change; with more execution guidelines disclosed in the problem specification, the trial effort provision by each participant increases, but the number of participants in a contest does not change. With that knowledge, we are able to formulate seekers' optimal decisions on problem specifications, and find that, to maximize the expected quality of the best solution to crowdsourced design problems, seekers should always provide more execution guidelines, and only a moderate number of conceptual objectives.https://deepblue.lib.umich.edu/bitstream/2027.42/146143/1/1388_Jiang.pd