Modeling crowdsourcing as collective problem solving

Crowdsourcing is a process of accumulating the ideas, thoughts or information from many independent participants, with aim to find the best solution for a given challenge. Modern information technologies allow for massive number of subjects to be involved in a more or less spontaneous way. Still, the full potentials of crowdsourcing are yet to be reached. We introduce a modeling framework through which we study the effectiveness of crowdsourcing in relation to the level of collectivism in facing the problem. Our findings reveal an intricate relationship between the number of participants and the difficulty of the problem, indicating the optimal size of the crowdsourced group. We discuss our results in the context of modern utilization of crowdsourcing.Comment: 19 pages, 3 figure

Humans best judge how much to cooperate when facing hard problems in large groups

We report the results of a game-theoretic experiment with human players who solve the problems of increasing complexity by cooperating in groups of increasing size. Our experimental environment is set up to make it complicated for players to use rational calculation for making the cooperative decisions. This environment is directly translated into a computer simulation, from which we extract the collaboration strategy that leads to the maximal attainable score. Based on this, we measure the error that players make when estimating the benefits of collaboration, and find that humans massively underestimate these benefits when facing easy problems or working alone or in small groups. In contrast, when confronting hard problems or collaborating in large groups, humans accurately judge the best level of collaboration and easily achieve the maximal score. Our findings are independent on groups' composition and players' personal traits. We interpret them as varying degrees of usefulness of social heuristics, which seems to depend on the size of the involved group and the complexity of the situation.Comment: 18 pages, 1 figure. In press for Scientific Report

Evolution of Cooperation through Power Law Distributed Conflicts

At an individual level, cooperation can be seen as a behaviour that uses personal resource to support others or the groups which one belongs to. In a conflict between two individuals, a selfish person gains an advantage over a cooperative opponent, while in a group-group conflict the group with more cooperators wins. In this work, we develop a population model with continual conflicts at various scales and show cooperation can be sustained even when interpersonal conflicts dominate, as long as the conflict size follows a power law. The power law assumption has been met in several observations from real-world conflicts. Specifically if the population is structured on a scale-free network, both the power law distribution of conflicts and the survival of cooperation can be naturally induced without assuming a homogeneous population or frequent relocation of members. On the scale-free network, even when most people become selfish from continual person-person conflicts, people on the hubs tend to remain unselfish and play a role as ???repositories??? of cooperation.clos

Simulating the Cost of Cooperation: A Recipe for Collaborative Problem-Solving

Collective problem-solving and decision-making, along with other forms of collaboration online, are central phenomena within ICT. There had been several attempts to create a system able to go beyond the passive accumulation of data. However, those systems often neglect important variables such as group size, the difficulty of the tasks, the tendency to cooperate, and the presence of selfish individuals (free riders). Given the complex relations among those variables, numerical simulations could be the ideal tool to explore such relationships. We take into account the cost of cooperation in collaborative problem solving by employing several simulated scenarios. The role of two parameters was explored: the capacity, the group’s capability to solve increasingly challenging tasks coupled with the collective knowledge of a group, and the payoff, an individual’s own benefit in terms of new knowledge acquired. The final cooperation rate is only affected by the cost of cooperation in the case of simple tasks and small communities. In contrast, the fitness of the community, the difficulty of the task, and the groups sizes interact in a non-trivial way, hence shedding some light on how to improve crowdsourcing when the cost of cooperation is high

Crowdsourced Sampling of a Composite Random Variable: Analysis, Simulation, and Experimental Test

A composite random variable is a product (or sum of products) of statistically distributed quantities. Such a variable can represent the solution to a multi-factor quantitative problem submitted to a large, diverse, independent, anonymous group of non-expert respondents (the “crowd”). The objective of this research is to examine the statistical distribution of solutions from a large crowd to a quantitative problem involving image analysis and object counting. Theoretical analysis by the author, covering a range of conditions and types of factor variables, predicts that composite random variables are distributed log-normally to an excellent approximation. If the factors in a problem are themselves distributed log-normally, then their product is rigorously log-normal. A crowdsourcing experiment devised by the author and implemented with the assistance of a BBC (British Broadcasting Corporation) television show, yielded a sample of approximately 2000 responses consistent with a log-normal distribution. The sample mean was within ~12% of the true count. However, a Monte Carlo simulation (MCS) of the experiment, employing either normal or log-normal random variables as factors to model the processes by which a crowd of 1 million might arrive at their estimates, resulted in a visually perfect log-normal distribution with a mean response within ~5% of the true count. The results of this research suggest that a well-modeled MCS, by simulating a sample of responses from a large, rational, and incentivized crowd, can provide a more accurate solution to a quantitative problem than might be attainable by direct sampling of a smaller crowd or an uninformed crowd, irrespective of size, that guesses randomly

How to specify healthcare process improvements collaboratively using rapid, remote consensus-building: a framework and a case study of its application.

BackgroundPractical methods for facilitating process improvement are needed to support high quality, safe care. How best to specify (identify and define) process improvements - the changes that need to be made in a healthcare process - remains a key question. Methods for doing so collaboratively, rapidly and remotely offer much potential, but are under-developed. We propose an approach for engaging diverse stakeholders remotely in a consensus-building exercise to help specify improvements in a healthcare process, and we illustrate the approach in a case study.MethodsOrganised in a five-step framework, our proposed approach is informed by a participatory ethos, crowdsourcing and consensus-building methods: (1) define scope and objective of the process improvement; (2) produce a draft or prototype of the proposed process improvement specification; (3) identify participant recruitment strategy; (4) design and conduct a remote consensus-building exercise; (5) produce a final specification of the process improvement in light of learning from the exercise. We tested the approach in a case study that sought to specify process improvements for the management of obstetric emergencies during the COVID-19 pandemic. We used a brief video showing a process for managing a post-partum haemorrhage in women with COVID-19 to elicit recommendations on how the process could be improved. Two Delphi rounds were then conducted to reach consensus.ResultsWe gathered views from 105 participants, with a background in maternity care (n = 36), infection prevention and control (n = 17), or human factors (n = 52). The participants initially generated 818 recommendations for how to improve the process illustrated in the video, which we synthesised into a set of 22 recommendations. The consensus-building exercise yielded a final set of 16 recommendations. These were used to inform the specification of process improvements for managing the obstetric emergency and develop supporting resources, including an updated video.ConclusionsThe proposed methodological approach enabled the expertise and ingenuity of diverse stakeholders to be captured and mobilised to specify process improvements in an area of pressing service need. This approach has the potential to address current challenges in process improvement, but will require further evaluation

Toward Crowdsourcing Translation Post-editing: A Thematic Systematic Review

Crowdsourcing Translation as a Post-Editing Method (CTPE) has emerged as a rapid and inexpensive method for translation and has drawn significant attention in recent years. This qualitative study aims to analyze and synthesize the approaches and aspects underpinning CTPE research and to identify its potential that is yet to be discovered. Through a systematic literature review focused on empirical papers, we examined the limited literature thematically and identified recurring central themes. Our review reveals that the topic of CTPE requires further attention and that its potential benefits are yet to be fully discovered. We discuss the eight core concepts that emerged during our analysis, including the purpose of CTPE, CTPE areas of application, ongoing CTPE processes, platform and crowd characteristics, motivation, CTPE domains, and future perspectives. By highlighting the strengths of CTPE, we conclude that it has the potential to be a highly effective translation method in various domains