1,097 research outputs found
Human-agent collectives
We live in a world where a host of computer systems, distributed throughout our physical and information environments, are increasingly implicated in our everyday actions. Computer technologies impact all aspects of our lives and our relationship with the digital has fundamentally altered as computers have moved out of the workplace and away from the desktop. Networked computers, tablets, phones and personal devices are now commonplace, as are an increasingly diverse set of digital devices built into the world around us. Data and information is generated at unprecedented speeds and volumes from an increasingly diverse range of sources. It is then combined in unforeseen ways, limited only by human imagination. People’s activities and collaborations are becoming ever more dependent upon and intertwined with this ubiquitous information substrate. As these trends continue apace, it is becoming apparent that many endeavours involve the symbiotic interleaving of humans and computers. Moreover, the emergence of these close-knit partnerships is inducing profound change. Rather than issuing instructions to passive machines that wait until they are asked before doing anything, we will work in tandem with highly inter-connected computational components that act autonomously and intelligently (aka agents). As a consequence, greater attention needs to be given to the balance of control between people and machines. In many situations, humans will be in charge and agents will predominantly act in a supporting role. In other cases, however, the agents will be in control and humans will play the supporting role. We term this emerging class of systems human-agent collectives (HACs) to reflect the close partnership and the flexible social interactions between the humans and the computers. As well as exhibiting increased autonomy, such systems will be inherently open and social. This means the participants will need to continually and flexibly establish and manage a range of social relationships. Thus, depending on the task at hand, different constellations of people, resources, and information will need to come together, operate in a coordinated fashion, and then disband. The openness and presence of many distinct stakeholders means participation will be motivated by a broad range of incentives rather than diktat. This article outlines the key research challenges involved in developing a comprehensive understanding of HACs. To illuminate this agenda, a nascent application in the domain of disaster response is presented
Artificial Collective Intelligence Engineering: a Survey of Concepts and Perspectives
Collectiveness is an important property of many systems--both natural and
artificial. By exploiting a large number of individuals, it is often possible
to produce effects that go far beyond the capabilities of the smartest
individuals, or even to produce intelligent collective behaviour out of
not-so-intelligent individuals. Indeed, collective intelligence, namely the
capability of a group to act collectively in a seemingly intelligent way, is
increasingly often a design goal of engineered computational systems--motivated
by recent techno-scientific trends like the Internet of Things, swarm robotics,
and crowd computing, just to name a few. For several years, the collective
intelligence observed in natural and artificial systems has served as a source
of inspiration for engineering ideas, models, and mechanisms. Today, artificial
and computational collective intelligence are recognised research topics,
spanning various techniques, kinds of target systems, and application domains.
However, there is still a lot of fragmentation in the research panorama of the
topic within computer science, and the verticality of most communities and
contributions makes it difficult to extract the core underlying ideas and
frames of reference. The challenge is to identify, place in a common structure,
and ultimately connect the different areas and methods addressing intelligent
collectives. To address this gap, this paper considers a set of broad scoping
questions providing a map of collective intelligence research, mostly by the
point of view of computer scientists and engineers. Accordingly, it covers
preliminary notions, fundamental concepts, and the main research perspectives,
identifying opportunities and challenges for researchers on artificial and
computational collective intelligence engineering.Comment: This is the author's final version of the article, accepted for
publication in the Artificial Life journal. Data: 34 pages, 2 figure
Crowd guilds: Worker-led reputation and feedback on crowdsourcing platforms
Crowd workers are distributed and decentralized. While decentralization is designed to utilize independent judgment to promote high-quality results, it paradoxically undercuts behaviors and institutions that are critical to high-quality work. Reputation is one central example: crowdsourcing systems depend on reputation scores from decentralized workers and requesters, but these scores are notoriously inflated and uninformative. In this paper, we draw inspiration from historical worker guilds (e.g., in the silk trade) to design and implement crowd guilds: centralized groups of crowd workers who collectively certify each other’s quality through double-blind peer assessment. A two-week field experiment compared crowd guilds to a traditional decentralized crowd work model. Crowd guilds produced reputation signals more strongly correlated with ground-truth worker quality than signals available on current crowd working platforms, and more accurate than in the traditional model
A Glimpse Far into the Future: Understanding Long-term Crowd Worker Quality
Microtask crowdsourcing is increasingly critical to the creation of extremely
large datasets. As a result, crowd workers spend weeks or months repeating the
exact same tasks, making it necessary to understand their behavior over these
long periods of time. We utilize three large, longitudinal datasets of nine
million annotations collected from Amazon Mechanical Turk to examine claims
that workers fatigue or satisfice over these long periods, producing lower
quality work. We find that, contrary to these claims, workers are extremely
stable in their quality over the entire period. To understand whether workers
set their quality based on the task's requirements for acceptance, we then
perform an experiment where we vary the required quality for a large
crowdsourcing task. Workers did not adjust their quality based on the
acceptance threshold: workers who were above the threshold continued working at
their usual quality level, and workers below the threshold self-selected
themselves out of the task. Capitalizing on this consistency, we demonstrate
that it is possible to predict workers' long-term quality using just a glimpse
of their quality on the first five tasks.Comment: 10 pages, 11 figures, accepted CSCW 201
Generating creative ideas through crowds: An experimental study of combination
The crowd is emerging as a new source of innovation, and here a new way of organizing the crowd to produce new ideas is discussed: an idea generation system using combination in which participants synthesize new designs from the efforts of their peers. A crowd generates designs; then another crowd combines the designs of the previous crowd. In an experiment with 540 participants, the combined designs are compared to the initial designs, and to a control condition in which fresh idea generation rather than combination is used. The results show that designs become more creative in later generations of the combination system, and the combination produces more creative ideas than the fresh idea generation. The model of crowdsourced idea generation discussed here may be used to instantiate systems that can be applied to a wide range of design problems. The work has pragmatic implications, and also theoretical implications: new forms of coordination are now possible, and, using the crowd, it is possible to build and test existing and emerging theories of coordination and design
A methodology to take account of diversity in collective adaptive system
Collective Adaptive Systems (CASs) are comprised of a heterogeneous set of components often developed in a distributed manner. Their users are diverse with respect to their profiles, preferences, interests and goals, and hence, have different requirements. We propose a typology for the diversity of these components, users, and their requirements. We then present a methodology which provides steps to integrate features that record diversity to support accountability. The foundation of accountability is provided by provenance data, and a CAS vocabulary, these knowledge representation languages provide the core vocabulary that can be exploited by agents and services
Microtasking models and managerial challenges
Proceedings of the CSCW 2016 - Workshop: Toward a Typology of Participation in CrowdworkThis paper offers a brief overview of basic microtasking models and connects its developments with emerging management challenges that will need to be addressed in order to fully harness the capacities and skills of the crowd in different domain areas
- …