Prototype tasks: Improving crowdsourcing results through rapid, iterative task design

Low-quality results have been a long-standing problem on microtask crowdsourcing platforms, driving away requesters and justifying low wages for workers. To date, workers have been blamed for low-quality results: they are said to make as little effort as possible, do not pay attention to detail, and lack expertise. In this paper, we hypothesize that requesters may also be responsible for low-quality work: they launch unclear task designs that confuse even earnest workers, under-specify edge cases, and neglect to include examples. We introduce prototype tasks, a crowdsourcing strategy requiring all new task designs to launch a small number of sample tasks. Workers attempt these tasks and leave feedback, enabling the requester to iterate on the design before publishing it. We report a field experiment in which tasks that underwent prototype task iteration produced higher-quality work results than the original task designs. With this research, we suggest that a simple and rapid iteration cycle can improve crowd work, and we provide empirical evidence that requester “quality” directly impacts result quality

The Daemo crowdsourcing marketplace

The success of crowdsourcing markets is dependent on a strong foundation of trust between workers and requesters. In current marketplaces, workers and requesters are often unable to trust each other’s quality, and their mental models of tasks are misaligned due to ambiguous instructions or confusing edge cases. This breakdown of trust typically arises from (1) flawed reputation systems which do not accurately reflect worker and requester quality, and from (2) poorly designed tasks. In this demo, we present how Boomerang and Prototype Tasks, the fundamental building blocks of the Daemo crowdsourcing marketplace, help restore trust between workers and requesters. Daemo’s Boomerang reputation system incentivizes alignment between opinion and ratings by determining the likelihood that workers and requesters will work together in the future based on how they rate each other. Daemo’s Prototype tasks require that new tasks go through a feedback iteration phase with a small number of workers so that requesters can revise their instructions and task designs before launch

Crowd guilds: Worker-led reputation and feedback on crowdsourcing platforms

Crowd workers are distributed and decentralized. While decentralization is designed to utilize independent judgment to promote high-quality results, it paradoxically undercuts behaviors and institutions that are critical to high-quality work. Reputation is one central example: crowdsourcing systems depend on reputation scores from decentralized workers and requesters, but these scores are notoriously inflated and uninformative. In this paper, we draw inspiration from historical worker guilds (e.g., in the silk trade) to design and implement crowd guilds: centralized groups of crowd workers who collectively certify each other’s quality through double-blind peer assessment. A two-week field experiment compared crowd guilds to a traditional decentralized crowd work model. Crowd guilds produced reputation signals more strongly correlated with ground-truth worker quality than signals available on current crowd working platforms, and more accurate than in the traditional model

Boomerang: Rebounding the consequences of reputation feedback on crowdsourcing platforms

Paid crowdsourcing platforms suffer from low-quality workand unfair rejections, but paradoxically, most workers and requesters have high reputation scores. These inflated scores, which make high-quality work and workers difficult to find,stem from social pressure to avoid giving negative feedback. We introduce Boomerang, a reputation system for crowdsourcing that elicits more accurate feedback by rebounding the consequences of feedback directly back onto the person who gave it. With Boomerang, requesters find that their highly rated workers gain earliest access to their future tasks, and workers find tasks from their highly-rated requesters at the top of their task feed. Field experiments verify that Boomerang causes both workers and requesters to provide feedback that is more closely aligned with their private opinions. Inspired by a game-theoretic notion of incentive-compatibility, Boomerang opens opportunities for interaction design to incentivize honest reporting over strategic dishonesty

The clinical utility of a suite of continence assessment tools for use in residential aged care facilities

According to the Institute of Medicine of the National Academies, 80% of direct care in nursing homes in the USA is provided by workforce that has least formal education about patient care (1). This situation is echoed in Australian residential aged care facilities where the day-to-day management is largely provided by unregulated workers (i.e. nursing assistants, personal carers and nursing aides) and is overseen by registered nurses. Some facilities additionally have access to expert advice from continence nurse advisors. In order to assist the residential aged care workforce to provide continence care that is evidence-based, a team of researchers developed and trialled a suite of continence assessment tools that were mainly targeted to unregulated workers. This paper presents information on the development of the tools (Stage 1) and on their evaluation (Stage 2).<br /

Teaching teamwork: an evaluation of an interprofessional training ward placement for health care students

Julia Morphet,1 Kerry Hood,2 Robyn Cant,2 Julie Baulch,3 Alana Gilbee,3 Kate Sandry4 1School of Nursing and Midwifery, Monash University, Frankston, Victoria, Australia; 2School of Nursing and Midwifery, Monash University, Clayton, Victoria, Australia; 3Southern Clinical School, Monash University, Monash Health, Clayton, Victoria, Australia; 4Dandenong Emergency Department, Monash Health, David St, Dandenong, Victoria, Australia Abstract: The establishment of interprofessional teamwork training in the preprofessional health care curriculum is a major challenge for teaching faculties. Interprofessional clinical placements offer an opportunity for teamwork education, as students in various professions can work and learn together. In this sequential, mixed-method study, focus group and survey techniques were used to evaluate students&#39; educational experiences after 2-week ward-based interprofessional clinical placements. Forty-five senior nursing, medicine, and other health care students cared for patients in hospital wards under professional supervision, with nursing-medicine student &quot;teams&quot; leading care. Thirty-six students attended nine exit focus groups. Five central themes that emerged about training were student autonomy and workload, understanding of other professional roles, communication and shared knowledge, interprofessional teamwork/collaboration, and the &quot;inner circle&quot;, or being part of the unit team. The learning environment was described as positive. In a postplacement satisfaction survey (n=38), students likewise rated the educational experience highly. In practicing teamwork and collaboration, students were able to rehearse their future professional role. We suggest that interprofessional clinical placements be regarded as an essential learning experience for senior preprofessional students. More work is needed to fully understand the effect of this interactive program on students&#39; clinical learning and preparation for practice. Keywords: education, medical, undergraduate, experiential learning, interprofessional learning, nursing, teamwor