Aligning Crowdworker Perspectives and Feedback Outcomes in Crowd-Feedback System Design

Leveraging crowdsourcing in software development has received growing attention in research and practice. Crowd feedback offers a scalable and flexible way to evaluate software design solutions and the potential of crowd-feedback systems has been demonstrated in different contexts by existing research studies. However, previous research lacks a deep understanding of the effects of individual design features of crowd-feedback systems on feedback quality and quantity. Additionally, existing studies primarily focused on understanding the requirements of feedback requesters but have not fully explored the qualitative perspectives of crowd-based feedback providers. In this paper, we address these research gaps with two research studies. In study 1, we conducted a feature analysis (N=10) and concluded that from a user perspective, a crowd-feedback system should have five core features (scenario, speech-to-text, markers, categories, and star rating). In the second study, we analyzed the effects of the design features on crowdworkers’ perceptions and feedback outcomes (N=210). We learned that offering feedback providers scenarios as the context of use is perceived as most important. Regarding the resulting feedback quality, we discovered that more features are not always better as overwhelming feedback providers might decrease feedback quality. Offering feedback providers categories as inspiration can increase the feedback quantity. With our work, we contribute to research on crowd-feedback systems by aligning crowdworker perspectives and feedback outcomes and thereby making the software evaluation not only more scalable but also more human-centered

Towards eliciting feedback for artworks on public displays

Abstract The internet and digital technologies have had a strong influence on how art is created, distributed and perceived. Museums and art galleries, however, are still predominantly places of passive consumption of art. In our project, we explore new forms of communication and feedback between artists and their audience, mediated by public displays. In this work-in-progress paper, we present a situated feedback system for giving feedback on artworks in a public setting. We present a preliminary evaluation of the system with artists and potential audience on their reactions to the system and eight different types of feedback