RATING SCALES FOR COLLECTIVE INTELLIGENCE IN INNOVATION COMMUNITIES:WHY QUICK AND EASY DECISION MAKING DOES NOT GET IT RIGHT

The increasing popularity of open innovation approaches has lead to the rise of various innovation platforms on the Internet which might contain 10.000s user-generated ideas. However, a company’s absorptive capacity is limited regarding such an amount of ideas so that there is a strong need for mechanism to identify the best ideas. Extending previous decision management research we focus on analyzing effective idea rating and selection mechanisms in online innovation communities and underlying explanations. Using a multi-method approach our research comprises a web-based rating experiment with 313 participants evaluating 24 ideas from a real-world innovation community, data from a survey measuring rating satisfaction of participants, and idea ratings from an independent expert jury. Our findings show that, despite its popular use in online innovation communities, simple rating mechanisms such as thumbs up/down rating or 5-star rating do not produce valid idea rankings and are significantly outperformed by the multi-attribute scale

When Life Gives You Lemons: How rating scales affect user activity and frustration in collaborative evaluation processes

Initiators of open innovation processes involving customers or employees often face vast amounts of idea proposals. These proposals vary greatly in terms of quality, which is why organizers often engage the users themselves in the evaluation process. Building on the concept of information overload, we evaluate the effects of three distinct rating scales on users’ activity and frustration measures. On the basis of an open innovation campaign for employees of a public-private institution in Germany, we systematically compare the novel “bag of lemons” method with conventional Likert scales and up-down-voting schemes. Our results demonstrate that the “bag of lemons”-approach yields higher levels of user activity, but is also perceived as significantly more frustrating. We find this effect to be fully mediated by perceived information overload, which points to potential avenues for the design of stimulating yet tolerably complex Information Systems for open innovation and rating techniques

NOT ALL TASKS ARE ALIKE: EXPLORING THE EFFECT OF TASK REPRESENTATION ON USER ENGAGEMENT IN CROWD-BASED IDEA EVALUATION

Crowdsourcing has experienced increasing popularity in recent years. While performance-based issues, such as the quantity or quality of output produced by the crowd, have been in the focus of research, users’ experience, which unfolds through interaction with the crowdsourcing platform and ultimately creates engagement, has been largely neglected. However, user engagement does not only determine the scope of effort users put into the crowdsourcing task, but is considered a determinant for future participation. This paper focusses on the role of task representation–manifested in mechanisms for crowd-based idea evaluation–as potential stimuli for user engagement. Therefore, we conduct a web-based experiment with 198 participants to investigate how different task representations translate into differences in users’ experience and their engagement. In particular, we analyze two distinctive task representations: sequential judgement tasks in form of multi-criteria rating scales and simultaneous choice tasks in the form of enterprise crowdfunding. We find differences in task representation to influence user engagement while mediated by a user’s perceived cognitive load. Moreover, our findings indicate that user engagement is determined by a user’s perceived meaningfulness of a task. These results enhance our understanding of user engagement in crowdsourcing and contribute to theory building in this emerging field

PEER RATINGS AND ASSESSMENT QUALITY IN CROWD-BASED INNOVATION PROCESSES

Social networks – whether public or in enterprises – regularly ask users to rate their peers’ content using different voting techniques. When employed in innovation challenges, these rating procedures are part of an open, interactive, and continuous engagement among customers, employees, or citizens. In this regard, assessment accuracy (i.e., correctly identifying good and bad ideas) in crowdsourced eval-uation processes may be influenced by the display of peer ratings. While it could sometimes be useful for users to follow their peers, it is not entirely clear under which circumstances this actually holds true. Thus, in this research-in-progress article, we propose a study design to systematically investigate the effect of peer ratings on assessment accuracy in crowdsourced idea evaluation processes. Based on the elaboration likelihood model and social psychology, we develop a research model that incorporates the mediating factors extraversion, locus of control, as well as peer rating quality (i.e., the ratings’ corre-lation with the evaluated content’s actual quality). We suggest that the availability of peer ratings de-creases assessment accuracy and that rating quality, extraversion, as well as an internal locus of control mitigate this effect

Assessing regional digital competence: Digital futures and strategic planning implications

Understanding strategic decisions aimed at addressing regional economic issues is of increasing interest among scholars and policy makers today. Thus, studies that proffer effective strategies to address digital futures concerns from social and policy perspectives are timely. In light of this, this research uses strengths, weaknesses, opportunities and threats (SWOT) analysis tool to frame a regional strategy for digital futures to enhance place-specific digital connectivity and socio-economic progress. Focus group discussions and a structured questionnaire were conducted to examine a SWOT for a digital economy strategy in the Southern Downs Region in Queensland, Australia. The findings show that while the proposed regional strategies for digital futures are susceptible to internal and external forces, strategic planning makes them manageable. The study’s findings also reveal that adaptive strategic planning can help regulate the effects of internal and external factors that shape individual and organisational responses to digital transformation, and that these factors promote regional competitiveness

Using Crowdfunding For Start-Up Evaluation: How Task Representation Influences Predition Accuracy of The Crowd

The paper at hand examines if the crowd can offer valuable support in evaluating start-ups. In doing so, we plan to conduct an experiment 1.) to test if the crowd is capable to support experts in evaluating start-ups 2.) to examine how differences in task-representation (i.e. rating scales vs. a crowdfunding mechanism) influences cognitive processing of the crowd and 3.) to examine how types of cognitive processing (i.e. system 1 thinking vs. system 2 thinking) relate to prediction accuracy of the crowd. To this end, we plan to introduce crowdfunding as a new evaluation mechanism to support the crowd in coming up with more accurate predictions of start-up value. Our theoretical contribution is twofold. First, we aim to show if the crowd can be used to support Venture capitalists in evaluating start-ups, in the sense that their evaluations agree with expert evaluations. Second, we plan to con-tribute to a better understanding about how the design of evaluation mechanisms influences peoples cognitive processing and the crowds ability to predict start-up value

Managing relational capital for the sustainability of the energy sector in the social media

Various management models have been proposed for intangible business assets in this new digital era. However, these models do not consider the relationships between intangible assets in business management, or their effect. This work has two main objectives: first, to show the effect of intangi-ble assets as expressed in digital media related to energy brands; second, to demonstrate the rela-tionships between the emotions, experiences and attitudes of the audiences. To do so, a novel model of intangibles is proposed and applied to the energy sector using IBEX 35 data. In this sce-nario, we determine that users' experiences extracted from digital environments have significant relationships with one of the most important intangibles in the energy business, namely emotion.Vários modelos de gestão foram propostos para ativos de negócios intangíveis nesta nova era digital. No entanto, esses modelos não consideram as relações entre ativos intangíveis na gestão ou os seus efeitos. Este estudo tem dois objetivos principais: primeiro, mostrar o efeito dos ativos intangíveis expressos nos meios digitais relacionados com as marcas associadas ao setor económico da energia; segundo, demonstrar as relações entre as emoções, experiências e atitudes do público. Para esse efeito, é proposto um novo modelo de gestão de intangíveis aplicado ao setor de energia utilizando dados do IBEX 35. Nesse cenário, concluímos que as experiências dos utilizadores extraídas de ambientes digitais têm relacionamentos significativos com um dos intangíveis mais importantes no negócio de energia, ou seja, a emoção.info:eu-repo/semantics/publishedVersio

Human experience in the natural and built environment : implications for research policy and practice

22nd IAPS conference. Edited book of abstracts. 427 pp. University of Strathclyde, Sheffield and West of Scotland Publication. ISBN: 978-0-94-764988-3