When Life Gives You Lemons: How rating scales affect user activity and frustration in collaborative evaluation processes

Initiators of open innovation processes involving customers or employees often face vast amounts of idea proposals. These proposals vary greatly in terms of quality, which is why organizers often engage the users themselves in the evaluation process. Building on the concept of information overload, we evaluate the effects of three distinct rating scales on users’ activity and frustration measures. On the basis of an open innovation campaign for employees of a public-private institution in Germany, we systematically compare the novel “bag of lemons” method with conventional Likert scales and up-down-voting schemes. Our results demonstrate that the “bag of lemons”-approach yields higher levels of user activity, but is also perceived as significantly more frustrating. We find this effect to be fully mediated by perceived information overload, which points to potential avenues for the design of stimulating yet tolerably complex Information Systems for open innovation and rating techniques

Ideate. Collaborate. Repeat. A Research Agenda for Idea Generation, Collaboration and Evaluation in Open Innovation

Open innovation has been and remains to be a rapidly changing field of research in Information Systems and various other disciplines. With the rise of professional open innovation platforms and the emergence of crowdsourcing as well as employee-driven innovation, studies on the front-end of open innovation – namely idea generation, collaboration and evaluation – are facing new challenges. In this structured literature review, we analyze a large body of prior research in order to derive a framework, which is able to classify and reflect the lively debate on open innovation. In addition, we identify important implications for practitioners with advise on the design of open innovation systems. Moreover, our study identifies several promising areas for future research

Organizational Online Participation

As today’s employees demand higher degrees of involvement in terms of how, when, and where they work, open innovation and (internal) crowdsourcing are being widely adopted. Despite recent efforts by many organizations to implement such systems in order to increase the possibilities for organizational participation, studies have only narrowly explored how their design affects employee opinions and communication as well as how organizational culture influences usage and adoption. This thesis investigates the conditions, capabilities and components for the design of organizational online participation systems, applying a Design Science Research approach. Following a literature review on idea generation, collaboration and evaluation in open innovation processes, we outline success factors for open innovation systems. We validate our success factors in practice by conducting semi-structured interviews with 20 experts from mid- and large-cap private and public organizations in Germany. Moreover, we derive three key challenges that guide our subsequent studies. First, we investigate the “Bag of Lemons” approach, a novel rating technique, and compare it to the standard techniques Likert scales and up- and down-voting. Our study with 141 participants in an open innovation engagement at a public-private research organization finds that BOL is perceived as more frustrating than the other two rating techniques, which is partly mediated by the significantly increased information overload. Second, we turn to anonymity in two distinct studies. We analyze the effect of anonymity, as compared to identifiability of user profiles, on communication persuasiveness – operationalized as actual opinion change – in a two-staged online experimental survey with 377 participants. We find anonymity to be a double-edged sword as it decreases perceived social presence, which in turn affects both user involvement as well as perceived user credibility. Thereafter, we investigate the design of a feature for optional anonymous contributions and its effect on participation and the choice of language in an internal crowdsourcing platform. Our analysis of an implementation and five-month test at a public organization with more than 110 employees shows the effectiveness of our “opt-in anonymity” feature as we elicit participation from otherwise reticent employees and no disinhibited language. Third, we analyze the design of an internal crowdsourcing system at this public organization in more detail, focusing on the influence of its organizational culture on usage and acceptance. We assert an IT-culture-conflict, as the organizational values do not match the open and communal approach transposed by the crowdsourcing system. We suggest that organizational online participation is a promising tool to enhance employee involvement, driving innovations and enabling organizational transformation