Cumulative Innovation & Open Disclosure of Intermediate Results: Evidence from a Policy Experiment in Bioinformatics

Recent calls for greater openness in our private and public innovation systems have particularly urged for more open disclosure and granting of access to intermediate works–early results, algorithms, materials, data and techniques–with the goals of enhancing overall research and development productivity and enhancing cumulative innovation. To make progress towards understanding implications of such policy changes we devised a large-scale field experiment in which 733 subjects were divided into matched independent subgroups to address a bioinformatics problem under either a regime of open disclosure of intermediate results or, alternatively, one of closed secrecy around intermediate solutions. We observe the cumulative innovation process in each regime with fine-grained measures and are able to derive inferences with a series of cross-sectional comparisons. Open disclosures led to lower participation and lower effort but nonetheless led to higher average problem-solving performance by concentrating these lesser efforts on the most performant technical approaches. Closed secrecy produced higher participation and higher effort, while producing less correlated choices of technical approaches that participants pursued, resulting in greater individual and collective experimentation and greater dispersion of performance. We discuss the implications of such changes to the ongoing theory, evidence and policy considerations with regards to cumulative innovation

Marginality and Problem Solving Effectiveness in Broadcast Search

We examine who the winners are in science problem solving contests characterized by open broadcast of problem information, self-selection of external solvers to discrete problems from the laboratories of large R&D intensive companies and blind review of solution submissions. Analyzing a unique dataset of 166 science challenges involving over 12,000 scientists revealed that technical and social marginality, being a source of different perspectives and heuristics, plays an important role in explaining individual success in problem solving. The provision of a winning solution was positively related to increasing distance between the solver’s field of technical expertise and the focal field of the problem. Female solvers – known to be in the “outer circle” of the scientific establishment - performed significantly better than men in developing successful solutions. Our findings contribute to the emerging literature on open and distributed innovation by demonstrating the value of openness, at least narrowly defined by disclosing problems, in removing barriers to entry to non-obvious individuals. We also contribute to the knowledge-based theory of the firm by showing the effectiveness of a market-mechanism to draw out knowledge from diverse external sources to solve internal problems

“Open” disclosure of innovations, incentives and follow-on reuse: Theory on processes of cumulative innovation and a field experiment in computational biology

AbstractMost of society's innovation systems – academic science, the patent system, open source, etc. – are “open” in the sense that they are designed to facilitate knowledge disclosure among innovators. An essential difference across innovation systems is whether disclosure is of intermediate progress and solutions or of completed innovations. We theorize and present experimental evidence linking intermediate versus final disclosure to an ‘incentives-versus-reuse’ tradeoff and to a transformation of the innovation search process. We find intermediate disclosure has the advantage of efficiently steering development towards improving existing solution approaches, but also has the effect of limiting experimentation and narrowing technological search. We discuss the comparative advantages of intermediate versus final disclosure policies in fostering innovation

Performance Responses to Competition Across Skill-Levels in Rank Order Tournaments: Field Evidence and Implications for Tournament Design

Tournaments are widely used in the economy to organize production and innovation. We study individual contestant-level data from 2,796 contestants in 774 software algorithm design contests with random assignment. Precisely conforming to theory predictions, the performance response to added contestants varies non-monotonically across contestants of different abilities, most respond negatively to competition, and highest-skilled contestants respond positively. In counterfactual simulations, we interpret a number of tournament design policies (number of competitors, prize allocation and structure, divisionalization, open entry) as a means of reconciling non-monotonic incentive responses to competition, effectively manipulating the number and skills distribution of contestants facing one another

The core and the periphery in distributed and self-organizing innovation systems

Thesis (Ph. D.)--Massachusetts Institute of Technology, Sloan School of Management, 2006.Includes bibliographical references.The Internet has enabled the large-scale mobilization of individuals to self-organize and innovate outside of formal organizations. My dissertation consists of three studies examining the functioning of such self-organizing and distributed innovation systems. I focus on the differing roles of core and peripheral participants in the distributed innovation process and explore the potential generality of this new form of innovating. The first study explores a systematic method of broadcast search used by corporations to search for solutions to internal R & D problems by involving peripheral problem solvers - those who are outside of their organizations. I find that innovative solutions to difficult scientific problems can be effectively identified by broadcasting problems to a large group of diverse solvers in different fields. Broadcast search yields innovative solutions by peripheral solvers who are crossing scientific disciplines. The central characteristic of problems that were successfully solved is the ability to attract specialized peripheral solvers with heterogeneous scientific interests. The second study examines how participants jointly innovate in a Free and Open Source Software community. I find that members at the periphery - those outside of the core project team - are responsible for developing a majority of functionally novel software features.(cont.) In contrast, core members develop performance-related features. Peripheral members also initiate the majority of the development activity and provide critical input into the technical problem solving processes. Ongoing interactions between core and peripheral members are the primary enablers of collective problem solving. I discuss how core and peripheral members enact six work practices in jointly producing software in a distributed and virtual setting. The third study examines the motivation of core participants in 287 Free and Open Source Software communities. Theorizing on individual motivations for participating in communities has posited that external motivational factors in the form of extrinsic benefits as the main drivers of effort. I find that enjoyment-based intrinsic motivation, namely how creative a person feels when working on the project was the strongest and most pervasive driver of effort. I also find that user need, intellectual stimulation derived from writing code, and improving programming skills as top motivators for project participation.by Karim R. Lakhani.Ph.D

The Novelty Paradox & Bias for Normal Science: Evidence from Randomized Medical Grant Proposal Evaluations

Central to any innovation process is the evaluation of proposed projects and allocation of resources. We investigate whether novel research projects, those deviating from existing research paradigms, are treated with a negative bias in expert evaluations. We analyze the results of a peer review process for medical research grant proposals at a leading medical research university, in which we recruited 142 expert university faculty members to evaluate 150 submissions, resulting in 2,130 randomly-assigned proposal-evaluator pair observations. Our results confirm a systematic penalty for novel proposals; a standard deviation increase in novelty drops the expected rank of a proposal by 4.5 percentile points. This discounting is robust to various controls for unobserved proposal quality and alternative explanations. Additional tests suggest information effects rather than strategic effects account for the novelty penalty. Only a minority of the novelty penalty could be related to perceptions of lesser feasibility of novel proposals