Does the Committee Peer Review Select the Best Applicants for Funding? An Investigation of the Selection Process for Two European Molecular Biology Organization Programmes

Does peer review fulfill its declared objective of identifying the best science and the best scientists? In order to answer this question we analyzed the Long-Term Fellowship and the Young Investigator programmes of the European Molecular Biology Organization. Both programmes aim to identify and support the best post doctoral fellows and young group leaders in the life sciences. We checked the association between the selection decisions and the scientific performance of the applicants. Our study involved publication and citation data for 668 applicants to the Long-Term Fellowship programme from the year 1998 (130 approved, 538 rejected) and 297 applicants to the Young Investigator programme (39 approved and 258 rejected applicants) from the years 2001 and 2002. If quantity and impact of research publications are used as a criterion for scientific achievement, the results of (zero-truncated) negative binomial models show that the peer review process indeed selects scientists who perform on a higher level than the rejected ones subsequent to application. We determined the extent of errors due to over-estimation (type I errors) and under-estimation (type 2 errors) of future scientific performance. Our statistical analyses point out that between 26% and 48% of the decisions made to award or reject an application show one of both error types. Even though for a part of the applicants, the selection committee did not correctly estimate the applicant's future performance, the results show a statistically significant association between selection decisions and the applicants' scientific achievements, if quantity and impact of research publications are used as a criterion for scientific achievement

Explicitly searching for useful inventions: dynamic relatedness and the costs of connecting versus synthesizing

Inventions combine technological features. When features are barely related, burdensomely broad knowledge is required to identify the situations that they share. When features are overly related, burdensomely broad knowledge is required to identify the situations that distinguish them. Thus, according to my first hypothesis, when features are moderately related, the costs of connecting and costs of synthesizing are cumulatively minimized, and the most useful inventions emerge. I also hypothesize that continued experimentation with a specific set of features is likely to lead to the discovery of decreasingly useful inventions; the earlier-identified connections reflect the more common consumer situations. Covering data from all industries, the empirical analysis provides broad support for the first hypothesis. Regressions to test the second hypothesis are inconclusive when examining industry types individually. Yet, this study represents an exploratory investigation, and future research should test refined hypotheses with more sophisticated data, such as that found in literature-based discovery research

Decoding symbiotic endogeneity: the stochastic input-output analysis of university-business-government alliances

The theoretical population ecology constructs of commensalism, parasitism, and amensalism are applied in an analysis of the Knowledge Cluster Initiative (KCI), a unique social experiment establishing university-business-government alliances for knowledge-intensive innovative clusters in Japan. An analysis based on multiple negative binomial regressions to confirm the interdependence of the triple helix variables revealed that new startup venture firms served as an input factor for filing new patents and developing new products. Although the ultimate goal of the KCI was to promote new startups from the university, the startups had a commensal effect on patents and new product development. The Japanese cluster creation policy encouraged academic participation and created a mutual effect of launching new university startups. The resulting increase in the number of university researchers promoted the establishment of new startups. These startups had commensal relationships with patent applications and new product development. Although the creation of new venture startups was the ultimate goal of the cluster promotion policy, the results of this study indicate that it was the universities that benefited from the new startups that commensally contributed to increasing quantities of alliance outputs such as patents and new products