EVALUATION REPORT OF LAUREA UNIVERSITY OF APPLIED SCIENCES 2010

In this report, the evaluation board considers international level RDI and a high quality research culture as a central means for attaining Laurea’s vision. In order to support this development the evaluation board has provided the Laurea community with valuable external feedback regarding its performance and its way of orchestrating its RDI operations

What Stimulates Researchers to Make Their Research Usable? Towards an Openness Approach

Ambiguity surrounding the effect of external engagement on academic research has raised questions about what motivates researchers to collaborate with third parties. We argue that what matters for society is research that can be absorbed by users. We define openness as a willingness by researchers to make research more usable by external partners by responding to external influences in their own research practices. We ask what kinds of characteristics define those researchers who are more open to creating usable knowledge. Our empirical study analyses a sample of 1583 researchers working at the Spanish Council for Scientific Research (CSIC). Results demonstrate that it is personal factors (academic identity and past experience) that determine which researchers have open behaviours. The paper concludes that policies to encourage external engagement should focus on experiences which legitimate and validate knowledge produced through user encounters, both at the academic formation career stage as well as through providing ongoing opportunities to engage with third parties.The data used for this study comes from the IMPACTO project funded by the Spanish Council for Scientific Research - CSIC (Ref. 200410E639). The work also benefited from a mobility grant awarded by Eu-Spri Forum to Julia Olmos Penuela & Paul Benneworth for her visiting research to the Center of Higher Education Policy Studies. Finally, Julia Olmos Penuela also benefited from a post-doctoral grant funded by the Generalitat Valenciana (APOSTD-2014-A-006).Olmos-Peñuela, J.; Benneworth, P.; Castro-Martínez, E. (2015). What Stimulates Researchers to Make Their Research Usable? Towards an Openness Approach. Minerva. 53(4):381-410. https://doi.org/10.1007/s11024-015-9283-4S381410534Abreu, Maria, Vadim Grinevich, Alan Hughes, and Michael Kitson. 2009. Knowledge exchange between academics and the business, public and third sectors. Cambridge: Centre for Business Research and UK-IRC.Aghion, Philippe, Mathias Dewatripont, and Jeremy C. Stein. 2008. Academic freedom, private-sector focus, and the process of innovation. RAND Journal of Economics 39: 617–635.Ajzen, Icek. 2001. Nature and operation of attitudes. Annual Review of Psychology 52(1): 27–58.Alrøe, Hugo Fjelsted, and Erik Steen Kristensen. 2002. Towards a systemic research methodology in agriculture: Rethinking the role of values in science. Agriculture and Human Values 19(1): 3–23.Audretsch, David B., Werner Bönte, and Stefan Krabel. 2010. Why do scientists in public research institutions cooperate with private firms. In DRUID Working Paper, 10–27.Baldini, Nicola, Rosa Grimaldi, and Maurizio Sobrero. 2007. To patent or not to patent? A survey of Italian inventors on motivations, incentives, and obstacles to university patenting. Scientometrics 70(2): 333–354.Bandura, Albert. 1977. Social learning theory. Englewood Cliffs, NJ: Prentice-Hall.Barnett, R. 2009. Knowing and becoming in the higher education curriculum. Studies in Higher Education 34(4): 429–440.Becher, Tony. 1994. The significance of disciplinary differences. Studies in Higher Education 19(2): 151–161.Becher, Tony, and Paul Trowler. 2001. Academic tribes and territories: Intellectual enquiry and the culture of disciplines. McGraw-Hill International.Bekkers, Rudi, and Isabel Maria Bodas Freitas. 2008. Analysing knowledge transfer channels between universities and industry: To what degree do sectors also matter? Research Policy 37(10): 1837–1853.Belderbos, René, Martin Carree, Bert Diederen, Boris Lokshin, and Reinhilde Veugelers. 2004. Heterogeneity in R&D cooperation strategies. International Journal of Industrial Organization 22(8): 1237–1263.Benner, Mats, and Ulf Sandström. 2000. Institutionalizing the triple helix: Research funding and norms in the academic system. Research Policy 29(2): 291–301.Bercovitz, Janet, and Maryann Feldman. 2008. Academic entrepreneurs: Organizational change at the individual level. Organization Science 19(1): 69–89.Berman, Elizabeth Popp. 2011. Creating the market university: How academic science became an economic engine. Princeton University Press.Bleiklie, Ivar, and Roar Høstaker. 2004. Modernizing research training-education and science policy between profession, discipline and academic institution. Higher Education Policy 17(2): 221–236.Bozeman, Barry, Daniel Fay, and Catherine P. Slade. 2013. Research collaboration in universities and academic entrepreneurship: The-state-of-the-art. The Journal of Technology Transfer 38(1): 1–67.Collini, Stefan. 2009. Impact on humanities: Researchers must take a stand now or be judged and rewarded as salesmen. The Times Literary Supplement 5563: 18–19.D’Este, Pablo, and Markus Perkmann. 2011. Why do academics engage with industry? The entrepreneurial university and individual motivations. The Journal of Technology Transfer 36(3): 316–339.D’Este, Pablo, Oscar Llopis, and Alfredo Yegros. 2013. Conducting pro-social research: Cognitive diversity, research excellence and awareness about the social impact of research: INGENIO (CSIC-UPV) Working Paper Series.Deem, Rosemary, and Lisa Lucas. 2007. Research and teaching cultures in two contrasting UK policy contexts: Academic life in education departments in five English and Scottish universities. Higher Education 54(1): 115–133.DiMaggio, Paul J., and Walter W. Powell. 1983. The iron cage revisited: Institutional isomorphism and collective rationality in organizational fields. American Sociological Review 48(2): 147–160.Downing, David B. 2005. The knowledge contract: Politics and paradigms in the academic workplace. Lincoln: Nebraska University of Nebraska Press.Donovan, Claire. 2007. The qualitative future of research evaluation. Science and Public Policy 34(8): 585–597.Durning, Bridget. 2004. Planning academics and planning practitioners: Two tribes or a community of practice? Planning Practice and Research 19(4): 435–446.Edquist, Charles. 1997. System of innovation approaches: Their emergence and characteristics. In Systems of innovation: Technologies, institutions and organizations, ed. C. Edquist, 1–35. London: Pinter.Etzkowitz, Henry, and Loet Leydesdorff. 2000. The dynamics of innovation: from National Systems and “Mode 2” to a Triple Helix of university–industry–government relations. Research Policy 29(2): 109–123.Fromhold-Eisebith, Martina, Claudia Werker, and Marcel Vojnic. 2014. Tracing the social dimension in innovation networks. In The social dynamics of innovation networks, eds. Roel Rutten, Paul Benneworth, Frans Boekema, and Dessy Irawati. London: Routledge (in press).Geuna, Aldo, and Alessandro Muscio. 2009. The governance of university knowledge transfer: A critical review of the literature. Minerva 47(1): 93–114.Gibbons, Michael, Camille Limoges, Helga Nowotny, Simon Schwartzman, Peter Scott, and Martin Trow. 1994. The new production of knowledge: The dynamics of science and research in contemporary societies. London: Sage.Gläser, Jochen. 2012. How does Governance change research content? On the possibility of a sociological middle-range theory linking science policy studies to the sociology of scientific knowledge. Technical University Berlin. Technology Studies Working Papers. http://www.ts.tu-berlin.de/fileadmin/fg226/TUTS/TUTS-WP-1-2012.pdf . Accessed 16 Feb 2015.Goethner, Maximilian, Martin Obschonka, Rainer K. Silbereisen, and Uwe Cantner. 2012. Scientists’ transition to academic entrepreneurship: Economic and psychological determinants. Journal of Economic Psychology 33(3): 628–641.Gulbrandsen, Magnus, and Jens-Christian Smeby. 2005. Industry funding and university professors’ research performance. Research Policy 34(6): 932–950.Haeussler, Carolin, and Jeannette Colyvas. 2011. Breaking the ivory tower: Academic entrepreneurship in the life sciences in UK and Germany. Research Policy 40(1): 41–54.Hessels, Laurens K., Harro van Lente, John Grin, and Ruud E.H.M. Smits. 2011. Changing struggles for relevance in eight fields of natural science. Industry and Higher Education 25(5): 347–357.Hessels, Laurens K., and Harro Van Lente. 2008. Re-thinking new knowledge production: A literature review and a research agenda. Research Policy 37(4): 740–760.Hoye, Kate, and Fred Pries. 2009. ‘Repeat commercializers’, the ‘habitual entrepreneurs’ of university–industry technology transfer. Technovation 29(10): 682–689.Jacobson, Nora, Dale Butterill, and Paula Goering. 2004. Organizational factors that influence university-based researchers’ engagement in knowledge transfer activities. Science Communication 25(3): 246–259.Jain, Sanjay, Gerard George, and Mark Maltarich. 2009. Academics or entrepreneurs? Investigating role identity modification of university scientists involved in commercialization activity. Research Policy 38(6): 922–935.Jasanoff, Sheila, and Sang-Hyun Kim. 2013. Sociotechnical imaginaries and national energy policies. Science as Culture 22(2): 189–196.Jensen, Pablo. 2011. A statistical picture of popularization activities and their evolutions in France. Public Understanding of Science 20(1): 26–36.Kitcher, Philip. 2001. Science, truth, and democracy. Oxford: Oxford University Press.Knorr-Cetina, Karin. 1981. The manufacture of knowledge: An essay on the constructivist and contextual nature of science. Oxford: Pergamon Press.Kronenberg, Kristin, and Marjolein Caniëls. 2014. Professional proximity in research collaborations. In The social dynamics of innovation networks, eds. Roel Rutten, Paul Benneworth, Frans Boekema, and Dessy Irawati. London: Routledge (in press).Krueger, Rob, and David Gibbs. 2010. Competitive global city regions and sustainable development’: An interpretive institutionalist account in the South East of England. Environment and planning A 42: 821–837.Lam, Alice. 2011. What motivates academic scientists to engage in research commercialization: ‘Gold’, ‘ribbon’ or ‘puzzle’? Research Policy 40(10): 1354–1368.Landry, Réjean, Malek Saïhi, Nabil Amara, and Mathieu Ouimet. 2010. Evidence on how academics manage their portfolio of knowledge transfer activities. Research Policy 39(10): 1387–1403.Lee, Alison, and David Boud. 2003. Writing groups, change and academic identity: Research development as local practice. Studies in Higher Education 28(2): 187–200.Lee, Yong S. 1996. ‘Technology transfer’ and the research university: A search for the boundaries of university–industry collaboration. Research Policy 25(6): 843–863.Lee, Yong S. 2000. The sustainability of university–industry research collaboration: An empirical assessment. The Journal of Technology Transfer 25(2): 111–133.Leisyte, Liudvika, Jürgen Enders, and Harry De Boer. 2008. The freedom to set research agendas—illusion and reality of the research units in the Dutch Universities. Higher Education Policy 21(3): 377–391.Louis, Karen Seashore, David Blumenthal, Michael E. Gluck, and Michael A. Stoto. 1989. Entrepreneurs in academe: An exploration of behaviors among life scientists. Administrative Science Quarterly 34(1): 110–131.Lowe, Philip, Jeremy Phillipson, and Katy Wilkinson. 2013. Why social scientists should engage with natural scientists. Contemporary Social Science 8(3): 207–222.Martín-Sempere, María José, Belén Garzón-García, and Jesús Rey-Rocha. 2008. Scientists’ motivation to communicate science and technology to the public: Surveying participants at the Madrid Science Fair. Public Understanding of Science 17(3): 349–367.Martin, Ben. 2003. The changing social contract for science and the evolution of the university. In Science and innovation: Rethinking the rationales for funding and governance, eds. A. Geuna, A.J. Salter, and W.E. Steinmueller, 7–29. Cheltenhan: Edward Elgar.Merton, Robert K. 1973. The sociology of science: Theoretical and empirical investigations. Chicago: University of Chicago Press.Miller, Thaddeus R., and Mark W. Neff. 2013. De-facto science policy in the making: how scientists shape science policy and why it matters (or, why STS and STP scholars should socialize). Minerva 51(3): 295–315.Muthén, Bengt O. 1998–2004. Mplus Technical Appendices. Muthén & Muthén. Los Angeles, CA.: Muthén & Muthén.Nedeva, Maria. 2013. Between the global and the national: Organising European science. Research Policy 42(1): 220–230.Neff, Mark William. 2014. Research prioritization and the potential pitfall of path dependencies in coral reef science. Minerva 52(2): 213–235.Nelson, Richard R. 2001. Observations on the post-Bayh-Dole rise of patenting at American universities. The Journal of Technology Transfer 26(1): 13–19.Nowotny, Helga, Peter Scott, and Michael Gibbons. 2001. Re-thinking science: Knowledge and the public in an age of uncertainty. Cambridge: Polity Press.Olmos-Peñuela, Julia, Paul Benneworth, and Elena Castro-Martínez. 2014a. Are ‘STEM from Mars and SSH from Venus’? Challenging disciplinary stereotypes of research’s social value. Science and Public Policy 41: 384–400.Olmos-Peñuela, Julia, Elena Castro-Martínez, and Manuel Fernández-Esquinas. 2014b. Diferencias entre áreas científicas en las prácticas de divulgación de la investigación: un estudio empírico en el CSIC. Revista Española de Documentación Científica. doi: 10.3989/redc.2014.2.1096 .Ouimet, Mathieu, Nabil Amara, Réjean Landry, and John Lavis. 2007. Direct interactions medical school faculty members have with professionals and managers working in public and private sector organizations: A cross-sectional study. Scientometrics 72(2): 307–323.Perkmann, Markus, Valentina Tartari, Maureen McKelvey, Erkko Autio, Anders Brostrom, Pablo D’Este, Riccardo Fini, et al. 2013. Academic engagement and commercialisation: A review of the literature on university-industry relations. Research Policy 42(2): 423–442.Philpott, Kevin, Lawrence Dooley, Caroline O’Reilly, and Gary Lupton. 2011. The entrepreneurial university: Examining the underlying academic tensions. Technovation 31(4): 161–170.Rutten, Roel, and Frans Boekema. 2012. From learning region to learning in a socio-spatial context. Regional Studies 46(8): 981–992.Sarewitz, Daniel, and Roger A. Pielke. 2007. The neglected heart of science policy: reconciling supply of and demand for science. Environmental Science & Policy 10(1): 5–16.Sauermann, Henry, and Paula Stephan. 2013. Conflicting logics? A multidimensional view of industrial and academic science. Organization Science 24(3): 889–909.Schein, Edgar H. 1985. Organizational culture and leadership: A dynamic view. San Francisco, CA: Jossey-Bass.Shane, Scott. 2000. Prior knowledge and the discovery of entrepreneurial opportunities. Organization Science 11(4): 448–469.Spaapen, Jack, and Leonie van Drooge. 2011. Introducing ‘productive interactions’ in social impact assessment. Research Evaluation 20(3): 211–218.Stokes, Donald E. 1997. Pasteur’s quadrant: Basic science and technological innovation. Washington, DC: Brookings Institution Press.Tartari, Valentina, and Stefano Breschi. 2012. Set them free: scientists’ evaluations of the benefits and costs of university–industry research collaboration. Industrial and Corporate Change 21(5): 1117–1147.Tinker, Tony, and Rob Gray. 2003. Beyond a critique of pure reason: From policy to politics to praxis in environmental and social research. Accounting, Auditing & Accountability Journal 16(5): 727–761.van Rijnsoever, Frank J., Laurens K. Hessels, and Rens L.J. Vandeberg. 2008. A resource-based view on the interactions of university researchers. Research Policy 37(8): 1255–1266.Venkataraman, Sankaran. 1997. The distinctive domain of entrepreneurship research: An editor’s perspective. Advances in Entrepreneurship, Firm Emergence, and Growth 3: 119–138.Verspagen, Bart. 2006. University research, intellectual property rights and European innovation systems. Journal of Economic Surveys 20(4): 607–632.Villanueva-Felez, Africa, Jordi Molas-Gallart, and Alejandro Escribá-Esteve. 2013. Measuring personal networks and their relationship with scientific production. Minerva 51(4): 465–483.Watermeyer, Richard. 2015. Lost in the ‘third space’: the impact of public engagement in higher education on academic identity, research practice and career progression. European Journal of Higher Education (online first, doi: 10.1080/21568235.2015.1044546 ).Weingart, Peter. 2009. Editorial for Issue 47/3. Minerva 47(3): 237–239.Ziman, John. 1996. ‘Postacademic science’: Constructing knowledge with networks and norms. Science Studies 1: 67–80.Zomer, Arend H., Ben W.A. Jongbloed, and Jürgen Enders. 2010. Do spin-offs make the academics’ heads spin? The impacts of spin-off companies on their parent research organisation. Minerva 48(3): 331–353

A New Evaluation Culture Is Inevitable

Changes in the production of research (more collaborative, more inter- and transdisciplinary, more oriented towards societal demand) are influencing the ways in which research is evaluated. Traditional methods of evaluation primarily focussing on the production of scientific articles have long since given way to more comprehensive methods in which researchers’ other activities are assessed too. Beyond these developments, evaluation also involves research endeavours concerning collaboration with other stakeholders in society, such as industry, NGO’s, consumer groups, or governmental organisations

Evaluation and monitoring of transdisciplinary collaborations

In this paper we focus on the governance, in particular evaluation and monitoring, of the growing number of transdisciplinary collaborations (TDC’s). Researchers and a variety of stakeholders collaborate in such TDC’s, the purpose of which is to address societal challenges, like renewable energy, healthy aging or better language teaching in schools. Commonly used practices for evaluation of scientific research (accountability, rankings and benchmarking, dedicated to scientific excellence) do not fit the goals of TDC’s. A bottom up or stakeholder oriented approach is better suited; one that stimulates mutual learning as well as the development of socially robust knowledge. We introduce the participatory impact pathways analysis (PIPA), as a method that suits the requirements. It has been developed in the context of development research. Two crucial features are the involvement of stakeholders from the start, and the joint development of a theory of change. This narrates what one wants to achieve and how that will be achieved. From this, stakeholders construct a logical frame that serves as a source for indicators. These indicators enable monitoring ex durante, during the TDC. We present evidence of the use of PIPA for a TDC. From this empirical evidence a number of issues with regard to evaluation, monitoring and indicators can be identified that require attention. Most prominent is the change of function of indicators. Instead of looking back and a focus on past performance, indicators look forward, in the short, intermediate and more distant future

Valorizing SSH research: Towards a new approach to evaluate SSH research’ value for society

Dieser Artikel versucht, eine Argumentation für eine valorisierende Evaluierung der SSH-Forschung darzustellen. Basierend auf einer Präsentation der Ergebnisse einer Umfrage zur SSH-Forschungsevaluation in Europa, die im Rahmen der COST-Aktion 15137 ENRESSH durchgeführt wurde. Die Umfrage ermöglichte es, die Wahrnehmungen über die wichtigsten Prinzipien der SSH-Forschungsevaluation zu erfassen, eine Typologie verschiedener Bewertungsmodelle voranzutreiben und die mit diesen Modellen verbundenen Probleme besser zu identifizieren. Im zweiten Teil wird anhand bewährter Verfahren und basierend auf diesen vorläufigen Ergebnissen einen Ansatz vorgestellt, der Leistung und Wirkung auf eine Weise kombiniert, die eine Lösung für die SSH-Evaluierung und möglicherweise darüber hinaus darstellen kann. Im dritten Teil wird eine Analyse der Schwierigkeiten die mit der Valorisierung von SSH durch Evaluation einhergehen, diskutiert. Diese wurden in der jüngeren Literatur in gewissem Umfang diskutiert, insbesondere nachdem verschiedene Länder Evaluierungssysteme etabliert haben, die vermehrt auf die "gesellschaftliche Auswirkungen" abzielen