Comparative evaluation of performance of national R&D programs with heterogeneous objectives: A DEA approach

The strategic importance of performance evaluation of national R&D programs is highlighted as the resource allocation draws more attention in R&D policy agenda. Due to the heterogeneity of national R&D programs'''''''' objectives, however, it is intractably difficult to relatively evaluate multiple programs and. consequently, few studies have been conducted on the performance comparison of the R&D programs. This study measures and compares the performance of national R&D programs using data envelopment analysis (DEA). Since DEA allows each DMU to choose the optimal weights of inputs and outputs which maximize its efficiency, it can mirror R&D programs'''''''' unique characteristics by assigning relatively high weights to the variables in which each program has strength. Every project in every R&D program is evaluated together based on the DEA model for comparison of efficiency among different systems. Kruskal-Wallis test with a post hoc Mann-Whitney U test is then run to compare performance of R&D programs. Two alternative approaches to incorporating the importance of variables, the AR model and output integration, are also introduced. The results are expected to provide policy implications for effectively formulating and implementing national R&D programs. (C) 2008 Elsevier B.V. All rights reserved.Cherchye L, 2007, SOC INDIC RES, V82, P111, DOI 10.1007/s11205-006-9029-7Wang EC, 2007, RES POLICY, V36, P260, DOI 10.1016/j.respol.2006.11.004Swink M, 2006, J OPER MANAG, V24, P542, DOI 10.1016/j.jom.2005.09.004Eilat H, 2006, EUR J OPER RES, V172, P1018, DOI 10.1016/j.ejor.2004.12.001KOCHER MG, 2006, SOCIOECONOMIC PLANNI, V40, P314Garg KC, 2005, SCIENTOMETRICS, V65, P151, DOI 10.1007/s11192-005-0264-5LEE H, 2005, ASIAN J TECHNOL INNO, V13, P207FENG YJ, 2004, INT T OPERATIONAL RE, V11, P181, DOI 10.1111/j.1475-3995.2004.00450.xZhang AM, 2003, J COMP ECON, V31, P444, DOI 10.1016/S0147-5967(03)00055-6TAKAMURA Y, 2003, SOCIOECONOMIC PLANNI, V37, P85Paradi JC, 2002, IEEE T ENG MANAGE, V49, P161, DOI 10.1109/TEM.2002.1010884Korhonen P, 2001, EUR J OPER RES, V130, P121*OECD, 2001, OECD SCI TECHN IND SCOOPER WW, 2000, DATA ENVELOPMENT ANAADAMS JD, 2000, EC ECONOMETRICS INNORosen D, 1998, J PROD ANAL, V9, P205Allen R, 1997, ANN OPER RES, V73, P13Graves SB, 1996, TECHNOL FORECAST SOC, V53, P125Lee MS, 1996, RES POLICY, V25, P805BRINN T, 1996, ACCOUNT BUSINESS RES, V26, P265SHANG J, 1995, EUR J OPER RES, V85, P297NEDERHOF AJ, 1993, RES POLICY, V22, P353BOUSSOFIANE A, 1991, EUR J OPER RES, V52, P1THOMPSON RC, 1990, MEAS SCI TECHNOL, V1, P93BANKER R, 1989, RES GOVT NONPROFIT A, V5, P125DYSON RG, 1988, J OPER RES SOC, V39, P563NEDERHOF AJ, 1987, SCIENTOMETRICS, V11, P330BANKER RD, 1984, MANAGE SCI, V30, P1078BOUND J, 1984, R D PATENT PRODUCTIVSCHERER FM, 1983, INT J IND ORGAN, V1, P107SAATY TL, 1980, ANAL HIERARCHY PROCECHARNES A, 1978, EUR J OPER RES, V2, P429

Comparative evaluation of performance of national R&D programs with heterogeneous objectives: A DEA approach

The strategic importance of performance evaluation of national R&D programs is highlighted as the resource allocation draws more attention in R&D policy agenda. Due to the heterogeneity of national R&D programs' objectives, however, it is intractably difficult to relatively evaluate multiple programs and, consequently, few studies have been conducted on the performance comparison of the R&D programs. This study measures and compares the performance of national R&D programs using data envelopment analysis (DEA). Since DEA allows each DMU to choose the optimal weights of inputs and outputs which maximize its efficiency, it can mirror R&D programs' unique characteristics by assigning relatively high weights to the variables in which each program has strength. Every project in every R&D program is evaluated together based on the DEA model for comparison of efficiency among different systems. Kruskal-Wallis test with a post hoc Mann-Whitney U test is then run to compare performance of R&D programs. Two alternative approaches to incorporating the importance of variables, the AR model and output integration, are also introduced. The results are expected to provide policy implications for effectively formulating and implementing national R&D programs.Data envelopment analysis R&D evaluation R&D performance National R&D program Analytic hierarchy process