Performance assessment of public-funded R&D organizations working on similar research streams: A multinational study

The subject of deriving a measure of efficiency of public-funded organizations (primarily not-for-profit organizations) and of ranking these efficiency measures have been major subjects of debate and discussion. In the present study, the methodology of data envelopment analysis (DEA) has been used to analyze the relative performances of public funded R&D organizations across multiple countries working in similar research streams with multiple measures of inputs and outputs. The keywords highlighting the major research areas in the field of non-metrology conducted by National Physical Laboratory (NPL) in India were utilized to select the global comparators working in similar research streams. These global comparators were three R&D organizations located in the USA and one each located in Germany and Japan. The relative efficiencies of the organizations were assessed with variables such as external cash flow (ECF) earned, technologies transferred, publications and patents as outputs and grants received from the parent body and scientific personnel as inputs. The study indicates suggested measures and a set of targets to achieve the best possible performance for NPL and other R&D organizations

To publish or not to publish? On the aggregation and drivers of research performance

This paper presents a methodology to aggregate multidimensional research output. Using a tailored version of the non-parametric Data Envelopment Analysis model, we account for the large heterogeneity in research output and the individual researcher preferences by endogenously weighting the various output dimensions. The approach offers three important advantages compared to the traditional approaches: (1) flexibility in the aggregation of different research outputs into an overall evaluation score; (2) a reduction of the impact of measurement errors and a-typical observations; and (3) a correction for the influences of a wide variety of factors outside the evaluated researcher’s control. As a result, research evaluations are more effective representations of actual research performance. The methodology is illustrated on a data set of all faculty members at a large polytechnic university in Belgium. The sample includes questionnaire items on the motivation and perception of the researcher. This allows us to explore whether motivation and background characteristics (such as age, gender, retention, etc.,) of the researchers explain variations in measured research performance

A robust nonparametric approach to the analysis of scientific productivity

Data on scientific productivity at institutes of the French INSERM and at biomedical research institutes of the Italian CNR for 1997 were analysed. Available data on human capital input and geographical agglomeration allowed the estimation and comparison of efficiency measures for. Nonparametric envelopment techniques were used, and robust nonparametric techniques was applied in this work for the first time for evaluating scientific productivity. It is shown as a useful tool to compute scientific productivity indicators and make institutional comparative analyses. Taking into account a large number of methodological problems, a meaningful and rigorous indirect comparison is made possible. Several possible explanations of the observed differences in productivity are commented on

The state of the art development of AHP (1979-2017): A literature review with a social network analysis

Although many papers describe the evolution of the analytic hierarchy process (AHP), most adopt a subjective approach. This paper examines the pattern of development of the AHP research field using social network analysis and scientometrics, and identifies its intellectual structure. The objectives are: (i) to trace the pattern of development of AHP research; (ii) to identify the patterns of collaboration among authors; (iii) to identify the most important papers underpinning the development of AHP; and (iv) to discover recent areas of interest. We analyse two types of networks: social networks, that is, co-authorship networks, and cognitive mapping or the network of disciplines affected by AHP. Our analyses are based on 8441 papers published between 1979 and 2017, retrieved from the ISI Web of Science database. To provide a longitudinal perspective on the pattern of evolution of AHP, we analyse these two types of networks during the three periods 1979?1990, 1991?2001 and 2002?2017. We provide some basic statistics on AHP journals and researchers, review the main topics and applications of integrated AHPs and provide direction for future research by highlighting some open questions

The state of the art development of AHP (1979-2017): a literature review with a social network analysis

Although many papers describe the evolution of the analytic hierarchy process (AHP), most adopt a subjective approach. This paper examines the pattern of development of the AHP research field using social network analysis and scientometrics, and identifies its intellectual structure. The objectives are: (i) to trace the pattern of development of AHP research; (ii) to identify the patterns of collaboration among authors; (iii) to identify the most important papers underpinning the development of AHP; and (iv) to discover recent areas of interest. We analyse two types of networks: social networks, that is, co-authorship networks, and cognitive mapping or the network of disciplines affected by AHP. Our analyses are based on 8441 papers published between 1979 and 2017, retrieved from the ISI Web of Science database. To provide a longitudinal perspective on the pattern of evolution of AHP, we analyse these two types of networks during the three periods 1979–1990, 1991–2001 and 2002–2017. We provide some basic statistics on AHP journals and researchers, review the main topics and applications of integrated AHPs and provide direction for future research by highlighting some open questions

АНАЛИЗ ЭФФЕКТИВНОСТИ НАУЧНО-ИССЛЕДОВАТЕЛЬСКОЙ ДЕЯТЕЛЬНОСТИ СРЕДИ РАЗВИТЫХ И РАЗВИВАЮЩИХСЯ СТРАН, ВКЛЮЧАЯ РЕСПУБЛИКУ БЕЛАРУСЬ, С ИСПОЛЬЗОВАНИЕМ МЕТОДА СТОХАСТИЧЕСКОГО АНАЛИЗА ДАННЫХ

This study evaluates efficiency of R&D activities based on the stochastic frontier analysis across 69 counties with developed and developing economies. Gross domestic expenditures on R&D in purchasing power parity, researchers per million inhabitants, technicians per million inhabitants are treated as inputs while patents granted to residents and scientific and technical journal articles are considered as outputs. According to the analysis results Costa Rica, Israel and Singapore are the most efficient in terms of transformation of available resources into the R&D results. What concerns Belarus it is necessary that additional investments in R&D go together with increasing efficiency of available resources’ usage. В статье исследуется вопрос оценки эффективности научно-исследовательской деятельности 69 стран мира с развитой и развивающейся экономикой с помощью метода стохастического анализа данных (Stochastic Frontier Analysis, SFA). В качестве входных показателей (ресурсов) для расчета эффективности использовали следующие индикаторы: количество ученых на один миллион населения, количество инженеров и технического персонала на один миллион населения, затраты на научно-исследовательские разработки по паритету покупательской способности (в долларах США). Такие показатели, как количество патентов, выданных национальными патентными бюро резидентам, и количество опубликованных научных статей, были использованы как результаты научно-исследователь-ской деятельности. Проведенный анализ показал, что имеется ряд стран, таких как Коста-Рика, Израиль и Сингапур, с наилучшими показателями трансформации имеющихся ресурсов в результаты научно-исследовательской деятельности. В то же время, если говорить о Республике Беларусь, то дополнительное финансирование научно-исследовательской деятельности должно сопровождаться повышением эффективности использования имеющихся ресурсов.

MODELS FOR MEASURING THE RESEARCH PERFORMANCE AND MANAGEMENT OF THE PUBLIC LABS

The science sector, in some European countries, is doing a strategic restructuring due to budget cuts (e.g. Italy). Thus, the measure and evaluation of research performance (metrics) of its units (public research institute) is needed. General models to assess the R&D performance of a public research lab are presented here. The methodology uses the discriminant analysis and the results are two canonical discriminant functions (direct and Wilks methods) that could provide indications about the performance of research bodies. The functions are successfully applied to 200 public research institutes belonging to the Italian National Research Council. These functions are also tools for appropriate decisions and actions to improve research performance, especially by the more effective use of existing resources and for reducing the X-inefficiency. Some policy and management implications are discussed.Research performance, Performance measurement, Performance indicators, R&D evaluation, Public research lab, Discriminant analysis, X-inefficiency

Efficiency measurement of national innovation systems of the european union countries: DEA model application

This paper focuses on measuring technical efficiency of National Innovation System across a sample of the European Union (EU) countries using data envelopment analysis. We used an output-oriented constant returns to scale model to calculate the efficiency of the units represented by the European Union countries. This nonparametric method measures efficiency of input utilisation as compared to the achieved outputs during the consumption process. Following previous studies on this topic, we have used the number of researchers and the expenditures on research and development (R&D) as inputs and published scientific journal articles and applied patents as outputs of the model. Stemming from the available data, we have covered the period of 2005– 2016. This period covers the economic period before the financial crisis and economic recession in which it was necessary to use resources effectively for the maximisation of final production. The study also presents data representing the era of growth trend in European economy. The only efficient countries in our study as of 2016 were found to be Cyprus, Luxembourg, Malta, and Romania. The number of efficient units decreased from six countries measured in 2005 from among the eight calculated in the years of the economic crisis to four efficient decision-making units (DMUs) in 2015. Germany, the best performer in the European Union with respect to patent rankings, was classified as an inefficient unit with a 0.50 efficiency scale. The results indicate differences between the innovation performance investigated by various indices available in public databases and the DEA technical efficiency. The best performers can be considered as inefficient in the utilisation of resources entering the National Innovation System (NIS) for which technical efficiency is examined in the present study. © Foundation of International Studies, 2019

МЕЖСТРАНОВЫЙ АНАЛИЗ ЭФФЕКТИВНОСТИ ИННОВАЦИОННОЙ ДЕЯТЕЛЬНОСТИ НА ОСНОВЕ МЕТОДА ОБОЛОЧЕЧНОГО АНАЛИЗА ДАННЫХ СРЕДИ ГОСУДАРСТВ С РАЗВИТОЙ И РАЗВИВАЮЩЕЙСЯ ЭКОНОМИКАМИ, ВКЛЮЧАЯ РЕСПУБЛИКУ БЕЛАРУСЬ

The paper considers a problem on efficiency evaluation of innovation activity in 63 countries with developed and developing economies while using a method for data envelopment analysis. The following results of innovation activity have been used for calculation of an efficiency factor: export of high-technology products as percentage of industrial product export, export of ICT services as percentage of services export and payments obtained due to realization of intellectual property rights (in US dollars). A model of the data envelopment analysis with a changeable scale-dependent effect and which is directed on maximization of the obtained results (output-oriented VRS model) has been used for the analysis. The evaluation has shown that such countries as the USA, Israel, Sweden and some others have maximum efficiency of resource transformation into innovative activity output. The executed analysis has revealed that the Republic of Belarus has a potential for improvement of indices on innovation results.В статье исследуется вопрос оценки эффективности инновационной деятельности 63 стран мира с развитой и развивающейся экономиками с помощью метода оболочечного анализа данных. Для расчета показателя эффективности использовались следующие результаты инновационной деятельности: экспорт высокотехнологичных товаров в процентах от экспорта производственных товаров, экспорт услуг информационно-коммуникационных технологий в процентах от экспорта услуг и платежи, получаемые от реализации прав интеллектуальной собственности (в долларах США). В качестве показателей ресурсов для расчета показателя эффективности выбраны параметры наукоемкости ВВП, количество ученых, занятых в сфере НИОКР (на один миллион населения), и затраты на приобретение прав интеллектуальной собственности (в долларах США). Для анализа использовалась модель оболочечного анализа данных с переменным эффектом от масштаба, направленная на максимизацию полученных данных. В ходе оценки оказалось, что такие страны, как США, Израиль, Швеция и ряд других, имеют максимальную эффективность преобразования ресурсов в результаты инновационной деятельности. Из проведенного анализа следует, что у Республики Беларусь есть потенциал улучшения показателей результатов инновационной деятельности