This paper presents an overview of evaluation of ISTCP in China.  We discuss briefly the history of evaluation and the strengths and weaknesses of different assessment systems.  On this basis, with Analytical Hierarchy Process (AHP), we establish evaluation indicator system for ISTCP that includes research project establishment evaluation, mid-period evaluation system, effect evaluation system, and confirm the value of each indicator.  At the same time, we established expert database, project database, research organization database, researcher database etc.  We therefore establish an evaluation platform for international science and technology cooperation project.  We use it to realize full process supervision from evaluation expert selection to project management

Cheng, Su

Hong, Guo

Yuntao, Pan

Zheng, Ma

Zhenglu, Yu

English

E-LIS

  STUDY ON EVALUATION OF INTERNATIONAL SCIENCE AND TECHNOLOGY COOPERATION PROJECT (ISTCP) IN CHINA SU CHENG, GUO HONG, MA ZHENG, YU ZHENGLU, PAN YUNTAO  Institute of Scientific and Technical Information of China (ISTIC) No.15 Fuxing Road, Beijing 100038, the People’s Republic of China Abstract This paper presents an overview of evaluation of ISTCP in China.  We discuss briefly the history of evaluation and the strengths and weaknesses of different assessment systems.  On this basis, with Analytical Hierarchy Process (AHP), we establish evaluation indicator system for ISTCP that includes research project establishment evaluation, mid-period evaluation system, ?????? evaluation system, and confirm the value of each indicator.  At the same time, we established expert database, project database, research organization database, researcher database etc.  We therefore establish an evaluation platform for international science and technology cooperation project.  We use it to realize full process supervision from evaluation expert selection to project management.  1.  Survey on Evaluation of ISTCP in China Today, the economic globalization, R&D internationalization and technology and economy unification have converged to irreversible trend, which is even broadened and deepened by China’ s entry to WTO.  Abrupt increase of ISTCP in China is demanding more efficiently scientific and standardized management, of which the scientific evaluation towards the standardization of projects is an important part.   ISTCP backed by foreign government are evaluated according to different supporting countries, following their procedures and measures.  Recent years many organizations start to make deliberation process and measures for themselves, and with best of their knowledge work out standardized project selection basis.  There are also organizations or institutions stipulated some ordinances and rules to comprehensively and scientifically manage the project establishment appraisal and execution verification.  Following is the introduction of ISTCP evaluation methods adopted by Chinese Academy of Sciences (CAS) Academy of Science & Technology of Ministry of Aeronaut & Astronaut (ASTMAA) China State Shipbuilding Corporation (CSSC) Yun Nan Science & Technology Commission (YNSTC) 1.1. CAS ISTCP management of CAS includes project macro management, project organization and selection, project tracking management and evaluation and establishment of project management database, etc.   On project establishment appraisal, it stipulates the principles of project establishment, appraisal regulations, appraising items, appraising procedures, appraising standards, appraising personnel selecting conditions and so on.  It also builds appraising expert teams according to such main subjects as mathematics, physics, chemistry, biology, technology and geosciences.  These teams “discuss among collectivity while grade individually”.  On project tracking management and evaluation, it takes part in middle and long term planning and study of the important projects, putting forward cooperation suggestions according to the characters of the projects, and providing necessary information and allowance.  It takes part in mid-term evaluation and progress evaluation until getting the final achievements. 1.2. ASTMAA ASTMAA in practice has formed effective project management ways.  It masters chains like careful topic selection, plan carrying out, and periodical evaluation and so on.  It has built stably consisted international cooperation topic appraisal committee and reviews the execution of the cooperative projects last year.  It is improving the cooperation level continually. 1.3. CSSC CSSC emphasizes the core of cooperation and communication.  It strictly checks the project from establishment and supervises the whole process.  The projects are appraised timely and the headquarter holds intercommunication conference every 2~3 years.   1.4. YNSTC YNSTC has founded "ISTCP plan" and worked out “YNSTC ISTCP Allowance Management Guideline”, stipulating managing regulations on allowance application, project appraisal, contract, funds allocation, checking, achievement appraisal, achievement publication and so on. It invites Chinese and foreign experts and entrepreneurs to take part in achievement appraisal.  For projects with trading potential, it uses the market compatibility after their being put into the market as an important basis to appraise them.  In total in China there has not formed unified and standardized project management and is still lacking in scientific appraising methods.  Problems often seen are: although most organizations have their procedure and methods to appraise the establishment of ISTCP, there are no rules or regulations, and no standardization of operation. Some organizations have certain randomicity in establishing a project, and have no specific requirement or methods for evaluation. 2. Evaluation System of ISTCP in China 2.1. Design of the Evaluation System of ISTCP in China We adopt ??? method, breaking down the evaluation of ISTCP to a sequential and terraced structural model.  There are 3 levels in this model. They are Based on large scale questionnaires, we held several expert proseminars. With Delphi method, we decided the object level, guide line level and indicator level of ISTCP evaluation system. When evaluation indicator system was initially decided, we invited experts to grade different indicators with level analyzing method combined with Delphi method. The value of each indicator is calculated by Expert Choice software with following flow chart.    Pass    Fig. 1:the flow chart of Design of the Evaluation System of ISTCP in China             2.2. China ISTCP Evaluation System and the Evaluation Standards China ISTCP evaluation system consists 3 parts: ISTCP project establishment evaluation system, ISTCP mid-term evaluation system and ISTCP effect evaluation system. 2.2.1  ISTCP Research Project Establishment Evaluation Comprehensive Indicator System ISTCP research project establishment evaluation comprehensive indicator system is very important for guaranteeing the project quality, fairness of evaluation and healthy development of International Science and Technology Cooperation.  pass No pass No pass Expert Fill judging matrix Calculate single level value Calculate general level value General level check Comprehensive evaluation Single level consistency check    Table 1: ISTCP research project establishment evaluation comprehensive indicator system. Indicator Evaluation Standards Indicator Value Cooperating country selection Cooperating country’ s attitude and openness towards Chin and its social, political and economical environment 0.043 Strategic relativity If the project is closely related to China’ s emphasized development fields and if it is listed in 863 plan, basic research plan and tackle key plans, if is listed in governmental ISTCP protocol.  If the project can resolve important social problem, economic problem, if it helps building up commercial ties and helping technological product and knowledge products and service to go to the international market. 0.189 Talent training Help China’ s scholars understand science and technology frontier and broaden field of vision; study advanced technology and operation of advanced equipment; learn advanced scientific project management experience; train the elite of scientific fields and import excellent talent 0.096 Political and diplomatic needs If it meets China’ s foreign policy object, improves international relationship and strengthen China’ s international influence 0.096 Patent protection If it is related to achievement sharing, patent application, copyright ownership, resource and other right protection agreement 0.076 Project allowance budge If allowance applied is reasonable; allowance allocation is reasonable; total project financial plan is reasonable 0.080 Chinese part level If the academic and technological levels of the China part are advanced in that field in China; if the project principal of China part is the lead in that field in china; the basis and accumulation of China part is well compensative to foreign part; if both parts can form high quality cooperative team; if the China part is the best one for that task in China 0.060 Foreign part executive level If the academic and technological levels of the foreign part are advanced in that field in that country; if the project principal of foreign part is the lead in that field in china; the basis and accumulation of China part is well compensative to foreign part; if both parts can form high quality cooperative team; if the foreign part is the best one for that task in that country 0.047 Potential Influence If the cooperative achievement can accelerate creativity.  The cooperative achievement may lead to breakthrough progress in its scientific field and have influence to relative fields; it may have strategic influence to resolve significant social and economical problems 0.064 Rationality of Anticipated Goal The objects put up is definite and clear, appropriate and broad, guaranteeing feasibility while exceeding current latest development 0.099 Cooperation Plan and Manner Cooperation plan, rationality of method, maneuverability, advanced research artifice, correctness of research routine and creativity; rationality of time arrangement, possibility of fulfill anticipated object with high quality and in time 0.074 Scientific and Technological meaning If they are science frontier, top technology, international hotspot, can fill blank field in China; if they can improve creativity and realize span development 0.044 Risk resistance of the project Scientific research is born somewhat risky.  However, considering project leader’ s ability and project implementing parts’ social, economic and political elements, if there are risk resistant measures and if there is hope to reach the goal successfully 0.032    2.2.2 ISTCP Mid-term Evaluation Comprehensive Indicator System Deviation may occur during the process of project execution due to changes of environment or executive persons themselves. If such deviation is not discovered in time it may influence the realization of the project object. Therefore it is necessary to tracking and evaluating projects and controlling the whole project process in time. At the same time, judge the problems and their influence to the direction of the projects. To guarantee healthy development of the projects and enhance the success rate, we have built ISTCP mid-term evaluation indicator system. Table 2: ISTCP Mid-term Evaluation Comprehensive Indicator System. Indicator Evaluation Standards Indicator Value Resource utilization status HR and equipment investment of both sides, outlay’ s carrying out status, if the outlay is used rationally, if is in accordance with plan, if it is truly used in the projects 0.127 Project phase achievement If the projects are fulfilled as expected, if quantity and quality of themes, new products, new material, new equipment and patents meet the anticipation 0.200 Expected completion of Project  Inferring the possibility of project’ s successful fulfillment from current process 0.131 Project management How complete the management regulations and financial systems of both parts are, and how effective they are during the implementation 0.257 Intercommunication system How complete the coordination systems of both parts are, how smooth the communication is, how effective the system is during the implementation 0.128 Patent protection Sharing of achievement, patent application, ownership of copyright, protection of resources and other cooperative benefits 0.157 2.2.3 ISTCP effect evaluation comprehensive indicator system A very important step of a project is the final effect evaluation. The evaluation should generalize the achievement of the project, and compare to the object of project establishment to determine the success of the project.  Effect evaluation can also be reference for future project establishment evaluations.  The result can also help best allocating of resources and guarantee the maximum proceeds of ISTCP.  The evaluation can also reflect the experience and lessens of the decisions made for the projects and enhance the level of project selection and project management. Formatted: Font:(Asian) Chinese (PRC)wu ! 3/14/06 10:57 AM  Table 3: ISTCP effect Evaluation Comprehensive Indicator System. Indicator  Evaluation Standards Indicator Value Science and technology level The Research achievements meet what scientific and technological level (lead internationally, equal to international level, lead in China, common in China, lower than China’ s level). Quantity and quality of themes and research reports, new products, new material, new technology, new process, and patents 0.155 Evaluation on Creativity Academic and technological creation.  Academic creation means to build new concepts discovering new objects, new disciplines, build new paradigms,  new theories,  new methods.  Technological creation means create or complete current production methods and measures, degree of adopting new theories, new technologies and advanced equipment 0.149 Evaluation On Practicability The advantage of achievement to be transferred to productivity, and usability in production 0.124 Scientific and Technological benefit To achieve valuable scientific and technological data, equipment, sample, resources and latest science and technology information; improve China’ s scientific research level and realize span development; enhance China’ s scientific project management level, push the build of new creativity system 0.073  Social benefit Research achievement should benefit the material culture and spiritual culture, strengthen the potential and science and technology, social and nation defense development, protect natural environment and bio-system, and enhance China’ s international reputation 0.030  Economical benefit The research achievement should improve social productivity and increase GDP, strengthen international business tie, build new trading partnership 0.036  Investment to productivity ratio Comparing to China and abroad’ s similar projects, if this project has advantage in the investment (HR, finance and equipment) and output (themes, research report, patents, new products, new material, new technology, social and economic benefits ) 0.039  Talent training Talent importing status; talent exporting status, numbers of lecturing experts, researching personnel’ s achieving new knowledge and enhancing project management ability; subject and field leaders’ being trained 0.079  Project progress Project be fulfilled according to planned timetable and progress of the goals in cooperation plans (fulfilled in advance, fulfilled, not fulfilled) 0.155  Organization managing level The completion of coordination system, the smoothness of mutual communication, the effectiveness of the system implement, the healthiness of financial system. 0.096  Patent protection Achievement sharing, patent application, copyright adscription, resources and other benefit protection status 0.064  3. ISTCP Evaluation System Platform Utilization of modern technology especially computer technology to automatic management is great extension of managing methods and revolution of modern management.  ISTCP evaluation system needs a good evaluation platform, thus we built an ISTCP evaluation platform based on Internet.  It   will greatly improve ISTCP managing efficiency and level, and bring China’s international cooperation to a new ground.  3.1. Brief introduction to ISTCP project evaluation platform The ISTCP evaluation platform has already developed the some functions: project establishment evaluating system; ISTCP mid-term evaluating system; ISTCP effect evaluating system; indicator managing system; user managing system; project managing system.  The website is:http://168.160.12.24/sub/  Here the function and guide of ISTCP evaluation platform are introduced, with example being ISTCP establishment evaluation system. The evaluating system of ISTCP has the following interface. The experts log on http://168.160.12.24/sub/ with their user names and passwords, select projects to be evaluated, then they enter the evaluating system. Grading standards for each indicator are linked by clicking the superlink following that indicator.  The grade of each indicator is filled into the blank (full grade is 10), and click “finish” after giving the grades.  Evaluation system backup database will automatically calculate the grades given by this expert, but he can not see the grade given by other experts.  The administrator, after logging in, can see the grades given by all experts. Fig. 2: ISTCP project evaluation platform  Other evaluating systems have similar operations and functions, and are not to be introduced on by one here. Platform of ISTCP automatizes partial works, enhances efficiency, and standardizes and systematizes the daily management of ISTCP.  It also helps with the tracking and evaluation of projects.  The database system realizes easy indicator and inquiry, making latest progress be available at once, and history data be referred for the selection and evaluation of new project.  References  1. Xu MK, Inspiration of the system of scientific and technological evaluation of developed countries, SCIENCE-TECHNOLOGY AND MANAGEMENT, 2005, 7 (6) 2. Bai Ning, International Science & Technology Cooperation projects’ management of CAS,  International S&T Cooperation, 2003 (3) : 34-36 3. Japanese S?????? and Technology Basic Plan (2001-2005), International S&T Cooperation, 2003 (2) : 35-43 4. Zhang, Ping, S& T Evaluation in Japan and Its Use for Reference to China, FORUM ON SCIENCE AND TECHNOLOGY IN CHINA, 2005 (2) 5. Liu Yun, International Science & Technology Cooperation in New Times, Forum on Science and Technology in China 2003 (3) : 3-6 6. Liu Yun, Capitalization And Commodity of Scientific And Technological Achievement And Protection of Intellectual Property Right, Science and Technology Management Research 1999,17 (5) 5-9 7. Dai Yanjun, The Present Situation and Countermeasures of China's Sci- tech International Cooperation, SCIENCE OF SCIENCE AND MANAGEMENT OF S.&.T. 2001, 22 (12) 8. FU Jian-qiu, The New Trend of Science- Technology Cooperation and in Development of China, TECHNOLOGY AND INNOVATION MANAGEMENT, 2005, 26 (1) 9. Zhao HuanCheng, Analytical Hierarchy Process, Beijing: Science Publishing, 1986 10. Current status of the METI technology evaluation, technology evaluation and research division, industrial science and technology environment bureau. Technology Evaluation and Research Division. April, 2002.   11. UNDP/EO.A Handbook for Programme Managers: Results Oriented Monitoring and Evaluation . 1999.  

Study on evaluation of International Science and Technology Cooperation Project (ISTCP) in China

E-LIS repository

  
STUDY ON EVALUATION OF INTERNATIONAL 
SCIENCE AND TECHNOLOGY COOPERATION 
PROJECT (ISTCP) IN CHINA 
SU CHENG, GUO HONG, MA ZHENG, YU ZHENGLU, PAN YUNTAO 
 
Institute of Scientific and Technical Information of China (ISTIC) 
No.15 Fuxing Road, Beijing 100038, the People’s Republic of China 
Abstract 
This paper presents an overview of evaluation of ISTCP in China.  We discuss briefly the history of evaluation 
and the strengths and weaknesses of different assessment systems.  On this basis, with Analytical Hierarchy 
Process (AHP), we establish evaluation indicator system for ISTCP that includes research project establishment 
evaluation, mid-period evaluation system, ?????? evaluation system, and confirm the value of each indicator.  
At the same time, we established expert database, project database, research organization database, researcher 
database etc.  We therefore establish an evaluation platform for international science and technology 
cooperation project.  We use it to realize full process supervision from evaluation expert selection to project 
management.  
1.  Survey on Evaluation of ISTCP in China 
Today, the economic globalization, R&D internationalization and technology and economy unification 
have converged to irreversible trend, which is even broadened and deepened by China’ s entry to 
WTO.  
Abrupt increase of ISTCP in China is demanding more efficiently scientific and standardized 
management, of which the scientific evaluation towards the standardization of projects is an important 
part.  
 ISTCP backed by foreign government are evaluated according to different supporting countries, 
following their procedures and measures.  Recent years many organizations start to make deliberation 
process and measures for themselves, and with best of their knowledge work out standardized project 
selection basis.  There are also organizations or institutions stipulated some ordinances and rules to 
comprehensively and scientifically manage the project establishment appraisal and execution 
verification. 
 Following is the introduction of ISTCP evaluation methods adopted by Chinese Academy of 
Sciences (CAS) Academy of Science & Technology of Ministry of Aeronaut & Astronaut 
(ASTMAA) China State Shipbuilding Corporation (CSSC) Yun Nan Science & Technology 
Commission (YNSTC) 
1.1. CAS 
ISTCP management of CAS includes project macro management, project organization and selection, 
project tracking management and evaluation and establishment of project management database, etc. 
  
On project establishment appraisal, it stipulates the principles of project establishment, appraisal 
regulations, appraising items, appraising procedures, appraising standards, appraising personnel 
selecting conditions and so on.  It also builds appraising expert teams according to such main 
subjects as mathematics, physics, chemistry, biology, technology and geosciences.  These teams 
“discuss among collectivity while grade individually”. 
 On project tracking management and evaluation, it takes part in middle and long term planning and 
study of the important projects, putting forward cooperation suggestions according to the characters of 
the projects, and providing necessary information and allowance.  It takes part in mid-term evaluation 
and progress evaluation until getting the final achievements. 
1.2. ASTMAA 
ASTMAA in practice has formed effective project management ways.  It masters chains like careful 
topic selection, plan carrying out, and periodical evaluation and so on.  It has built stably consisted 
international cooperation topic appraisal committee and reviews the execution of the cooperative 
projects last year.  It is improving the cooperation level continually. 
1.3. CSSC 
CSSC emphasizes the core of cooperation and communication.  It strictly checks the project from 
establishment and supervises the whole process.  The projects are appraised timely and the 
headquarter holds intercommunication conference every 2~3 years.   
1.4. YNSTC 
YNSTC has founded "ISTCP plan" and worked out “YNSTC ISTCP Allowance Management 
Guideline”, stipulating managing regulations on allowance application, project appraisal, contract, 
funds allocation, checking, achievement appraisal, achievement publication and so on. It invites 
Chinese and foreign experts and entrepreneurs to take part in achievement appraisal.  For projects 
with trading potential, it uses the market compatibility after their being put into the market as an 
important basis to appraise them. 
 In total in China there has not formed unified and standardized project management and is still 
lacking in scientific appraising methods.  Problems often seen are: although most organizations have 
their procedure and methods to appraise the establishment of ISTCP, there are no rules or regulations, 
and no standardization of operation. Some organizations have certain randomicity in establishing a 
project, and have no specific requirement or methods for evaluation. 
2. Evaluation System of ISTCP in China 
2.1. Design of the Evaluation System of ISTCP in China 
We adopt ??? method, breaking down the evaluation of ISTCP to a sequential and terraced structural 
model.  There are 3 levels in this model. They are Based on large scale questionnaires, we held 
several expert proseminars. With Delphi method, we decided the object level, guide line level and 
indicator level of ISTCP evaluation system. When evaluation indicator system was initially decided, 
we invited experts to grade different indicators with level analyzing method combined with Delphi 
method. The value of each indicator is calculated by Expert Choice software with following flow 
chart. 
 
  
Pass 
 
 
 
Fig. 1:the flow chart of Design of the Evaluation System of ISTCP in China 
 
 
 
 
 
 
 
 
 
 
 
 
2.2. China ISTCP Evaluation System and the Evaluation Standards 
China ISTCP evaluation system consists 3 parts: ISTCP project establishment evaluation system, 
ISTCP mid-term evaluation system and ISTCP effect evaluation system. 
2.2.1  ISTCP Research Project Establishment Evaluation Comprehensive Indicator 
System 
ISTCP research project establishment evaluation comprehensive indicator system is very important for 
guaranteeing the project quality, fairness of evaluation and healthy development of International 
Science and Technology Cooperation.  
pass 
No pass 
No pass 
Expert Fill 
judging 
matrix 
Calculate 
single level 
value 
Calculate 
general level 
value 
General level check Comprehensi
ve evaluation 
Single level 
consistency check 
 
  
Table 1: ISTCP research project establishment evaluation comprehensive indicator system. 
Indicator Evaluation Standards Indicator 
Value 
Cooperating 
country selection 
Cooperating country’ s attitude and openness towards Chin and its social, political 
and economical environment 0.043 
Strategic 
relativity 
If the project is closely related to China’ s emphasized development fields and if it is 
listed in 863 plan, basic research plan and tackle key plans, if is listed in 
governmental ISTCP protocol.  If the project can resolve important social problem, 
economic problem, if it helps building up commercial ties and helping technological 
product and knowledge products and service to go to the international market. 
0.189 
Talent training 
Help China’ s scholars understand science and technology frontier and broaden field 
of vision; study advanced technology and operation of advanced equipment; learn 
advanced scientific project management experience; train the elite of scientific fields 
and import excellent talent 
0.096 
Political and 
diplomatic needs 
If it meets China’ s foreign policy object, improves international relationship and 
strengthen China’ s international influence 0.096 
Patent protection If it is related to achievement sharing, patent application, copyright ownership, 
resource and other right protection agreement 0.076 
Project 
allowance budge 
If allowance applied is reasonable; allowance allocation is reasonable; total project 
financial plan is reasonable 0.080 
Chinese part 
level 
If the academic and technological levels of the China part are advanced in that field in 
China; if the project principal of China part is the lead in that field in china; the basis 
and accumulation of China part is well compensative to foreign part; if both parts can 
form high quality cooperative team; if the China part is the best one for that task in 
China 
0.060 
Foreign part 
executive level 
If the academic and technological levels of the foreign part are advanced in that field 
in that country; if the project principal of foreign part is the lead in that field in china; 
the basis and accumulation of China part is well compensative to foreign part; if both 
parts can form high quality cooperative team; if the foreign part is the best one for that 
task in that country 
0.047 
Potential 
Influence 
If the cooperative achievement can accelerate creativity.  The cooperative 
achievement may lead to breakthrough progress in its scientific field and have 
influence to relative fields; it may have strategic influence to resolve significant social 
and economical problems 
0.064 
Rationality of 
Anticipated Goal 
The objects put up is definite and clear, appropriate and broad, guaranteeing 
feasibility while exceeding current latest development 0.099 
Cooperation Plan 
and Manner 
Cooperation plan, rationality of method, maneuverability, advanced research artifice, 
correctness of research routine and creativity; rationality of time arrangement, 
possibility of fulfill anticipated object with high quality and in time 0.074 
Scientific and 
Technological 
meaning 
If they are science frontier, top technology, international hotspot, can fill blank field 
in China; if they can improve creativity and realize span development 
0.044 
Risk resistance 
of the project 
Scientific research is born somewhat risky.  However, considering project leader’ s 
ability and project implementing parts’ social, economic and political elements, if 
there are risk resistant measures and if there is hope to reach the goal successfully 0.032 
  
 
2.2.2 ISTCP Mid-term Evaluation Comprehensive Indicator System 
Deviation may occur during the process of project execution due to changes of environment or 
executive persons themselves. If such deviation is not discovered in time it may influence the 
realization of the project object. Therefore it is necessary to tracking and evaluating projects and 
controlling the whole project process in time. At the same time, judge the problems and their influence 
to the direction of the projects. To guarantee healthy development of the projects and enhance the 
success rate, we have built ISTCP mid-term evaluation indicator system. 
Table 2: ISTCP Mid-term Evaluation Comprehensive Indicator System. 
Indicator Evaluation Standards Indicator 
Value 
Resource utilization 
status 
HR and equipment investment of both sides, outlay’ s carrying out status, if the 
outlay is used rationally, if is in accordance with plan, if it is truly used in the 
projects 
0.127 
Project phase 
achievement 
If the projects are fulfilled as expected, if quantity and quality of themes, new 
products, new material, new equipment and patents meet the anticipation 
0.200 
Expected completion of 
Project  
Inferring the possibility of project’ s successful fulfillment from current process 0.131 
Project management How complete the management regulations and financial systems of both parts 
are, and how effective they are during the implementation 
0.257 
Intercommunication 
system 
How complete the coordination systems of both parts are, how smooth the 
communication is, how effective the system is during the implementation 
0.128 
Patent protection Sharing of achievement, patent application, ownership of copyright, protection of 
resources and other cooperative benefits 
0.157 
2.2.3 ISTCP effect evaluation comprehensive indicator system 
A very important step of a project is the final effect evaluation. The evaluation should generalize the 
achievement of the project, and compare to the object of project establishment to determine the 
success of the project.  Effect evaluation can also be reference for future project establishment 
evaluations.  The result can also help best allocating of resources and guarantee the maximum 
proceeds of ISTCP.  The evaluation can also reflect the experience and lessens of the decisions made 
for the projects and enhance the level of project selection and project management. 
Formatted: Font:(Asian) Chinese (PRC)
wu ! 3/14/06 10:57 AM
  
Table 3: ISTCP effect Evaluation Comprehensive Indicator System. 
Indicator  Evaluation Standards Indicator 
Value 
Science and 
technology level 
The Research achievements meet what scientific and technological level (lead 
internationally, equal to international level, lead in China, common in China, lower than 
China’ s level). Quantity and quality of themes and research reports, new products, new 
material, new technology, new process, and patents 
0.155 
Evaluation on 
Creativity 
Academic and technological creation.  Academic creation means to build new concepts 
discovering new objects, new disciplines, build new paradigms,  new theories,  new 
methods.  Technological creation means create or complete current production methods 
and measures, degree of adopting new theories, new technologies and advanced 
equipment 
0.149 
Evaluation On 
Practicability 
The advantage of achievement to be transferred to productivity, and usability in 
production 
0.124 
Scientific and 
Technological 
benefit 
To achieve valuable scientific and technological data, equipment, sample, resources and 
latest science and technology information; improve China’ s scientific research level and 
realize span development; enhance China’ s scientific project management level, push the 
build of new creativity system 
0.073  
Social benefit Research achievement should benefit the material culture and spiritual culture, strengthen 
the potential and science and technology, social and nation defense development, protect 
natural environment and bio-system, and enhance China’ s international reputation 
0.030  
Economical 
benefit 
The research achievement should improve social productivity and increase GDP, 
strengthen international business tie, build new trading partnership 
0.036  
Investment to 
productivity ratio 
Comparing to China and abroad’ s similar projects, if this project has advantage in the 
investment (HR, finance and equipment) and output (themes, research report, patents, 
new products, new material, new technology, social and economic benefits ) 
0.039  
Talent training Talent importing status; talent exporting status, numbers of lecturing experts, researching 
personnel’ s achieving new knowledge and enhancing project management ability; 
subject and field leaders’ being trained 
0.079  
Project progress Project be fulfilled according to planned timetable and progress of the goals in 
cooperation plans (fulfilled in advance, fulfilled, not fulfilled) 
0.155  
Organization 
managing level 
The completion of coordination system, the smoothness of mutual communication, the 
effectiveness of the system implement, the healthiness of financial system. 
0.096  
Patent protection Achievement sharing, patent application, copyright adscription, resources and other 
benefit protection status 
0.064  
3. ISTCP Evaluation System Platform 
Utilization of modern technology especially computer technology to automatic management is great 
extension of managing methods and revolution of modern management.  ISTCP evaluation system 
needs a good evaluation platform, thus we built an ISTCP evaluation platform based on Internet.  It 
  
will greatly improve ISTCP managing efficiency and level, and bring China’s international 
cooperation to a new ground.  
3.1. Brief introduction to ISTCP project evaluation platform 
The ISTCP evaluation platform has already developed the some functions: project establishment 
evaluating system; ISTCP mid-term evaluating system; ISTCP effect evaluating system; indicator 
managing system; user managing system; project managing system.  The website 
is:http://168.160.12.24/sub/  Here the function and guide of ISTCP evaluation platform are 
introduced, with example being ISTCP establishment evaluation system. 
The evaluating system of ISTCP has the following interface. The experts log on 
http://168.160.12.24/sub/ with their user names and passwords, select projects to be evaluated, then 
they enter the evaluating system. Grading standards for each indicator are linked by clicking the 
superlink following that indicator.  The grade of each indicator is filled into the blank (full grade is 
10), and click “finish” after giving the grades.  Evaluation system backup database will automatically 
calculate the grades given by this expert, but he can not see the grade given by other experts.  The 
administrator, after logging in, can see the grades given by all experts. 
Fig. 2: ISTCP project evaluation platform 
 
Other evaluating systems have similar operations and functions, and are not to be introduced on by 
one here. 
Platform of ISTCP automatizes partial works, enhances efficiency, and standardizes and systematizes 
the daily management of ISTCP.  It also helps with the tracking and evaluation of projects.  The 
database system realizes easy indicator and inquiry, making latest progress be available at once, and 
history data be referred for the selection and evaluation of new project.
  
References  
1. Xu MK, Inspiration of the system of scientific and technological evaluation of developed countries, 
SCIENCE-TECHNOLOGY AND MANAGEMENT, 2005, 7 (6) 
2. Bai Ning, International Science & Technology Cooperation projects’ management of CAS,  
International S&T Cooperation, 2003 (3) : 34-36 
3. Japanese S?????? and Technology Basic Plan (2001-2005), International S&T Cooperation, 2003 
(2) : 35-43 
4. Zhang, Ping, S& T Evaluation in Japan and Its Use for Reference to China, FORUM ON SCIENCE AND 
TECHNOLOGY IN CHINA, 2005 (2) 
5. Liu Yun, International Science & Technology Cooperation in New Times, Forum on Science and 
Technology in China 2003 (3) : 3-6 
6. Liu Yun, Capitalization And Commodity of Scientific And Technological Achievement And 
Protection of Intellectual Property Right, Science and Technology Management Research 1999,17 
(5) 5-9 
7. Dai Yanjun, The Present Situation and Countermeasures of China's Sci- tech International 
Cooperation, SCIENCE OF SCIENCE AND MANAGEMENT OF S.&.T. 2001, 22 (12) 
8. FU Jian-qiu, The New Trend of Science- Technology Cooperation and in Development of China, 
TECHNOLOGY AND INNOVATION MANAGEMENT, 2005, 26 (1) 
9. Zhao HuanCheng, Analytical Hierarchy Process, Beijing: Science Publishing, 1986 
10. Current status of the METI technology evaluation, technology evaluation and research division, 
industrial science and technology environment bureau. Technology Evaluation and Research 
Division. April, 2002.   
11. UNDP/EO.A Handbook for Programme Managers: Results Oriented Monitoring and Evaluation . 
1999. 
 


http://eprints.rclis.org/7430/1/ISTCP.pdf

Study on evaluation of International Science and Technology Cooperation Project (ISTCP) in China

Abstract

Similar works

Full text

Available Versions

E-LIS

E-LIS repository