Faculty annual evaluation is critical for faculty career
development and department success. Our department
developed a quantitative faculty evaluation metrics system
that is more objective, consistent, flexible, personalized,
transparent, and dynamic scoring system. We have
implemented the system for 2 years and the outcomes are
very encouraging in promoting faculty and department
success.Cockrell School of Engineerin

Han, Hai-Chao

Wan, Hung-da

Wang, Xiaodu

English

Texas ScholarWorks

 Proceedings of the 2018 ASEE Gulf-Southwest Section Annual Conference The University of Texas at Austin April 4-6, 2018 Development of a Personalized Quantitative Faculty Annual Evaluation System Hai-Chao Han, Hung-da Wan, Xiaodu Wang Department of Mechanical Engineering, University of Texas at San Antonio, USA  E-mail: hchan@utsa.edu  Abstract Faculty annual evaluation is critical for faculty career development and department success. Our department developed a quantitative faculty evaluation metrics system that is more objective, consistent, flexible, personalized, transparent, and dynamic scoring system. We have implemented the system for 2 years and the outcomes are very encouraging in promoting faculty and department success.  1. Introduction Faculty annual performance evaluation is an important element of assessment in higher education and is critical for faculty career development and department success. It is essential for promoting excellence, by both encouraging good performance and providing positive feedback for improvement. Many departments develop their own evaluation rubrics or metrics to evaluate faculty on measures deemed very important for faculty and department success [1-3]. Department chairs and/or review committees reviews faculty performance based on evaluation rubrics to assign scores in the areas of teaching, research, and service and then calculate the overall score based on faculty efforts in these areas. However, the commonly used scoring methods are often more or less subjective in assigning scores for either individual activities or areas and even overall score. Our department has developed an objective and quantitative evaluation metrics and scoring system that 1) calculates score on each activity/measure using a formula that is based on expectations, 2) provides flexible range of weight for each measure, 3) focuses on key activities/ measures, and 4) considers both efforts and outcomes.  2. Methods      Our college policy requires that faculty workload being assigned based on their “tracks” which determines their effort distribution among teaching, research and service. A quantitative faculty performance expectations metrics was developed by faculty merit review committee based on faculty’s “rank”, “track” and workload policy of the college.  2.1 Selection of key measures Based on college and department goals and needs as well as faculty professional development needs, we selected key activity measures for the area of teaching, research, and service. For example, key measures for teaching include efforts in continuous improvement, student course evaluation, and student mentoring. Key measures for research include research expenditure, journal publication and graduate student support.   2.2 Score system      Per university guideline, we use a 5-point score system with 5 being the best. Specifically,  3.5 – 5.0 Exceeds Expectation 2.0 – 3.49 Meets Expectation 1.0 – 1.99 Fails to Meet Expectation 0 – 0.99 Unsatisfactory This scale was used for all individual measures, areas, and overall scores. For individual measures, the ranges of expectations were used to determine the performance scores using a linear interpolation equation of the upper and lower expectation values:  02491 ..*LE)(UELE)(XScore                       (1) where X is the performance value achieved (such as the number of publications), LE and UE are the lower and upper expectations, respectively, and 1.49 is the score range for meeting expectations (2.0-3.49). For measures with a given expectation target value, the score was determined by a linear interpolation of the expectation values (EV):   Score= (X- EV)/EV*1.49+2.0                  (2) A flexible (personalized) weighting scheme was used for calculating the area score from these measure scores. The weights of different measures were determined based on their importance in achieving department goals and needs and faculty career goals and needs. The most unique feature of our evaluation system is allowing individual faculty members to have certain flexibility in weighing the major measures in their performance. For example, in the area of research, a faculty member may put more focus on funding or journal publications based on their research program progress and needs, so they may weigh a little more on either funding or journal papers which they performed better for  Proceedings of the 2018 ASEE Gulf-Southwest Section Annual Conference The University of Texas at Austin April 4-6, 2018 the reporting year.  In teaching, a faculty member may give a slightly higher weight on student evaluation, class size or teaching improvement based on their effort and outcome. By setting different upper and low limits on the allowable weight ranges, the evaluation system highlights important measures and priorities for the department and faculty success.   Score of each area in teaching, research, and service were calculated as the weighted total of scores on the key measures. Overall evaluation score was then calculated based the scores in the 3 areas weighted by faculty member’s efforts distribution in the 3 areas.  3. Results To facilitate the implementation of the new system, a standard electronic spreadsheet form were developed to take faculty data input and programed to calculate the scores for individual measures and weighted scores with an expectations matrix that runs in the background. Using this electronic form, faculty members select rank and track, fill their performance data (and brief notes supporting the data), and immediately see the raw scores automatically calculated. They can adjust their preferred weight within the allowable range on each measure based on the raw score to optimize their scores in each area of teaching, research and service. The overall score is then automatically calculated based on the scores and effort distribution in these areas. Upon individual faculty members submitting their report, the committee reviews the forms and makes any necessary adjustment and correction (often very minimal) and confirms the final evaluation reports with the faculty members.    4. Discussion and Conclusion We developed a quantitative faculty evaluation system that focuses on key activity measures that are important to department needs and goals and faculty success.  The objective scoring method is a big advantage of this evaluation system. The scores are objectively calculated, not at the mercy of department chair or faculty merit review committee. The evaluators do not have to knock their heads to decide between two scores to split hairs and worry about subjectivity. It provides valuable data that makes it easier for department chair and dean to make merit raise decisions on an objective basis.  Our follow-up survey showed that the flexible individualized weights on individual measures is a unique feature most appreciated by faculty. This flexibility also eases the challenging issue of comparing different measures.    Vagueness can often make faculty members feel unfairly evaluated with subjectivity, which would beat the purpose of evaluation. The transparency and clarity of the system averts “unfairness” complains. In addition, the system combines qualitative and quantitative measures. Teaching and research excellence are encouraged with some measures and bonus points in the system. In conclusion, a personalized quantitative faculty evaluation metrics and score system is developed and successfully implemented in our department. It is an objective, consistent, quantitative, flexible, transparent, and dynamic scoring system that reflects faculty performance in three areas (i.e., teaching, research and service). This evaluation system has provided us a valuable base for making better decisions on merit raises and other awards decisions. Our Faculty productivity has significantly increased over the past two years with the implementation of this evaluation system.  This evaluation system can be easily adapted to meet the evolving specific goals, needs and culture of an academic department or tailored to distinct needs and culture for different departments. The major measures, expectation range, and scoring weights, as well as workload distribution can be determined based on the goals of individual departments.  Acknowledgement      We thank the faculty of department of Mechanical Engineering for their support, feedback and participation in using the current evaluation metrics system.  References 1. Wiegers, S.E., S.R. Houser, H.E. Pearson, A. Untalan, J.Y. Cheung, S.G. Fisher, L.R. Kaiser, and A.M. Feldman, A Metric-Based System for Evaluating the Productivity of Preclinical Faculty at an Academic Medical Center in the Era of Clinical and Translational Science. Clin Transl Sci, 2015. 8(4): p. 357-61. 2. Hardre, P. and M. Cox, Evaluating faculty work: expectations and standards of faculty performance in research universities. Research Papers in Education, 2009. 24(4): p. 383-419. 3. Fuhs, F.P., H.R. Weistroffer, M. Spinelli, and G. Canavos, Annual faculty output evaluation - A structured approach based on quality and quantity. Decision Sciences Institute, 1997 Annual Meeting, Proceedings, Vols 1-3, 1997: p. 467-469.  

Development of a Personalized Quantitative Faculty Annual Evaluation System

https://repositories.lib.utexas.edu/bitstream/2152/79872/2/development-of-a-personalized-quantitative-faculty-annual-evaluation-system.pdf

Development of a Personalized Quantitative Faculty Annual Evaluation System

Abstract

Similar works

Full text

Available Versions

Texas ScholarWorks