The Electronic Lecturer Evaluation System (E-LEva) is an online system to evaluate the performance of UiTM Pahang lecturers at the end of every semester. The main users of e-LEva are students, the Academic Affairs (HEA) Department of UiTM Pahang and the lecturers. The main purpose of e-LEva is to let the students have a better platform to evaluate their lecturers. The HEA can immediately retrieve the statistics of the evaluation and lecturers can identify the level of the students’ satisfaction regarding the teaching and learning process. This information is also important to inform the lecturers of the quality of their teaching styles implemented in the classroom. The previous traditional process was very complicated and tedious; the process of data compilation itself took more than three months. Thus, the implementation of e-LEva helps the management especially the HEA of UiTM Pahang in reducing time, resources and cost. e-LEva was developed based on the specific requirements identified by the HEA. To fulfill the requirements of the university, academics and students, a lot of improvements have been done on e-LEva within the two years of its implementation. The objective of this paper is to study the feedback of the lecturers in order to investigate the level of acceptance of e-LEva. Data were collected via an online survey questionnaire

Abd Halim, Khairul Nizam

Razali, Razulaimi

English

Universiti Teknologi MARA Institutional Repository

35
Moving Towards Electronic Lecturer
Evaluation System (E-LEva) in UiTM
Pahang: A Study Across Lecturers
Khairul Nizam Abd Halim
Razulaimi Razali
Faculty of Information Technology and Quantitative Sciences
Universiti Teknologi MARA (UiTM), Malaysia
Email: khairulnizam@pahang.uitm.edu.my
Email: razulaimi@pahang.uitm.edu.my
ABSTRACT
The Electronic Lecturer Evaluation System (e-LEva) is an online
system to evaluate the performance of UiTM Pahang lecturers at
the end of every semester. The main users of e-LEva are students,
the Academic Affairs (HEA) Department of UiTM Pahang and the
lecturers. The main purpose of e-LEva is to let the students have a
better platform to evaluate their lecturers. The HEA can immediately
retrieve the statistics of the evaluation and lecturers can identify
the level of the students’ satisfaction regarding the teaching and
learning process. This information is also important to inform the
lecturers of the quality of their teaching styles implemented in the
classroom. The previous traditional process was very complicated
and tedious; the process of data compilation itself took more than
three months. Thus, the implementation of e-LEva helps the
management especially the HEA of UiTM Pahang in reducing time,
resources and cost. e-LEva was developed based on the specific
requirements identified by the HEA. To fulfill the requirements of
the university, academics and students, a lot of improvements have
been done on e-LEva within the two years of its implementation.
The objective of this paper is to study the feedback of the lecturers
Gading Business and Management Journal Vol. 12 No. 2, 35-46, 2008
ISSN 0128-5599
© 2008 Universiti Teknologi MARA (UiTM), Malaysia
36
Gading Business and Management Journal
in order to investigate the level of acceptance of e-LEva. Data were
collected via an online survey questionnaire.
Keywords: e-LEva, education, evaluation in education, lecturer
evaluation, teaching evaluation
Introduction
e-LEva has been developed and implemented since 2006. The
improvement of e-LEva is made based on responses from the users. It
is essential in utilising teacher’s evaluation as a vehicle to improve
teaching and learning. An argument is made that an effective teacher
evaluation system, combined with strong leadership, facilitates the type
of learning organization. Today, learning organisation exists in rapidly
changing and increasingly complex society, so university must be able to
rapidly react, respond and adapt (Davis, Ellett and Annunziata, 2002).
By having e-LEva as the platform to evaluate teaching, UiTM Pahang
will be more efficient in improving the constructive method or system.
Within two years, the users’ feedback on the performance, reliability
and accuracy of e-LEva had been identified. There is a need to make
sure that e-LEva will continuously improve. This will create a good impact
to UiTM Pahang as a whole.
Evaluation is an important part in the process of learning. It has to
do with finding out from our students about the quality of their learning
and obtaining information about the effectiveness of our teaching (The
Evaluation of Learning and Teaching, 2002). In education, teaching
evaluation has been discussed seriously in other countries in order to
identify proper methods, tools and frameworks to do the evaluation.
Teaching evaluation involves more than one evaluator. Using multiple
data sources can provide more accurate measures of the teaching
performance, thus, enhancing the quality of teacher evaluation. The types
of evaluation in teaching are self-evaluation, peer evaluation, and student
evaluation of teachers (Liu and Teddlie, 2005). The teaching evaluation
discussed in this paper concerns the evaluation of the lecturers assessed
by their students. This evaluation is part of the total evaluation on the
lecturers. The evaluation process cannot solely depend on the students
because students might not understand their purpose of evaluating the
lecturers; their role and the importance of the evaluation. Also, some
students might perceive the evaluation as an opportunity to take revenge
37
Electronic Lecturer Evaluation System (E-LEva)
on a lecturer for a variety of reasons (Liu & Teddlie, 2005). If these
instruments are used in isolation, as they frequently are, and without
alternative and collaborative measures, then students become the primary
determinant of the lecturer’s success or failure in his or her academic
career (Charles, Tracy and Robert, 2003).
Students’ evaluation of teaching performance is a key component in
getting feedback for continuous improvement of educational programmes
(Andersen, 2006). Evaluation is important in order to bring about
improvements in areas such as student achievements, use of public funds
or educational materials and programs. Different evaluation approaches
exist depending on who requires the information and the purpose for
which the information is needed (Moses Waithanji & Mwangi, 2005).
Evaluation has multiple functions deriving from the purposes of the
evaluation activity. The three main functions of educational evaluation
are accountability, certification and learning. These functions are also
embedded in the evaluation of teaching but they need to be expanded
and more clearly defined (Smith, 2005). Most universities solicit feedback
from students at the end of a semester in order to assess students’
perceptions on the conduct of the subject (Bhattacharyya, 2004).
Traditionally, questionnaires have been given to students to be completed
on paper. For lecturer evaluation, this can mean a major logistical exercise
and a considerable expense for the university. Printing the questionnaires
can be a complex exercise and some of the items on the questionnaire
can be chosen by lecturers. Administering questionnaires in class takes
a lot of time and organisational effort. Collecting feedbacks from students
has become a costly and time consuming process for most universities.
Converting to data collection through the Internet, rather than completion
on paper, can result in a cheaper and more efficient process (Leung and
Kember, 2005). An important concern in the research was determining
the reliability of the tools used in evaluating teaching effectiveness.
Reliability refers to the extent to which a measuring tool is consistent
and stable in measuring what it is supposed to measure. This is important
if the results of the measuring tools are to be taken as an unbiased
representation of the characteristics being measured (Moses Waithanji
and Mwangi, 2005). In the case of e-LEva, high reliability would indicate
that students can objectively evaluate their lecturers; hence, the results
can be confidently used in decision-making.
38
Gading Business and Management Journal
e-LEva Framework
UiTM Pahang uses e-LEva as a vital part of the total assessment of a
lecturer’s performance and is documented every semester in the Teaching
Portfolio. Previously the lecturer evaluation was done manually which
involved eleven steps in total and took at least three months to complete
(Khairul Nizam Abd Halim and Razulaimi Razali, 2008). Figure 1 show
all the steps involved in the manual process. After years of manually
processing lecturers’ performance evaluation in the context of learning
and teaching in UiTM Pahang, problems surfaced in relation to time,
resources and cost. The problems identified in the manual process were:
1. The manual process took about three months to produce the result.
2. The process of gathering the data, analysing the data and producing
the results needed the involvement of many parties.
3. The management (HEA) needed to bear costs on the overtime,
stationery and papers.
4. The accuracy of the data being used for the analysis and results.
The problem becomes more serious every semester. A solution was
identified to overcome the problems. A new framework of the evaluation
process was designed to ensure a more productive and effective
evaluation system. Together with the framework a new platform was
also designed to support the framework and the ‘Electronic Lecturer
Prepare
Question
Circulation
Management
Circulation Evaluation
Analysis Data Entry Compilation Submission
Produce
Report
Verification
Result
Circulation
  

 


Figure 1: Manual Process for Teaching and Learning Evaluation
(Khairul Nizam Abd Halim, 2006)
39
Electronic Lecturer Evaluation System (E-LEva)
Evaluation System’ or better known as e-LEva. This application is a
web-based application accessible from anywhere that has internet
access.
As the owner of this system, the HEA is authorised to control the
access of the screen for students and lecturers. Based on the e-LEva
framework, the HEA needs to set the questions for the evaluation. The
next process, data for students, lecturers and subjects for a particular
semester are channeled into the e-LEva database from the existing UiTM
system ISIS (student information system) and ICRESS (Lecturer and
subject information system). Once the extraction process is completed,
the HEA needs to verify the data. Then, the HEA allows the screen to
be accessed for students to do the evaluation while monitoring the
evaluation percentage. The monitoring is important to make sure that
the majority of students evaluate their lecturers. This exercise is continued
based on the time duration identified by the HEA. The lecturers can gain
access to the system once the evaluation process is completed within
the time duration given. At this moment the HEA and lecturers can
retrieve the results from e-LEva. The framework and the process flow
of e-LEva are shown in Figures 2 and 3.
 
HAL EHWAL
AKADEMIK (HEA)
TYPE OF 
REPORT
STATISTICAL
ANALYSIS
e -LEva 
DATABASE
iSIS
ICReSS
SUBJECT
REPORT
PROGRAM
REPORT
LECTURER
GRADE
CAMPUS
REPORT
GOOD
EXCELLENT
VERY
WEAK
WEAK
MODERATE
Figure 2: e-LEva Framework (Khairul Nizam Abd Halim, Razulaimi Razali,
& Roslan Jamaludin, 2006)
40
Gading Business and Management Journal
Method of Study
In fulfilling the purpose of this paper, a questionnaire was administered
to a total sample of 104 lecturers from UiTM Pahang. The questionnaire
Figure 3: The Process Flow of e-LEva (Khairul Nizam Abd Halim
& Razulaimi Razali, 2008)
 
 
Identify Semester
Prepare
Evaluation
Question
Check Lecturer 
& Subject 
Status
Update
Open Student 
Screen
Student Evaluate
Monitor Evaluation 
Status
View
Report
Start
End
No
Yes
41
Electronic Lecturer Evaluation System (E-LEva)
was prepared in a web-based form where all the lecturers can access it
through the Internet to participate in the survey. The information regarding
the survey and the instructions for the lecturers to participate in the
survey were communicated through e-mail. Reminders on the survey
were also sent through e-mail every week for three weeks of the survey
to make sure that at least 50% of the lecturers participate in the survey.
From the survey the writers expected to get responses from the lecturers
as users of e-LEva. The questionnaire had eighteen questions including
five questions on the demographic information of the respondents.
All data that were retrieved from the survey were imported to SPSS
version 16 for the analysis process. Two types of analysis were done:
i) descriptive analysis, and ii) frequency analysis. The main purpose of
the descriptive analysis is to identify the mean for the thirteen questions
in the questionnaire. From the results, the researchers will know whether
e-LEva is suitable and acceptable for teaching evaluation in UiTM
Pahang. The frequencies analysis is to identify the percentage of the
respondents in the first five questions in the questionnaire.
Results
Table 1 shows that 20.2% of the respondents were from the Faculty of
Applied Sciences and they represented the highest percentage compared
to other respondents. This is followed by the Faculty of IT and Quantitative
Sciences (17.3%), Language (13.5%), Faculty of Accountancy and
Table 1: Respondents and Department
Frequency Percent Valid Cumulative
Percent Percent
Valid Islamic Thought 7 6.7 6.7 6.7
Language 14 13.5 13.5 20.2
Applied Science 21 20.2 20.2 40.4
Accountancy 12 11.5 11.5 51.9
Business Management 12 11.5 11.5 63.5
Civil Engineering 9 8.7 8.7 72.1
Office Management and Technology 11 10.6 10.6 82.7
IT and Quantitative Sciences 18 17.3 17.3 100.0
Total 104 100.0 100.0
42
Gading Business and Management Journal
Table 2 shows the percentage of respondents based on the years of
service at the university. A majority of the respondents (30.8%) has 5 –
10 years working experience. This is followed by 27.9% with more than
10 years, 22.1% with 1 – 2 years working experience, 10.6% with 2 – 5
years and 8.7% of the respondents with less than 1 year working
experience.
Table 3 shows the percentage of respondents based on their academic
positions. Lecturers formed the largest percentage (65.4%), followed
by Young Lecturers (22.1%), Senior Lecturers (8.7%), Associate
Professors (2.9%) and Professor (1%). This conforms that the population
of lecturers in UiTM Pahang are currently dominated by the new
lecturers who have worked less than ten years. Table 4 shows the
percentage of respondents based on the status of job where the majority
of the respondents are permanent lecturers (86.5%).
Faculty of Business Management (both 11.5%), Faculty of Office
Management and Technology (10.6%), Faculty of Civil Engineering
(8.7%) and Islamic Thought (6.7%).
Table 2: Respondents and Years of Service
Frequency Percent Valid Cumulative
Percent Percent
Valid Less than 1 year 9 8.7 8.7 8.7
1-2 years 23 22.1 22.1 30.8
2-5 years 11 10.6 10.6 41.3
5-10 years 32 30.8 30.8 72.1
More than 10 years 29 27.9 27.9 100.0
Total 104 100.0 100.0
Table 3: Respondent Percentage Based on Position
Frequency Percent Valid Cumulative
Percent Percent
Valid Young Lecturer 23 22.1 22.1 22.1
Lecturer 68 65.4 65.4 87.5
Senior Lecturer 9 8.7 8.7 96.2
Associate Professor 3 2.9 2.9 99.0
Professor 1 1.0 1.0 100.0
Total 104 100.0 100.0
43
Electronic Lecturer Evaluation System (E-LEva)
Table 4: Respondents and Status of Job Position
Frequency Percent Valid Cumulative
Percent Percent
Valid Permanent 90 86.5 86.5 86.5
Contract 7 6.7 6.7 93.3
Full Time 5 4.8 4.8 98.1
Part Time 2 1.9 1.9 100.0
Total 104 100.0 100.0
Table 5 shows the descriptive summary of the thirteen criteria of e-
LEva. The statistics shows the impact of e-LEva on the lecturers
especially on the level of their acceptance towards the implementation
of e-LEva. Thirteen criteria were evaluated to identify the results. The
criteria were:
a. Ease at use of e-LEva
b. Clarity and effectiveness of the flow
c. Simple to use e-LEva
d. Comfortable to use e-LEva
e. User friendliness of using e-LEva
f. Productivity issues relating to e-LEva
g. Reliability
h. Consistency of e-LEva in meeting academicians’ expectation
i. Immediacy of producing results
j. Performance of e-LEva
k. Level of satisfaction on e-LEva
With reference to Table 5, the mean value for all the scales surveyed
is above 4.0. The result shows that e-LEva is generally accepted by
the academics. Even though the results show positive feedback from
the respondents, certain areas of e-LEva still need improvement to
make sure they really work for all users in UiTM Pahang. The lowest
mean value is the reliability of e-LEva with the value of 4.03. Further
studies need to be done to study the reliability of the system. The
reliability of e-LEva needs to be identified to make sure that it can be
improved. The reliability of the system is very important because the
system is being used as a platform for teaching evaluation as mentioned
before. Table 5 also shows that e-LEva is user friendly and well
accepted by the users.
44
Gading Business and Management Journal
The following conclusions reflect academics’ acceptance towards
the use of e-LEva:
a. Implementation of e-LEva is being accepted by the academicians.
b. e-LEva is user friendly.
c. The reliability of e-LEva still needs further study.
d. Ongoing studies and improvements on e-LEva need to be done to
make sure that the system always comply with UiTM Pahang’s
objectives.
Conclusion
Evaluation is not something new in education. In other countries,
universities, schools, colleges and other education bodies, research and
studies are being done to find the best method on teaching evaluation.
Besides the method, tools as the platform of the evaluation process are
also important to be identified to make sure the evaluation process can
be done smoothly and effectively. The implementation of e-LEva gives
positive impact on UiTM Pahang, especially on the management side.
Results from the study also show that e-LEva is positively accepted by
Table 5: Descriptive Summary of the 13 Scales of e-LEva
N Minimum Maximum Mean Std. Deviation
easy to use 104 1 5 4.44 .680
flow 104 1 5 4.38 .780
simple 104 2 5 4.52 .653
comfortable 104 1 5 4.38 .827
easy to learn 104 3 5 4.63 .561
productive 104 1 5 4.19 .915
reliable 104 1 5 4.03 1.065
consistency 104 1 5 4.12 .889
immediate result 104 2 5 4.36 .762
performance 104 1 5 4.37 .791
satisfied 104 1 5 4.26 .903
use 104 1 5 4.36 .799
recommend 104 1 5 4.31 .825
Valid N (listwise) 104
45
Electronic Lecturer Evaluation System (E-LEva)
the academics. Even though the results show positive feedback,
improvement on e-LEva has to be done continuously to make sure that
e-LEva will always fulfill UiTM Pahang’s objectives.
References
Andersen, K. (2006). e-Participation Behind Closed Doors: Online
Evaluation of Teaching Performance. Electronic Government.
Bhattacharyya, N. (2004). Student Evaluations and Moral Hazard.
Journal of Academic Ethics, 2(3), 263-271.
Charles, R. E., Tracy, R. K. & Robert, G. T. (2003). Return to academic
standards: a critique of student evaluations of teaching effectiveness.
Quality Assurance in Education, 11(1).
Davis, D. R., Ellett, C. D. & Annunziata, J. (2002). Teacher Evaluation,
Leadership and Learning Organizations. Journal of Personnel
Evaluation in Education, 16(4), 287-301.
The Evaluation of Learning and Teaching. (2002). A Handbook for
Medical Teachers.
Khairul Nizam Abd Halim. (2006). Satu Kajian Kes: Operasi E-Leva
di UiTM Pahang. Paper presented at the Kolokium 2004-2006
UiTM Pahang.
Khairul Nizam Abd Halim, & Razulaimi Razali. (2008). A Framework
For Electronic Lecturer Evaluation System (e-LEva). Paper
presented at The National Seminar on Science, Technology and Social
Sciences.
Khairul Nizam Abd Halim, Razulaimi Razali & Roslan Jamaludin. (2006).
e-LEva Medium Penyelesaian Masalah Pengurusan Sumber
Manusia Akademik. Paper presented at the Konferensi Akademik
2006 UiTM Pahang.
46
Gading Business and Management Journal
Leung, D. Y. P. & Kember, D. (2005). Comparability of Data Gathered
from Evaluation Questionnaires on Paper and Through the Internet.
Research in Higher Education, 46(5), 571-591.
Liu, S. & Teddlie, C. (2005). A Follow-up Study on Teacher Evaluation
in China: Historical Analysis and Latest Trends. Journal of
Personnel Evaluation in Education, 18(4), 253-272.
Moses Waithanji, N. & Mwangi, N. (2005). An improvement in
instructional quality: can evaluation of teaching effectiveness make
a difference? Quality Assurance in Education, 13(3).
Smith, K. (2005). New methods and perspectives on teacher evaluation.
In Teacher Professional Development in Changing Conditions
(pp. 95-114).


Moving towards electronic lecturer evaluation system (E-LEva) in UiTM Pahang: A study across lecturers/Khairul Nizam Abd Halim, Razulaimi Razali

http://ir.uitm.edu.my/id/eprint/29845/1/29845.pdf

Moving towards electronic lecturer evaluation system (E-LEva) in UiTM Pahang: A study across lecturers/Khairul Nizam Abd Halim, Razulaimi Razali

Abstract

Similar works

Full text

Available Versions

Universiti Teknologi MARA Institutional Repository