The use of web-based tests delivered through learning management systems has grown at university  level in the last years. One of their key advantages is the possibility of creating tests with some degree  of randomness that are automatically assessed in real time. Although the access to the learning  management system resources is controlled for each student by means of personal username and  password, the cheating among students when doing the tests cannot be avoided. However, if the  students finally learn, in spite of cheating, the process could still be considered to be successful. In  this work, the date, the required time to solve the test and the grades of quizzes undertaken by  students through a web based learning management system are analyzed and they are compared to  the grades obtained by the same students in a written test solved in an examination classroom under  the supervision of the teacher. The course in which this study has been developed (Signals and  Systems for Electrical and Electronics Engineering undergraduate students) is organized in 5 subjects  and the students make a quiz on the web for each subject. At the end of the course the students make  a final written exam that includes a true/false test. Around 50 questions for each subject of the course  have been created. The questions are organized in 5 to 8 categories for each subject. The learning  management system generates quizzes by arbitrarily selecting 1 or 2 items from the 5 to 8 categories  in a given subject to complete a 10-item quiz. Due to the reduced number of items for each category  and the large number of students that attend the course, several questions are repeated in quizzes  generated for different students. The authors have noticed that some students work in groups to solve  the quizzes. Some of them answer all the questions in a quiz in few minutes (less than 20 % of the  time used by the most of their mates) and obtain high scores. When the scores of the same students  in the final exam are analyzed, it is found that they also obtain good results. Then, it could be  concluded that although they have found a way of cheating to solve the web quizzes, this is still  pedagogically valid because they have learnt about the subject (they also obtain good results in the  written test)

Arriero Encinas, Luis

Fraile Muñoz, Rubén

Godino Llorente, Juan Ignacio

González de Sande, Juan Carlos

Osma Ruiz, Víctor

Osés del Campo, David

Servicio de Coordinación de Bibliotecas de la Universidad Politécnica de Madrid

CHEATING AND LEARNING THROUGH WEB BASED TESTS 
Juan Carlos de Sande, Rubén Fraile, Luis Arriero, Víctor Osma, David Oses, 
Juan Ignacio Godino 
Universidad Politécnica de Madrid (SPAIN) 
jcgsande@ics.upm.es 
Abstract 
The use of web-based tests delivered through learning management systems has grown at university 
level in the last years. One of their key advantages is the possibility of creating tests with some degree 
of randomness that are automatically assessed in real time. Although the access to the learning 
management system resources is controlled for each student by means of personal username and 
password, the cheating among students when doing the tests cannot be avoided. However, if the 
students finally learn, in spite of cheating, the process could still be considered to be successful. In 
this work, the date, the required time to solve the test and the grades of quizzes undertaken by 
students through a web based learning management system are analyzed and they are compared to 
the grades obtained by the same students in a written test solved in an examination classroom under 
the supervision of the teacher. The course in which this study has been developed (Signals and 
Systems for Electrical and Electronics Engineering undergraduate students) is organized in 5 subjects 
and the students make a quiz on the web for each subject. At the end of the course the students make 
a final written exam that includes a true/false test. Around 50 questions for each subject of the course 
have been created. The questions are organized in 5 to 8 categories for each subject. The learning 
management system generates quizzes by arbitrarily selecting 1 or 2 items from the 5 to 8 categories 
in a given subject to complete a 10-item quiz. Due to the reduced number of items for each category 
and the large number of students that attend the course, several questions are repeated in quizzes 
generated for different students. The authors have noticed that some students work in groups to solve 
the quizzes. Some of them answer all the questions in a quiz in few minutes (less than 20 % of the 
time used by the most of their mates) and obtain high scores. When the scores of the same students 
in the final exam are analyzed, it is found that they also obtain good results. Then, it could be 
concluded that although they have found a way of cheating to solve the web quizzes, this is still 
pedagogically valid because they have learnt about the subject (they also obtain good results in the 
written test). 
Keywords: Learning management systems, quizzes, cheating. 
1 INTRODUCTION  
Learning management systems (LMS) have become a common and very useful tool for instructors at 
any educational level [1-3]. They usually offer a series of resources as creating links to websites or 
files, creating many kinds of activities for students as chats, fora, wikis, questionnaires, surveys etc. In 
the currently ongoing process of creating the European Higher Education Area, the educational 
outcomes of university courses are being defined in terms of competences that are to be acquired by 
the students in order to get their degrees [4, 5]. Accordingly the assessment of the students must be 
based on competence acquirements and on the student’s workload so a continuous supervision of 
students work is often convenient. Continuous evaluation methods increase the instructors’ workload 
when they deal with numerous groups of students as usually is the case for basic courses of many 
engineering studies [6]. In this scenario, the LMS could be a very helpful tool for instructors. 
From the point of view of the instructor the online quizzes can help to follow the progress of his/her 
students and to automatically provide feedback to the students about this progress. Nowadays the 
LMS system offers several ways to create tests with some degree of randomness. The variations of 
the quizzes are usually obtained by picking questions from a large item bank, by changing the order in 
which the questions are presented, by changing the order in which the possible answers are 
presented (at least for multiple choice questions) and by changing numerical data when the solution is 
a single relation of these numerical data. Although the access to the LMS resources is controlled for 
each student by means of personal username and password, there is not an effective way to be sure 
of who is answering an online questionnaire. Additionally, when a large number of students attend a 
course, the probability that the same question is asked to several students is high. 
Proceedings of ICERI2010 Conference. 
15th-17th November 2010, Madrid, Spain.
ISBN: 978-84-614-2439-9
005004
The authors have used a LMS for assessing the progress of the students in a basic matter of Electrical 
and Electronics Engineering in the Escuela Universitaria de Ingeniería Técnica de Telecomunicaión 
(EUITT) at the Universidad Politécnica de Madrid (UPM). They have notice that some students 
answering one or several online tests in a very short time obtain surprisingly high scores. The purpose 
of this work is to analyze the date, required time to solve the tests and grades of quizzes undertaken 
by students through a web based LMS and to compare them to the grades obtained by the same 
students in a written test solved in an examination classroom under the supervision of the teacher 
2 METHOD 
Signals and Systems is a main topic for Electrical and Electronics Engineering undergraduate 
students. The course usually last 15 weeks and the students attend 4 h of lessons per week. At the 
EUITT this course is organized in 5 subjects:  
T.1. Introduction to signals and systems 
T.2. Time domain analysis of linear and time invariant systems 
T.3. Fourier analysis of continuous time signal and systems 
T.4. Laplace analysis of continuous time signal and systems 
T.5. Fourier and Z-transform analysis of discrete time signal and systems 
The assessment methods used in this Signal and Systems course are as follows: 
a) One online test after each subject (20% of the final mark).  
b) Exercises every week that are marked in the classroom (30% of the final mark). 
c) Final exam at the end of the semester with two parts: a test part (20%) and an open answer 
exercises part (30%).  
The final exam is mandatory to take into account the marks obtained through the weekly work of 
students. This exam is the same for all students enrolled in the course. The final exam is made under 
the supervision of all the teachers involved in Signals and Systems it is assumed that the students 
cannot cheat at this stage (at least is quite difficult in comparison with situations of online test and 
classroom exercises). For students that did not follow the continuous assessment method, the final 
exam counts 100% of their mark. 
This assessment method was adopted because the success rate during the last decade (when 
students were assessed only by means of a final exam) was decreasing and the drop out rate was 
increasing [7]. The authors have checked that the continuous assessment method yields to similar 
pass/fail rates than the final exam, but the drop out rate is lower [8]. 
At the EUITT there are around 450 students that attend this course every academic year. They are 
divided in 8 groups each of which is attended by an lecturer. Although the work of students is 
organized and supervised by their teacher, there are some common tasks for all students, for example 
the online tests and final exam. 
During the academic year 2009/10, the five online tests were solved through Moodle on the following 
dates dates: (i) October 15th to 19th; (ii) November 5th to 9th; (iii) November 19th to 25th; (iv) December 
11th to 16th and (v) January 14th to 20th. The date of the final exam was January 26th. The students had 
a limited time to solve each test once they had opened it (from 20 to 50 min depending on the 
subject). They could solve the test in any place where they had a personal computer with internet 
connection and at any time during the indicated periods for each test.  
The online tests were prepared in Moodle [1]. An item bank with around 50 questions for each subject 
of the course had been created. The questions were organized in 5 to 8 categories for each subject. 
The LMS generated quizzes by arbitrarily selecting 1 or 2 items from the 5 to 8 categories in a given 
subject to complete a 10-item quiz. When a student finished an online test he/she could see his/her 
mark but could not see the right solution till the test was closed for all students. 
All students that made the final exam were included in this study (244). This analysis involves the 
marks obtained by these students in the online tests and in both parts of the final exam. Correlations 
and analysis of variance were used to infer if several groups of marks were correlated or not and if the 
mean value of marks could be considered statistically different. 
005005
3 RESULTS 
The motivation of this work is to analyze how the marks of a single online test are distributed 
depending on the time taken for the student to solve it and the moment chosen by him/her to begin the 
test (at the beginning of the period or during the last days they could solve the test). Fig. 1 shows the 
marks obtained by the students that solved one of the online tests (the 3rd one for this case). The 
mean time required to solve this test was 30 min (with a standard deviation of 12 min). It can be 
observed that some students solved the test in a few minutes (less than 15 min). However, only 2 
students that solved the test in less than 15 min during the first days (Fig.1a) obtained a mark over 5. 
In contrast, a greater portion of students that solved the test in less than 10 min obtained a high score 
during the last days (Fig. 1b). This fact made the authors be suspicious about the way these students 
solved the test. The students that made the test during the last days could have some information 
about the questions that could appear form their mates that solved the test during the first days, so 
some of them could take advantage of this. 
  
Figure 1: Marks obtained in third online test by students versus required time to do it. Part a) shows 
the results of students that made the test during the first 4 days since the test was opened, and part b) 
the results of students that made the test during the last 3 days in which the test was open 
Fig. 2 shows the data corresponding to fourth online test. Similar observations to those of Fig. 1 can 
be made: part a) shows that students that made the test during the first days in which the test was 
opened required more than ten min to solve it and when they solved in less than 20 min they obtained 
a mark lower than 6. Part b) shows that some students finished the test in a surprisingly short time and 
obtained high scores (two of them solved it in 1 min and obtain near 5 or higher marks) during the last 
days. 
Fig. 3 shows a comparison of the marks obtained by the students in the final exam test versus the 
mean marks obtained in the online test. Only the results of students that made at least 1 online test 
and the final exam are shown. There were around 50 students that did not follow the continuous 
assessment and only made the final exam. It can be seen that there is a low correlation between final 
exam test marks and mean marks in online tests (correlation coefficient equal to 0.28). One reason 
could be that there was a group of students that only made the first and/or the second online test and 
they gave up and made the final exam. Some of these students obtained high marks (points in the left 
and upper part of Fig. 2). There is a second group of students that solved reasonably well several 
online test and obtained high mean mark in the online tests, but they did not learn from this practice 
and obtained a bad result in the final exam test (right and lower part of the graph). The analysis of 
variance of these two variables shows that the mean value of the final exam mark is 3.2 and this is 
statistically lower than the mean mark of the tests averaged for all students (4.1) with a p value lower 
than 10−5. The variances of both groups of marks are practically the same, 2.1, i.e. quite large, and this 
explains why the points cover a wide range of values in both horizontal and vertical directions. 
 
005006
  
Figure 2: Marks obtained in fourth online test by students versus required time to do it. Part a) shows 
the results of students that made the test during the first 4 days in which the test was open, and part b) 
the results of students that made the test during the last 2 days in which the test was open 
 
Figure 3: Marks obtained by students in the final exam test versus the mean mark obtained in the 
online tests 
The second step of this study is to track all students that solved at least 1 online test in a time that was 
less than half the mean time required by all students to solve that test and obtain a high mark. Fig. 4 
shows the results obtained by this group of students. It can be observed that the correlation between 
vertical and horizontal variables is quite higher than that of Fig. 3 (the correlation coefficient is 0.50 for 
the data of Fig. 4). This means that this group of students, those that obtained a high mean mark in 
the online tests (whatever the way they obtained it), also obtained high mark in the final exam test, so 
they learnt to solve this kind of tests.  
005007
 
Figure 4: Marks obtained by students in the final exam test versus required the mean mark obtained in 
the online tests. Only the results for students that finished at least one online test in a few minutes and 
obtained a high mark in that test are showed 
4 DISCUSION AND CONCLUSIONS 
After analyzing the results shown in the previous section we can see that there is a small group of 
students (those included in Fig.4) that solved at least one online test in a surprisingly short time (at 
most half the mean time required by their mates to solve the same test) and obtained a high score in 
that test.  
Figs. 1 and 2 show that 20-25 students finished the online test in a short time. It could be thought that 
they open the test just to see the kind of questions for each subject and answer them randomly. The 
probability of obtaining a mark over 5 in a ten single choice among four options questions quiz when it 
is solved in a random way is 2%. Then, it is hard to believe that around 10 of these students could 
obtain high marks. 
A reason to explain that observation could be that these students knew many questions that could 
appear in that test (and probably knew the right answer to most of them); it must be noticed that a 
large amount of students were enrolled in Signals and Systems course for a second or third time 
because they had failed in previous attempts. These students could know some of the questions 
because they were solving the online tests for a second or third time.  
A second reason might be that they solves the online tests in groups and had learned from classmates 
that had solved the test before; in this case, the last students that solved the test had some 
advantage. Some of the students represented in Fig. 4 could correspond to this profile. This practice 
could be censurable if many of these students finally obtained low marks (or lower marks than those of 
their mates that did not behave in the same way). 
Fig. 4 shows that students that, for any reason, had some knowledge about the questions for each 
subject and took advantage of this obtaining a high mark in online test also obtained higher scores in 
the final exam test (comparing Fig. 4 to Fig. 3). So they had learned to solve the test, somehow.  
Then it could be concluded that although they found a way of cheating to solve the web quizzes, this is 
still pedagogically valid because they also learnt about the subject (they also obtain good results in the 
written test). 
 
REFERENCES  
[1] Moodle.org: open-source community-based tools for learning, http://www.moodle.org, 
[Accessed 20 July 2010]. 
[2] Questionmark, http://www.questionmark.com/uk/index.aspx, [Accessed 20 July 2010]. 
[3] Openmark, https://openmark.dev.java.net/, [Accessed 20 July 2010]. 
005008
[4] European Ministers of Educ., “The European higher education area,” European Union, Bologna 
(Italy), Joint Declaration, 1999. [Online]. Available: 
http://www.ond.vlaanderen.be/hogeronderwijs/ bologna/documents/MDC/BOLOGNA 
DECLARATION1.pdf 
[5] R. Fraile, I. Argüelles, J. C. González, J. M. Gutiérrez-Arriola, J. I. Godino-Llorente, C. 
Benavente, L. Arriero, D. Osés. A Systematic Approach to the Pedagogic Design of Final Year 
Projects: Learning Outcomes, Supervision and Assessment. Int. J. Engng Ed. Vol. 26, No. 4, 
pp. 997–1007, 2010.  
[6] C. R. Smaill. The Implementation and Evaluation of OASIS: A Web-Based Learning and 
Assessment Tool for Large Classes. IEEE Transactions on Education, V. 48 (4), 658-663, 2005.  
[7] J. C. G. de Sande, L. Arriero, C. Benavente R. Fraile, J. I. Godino-Llorente, J. M. Gutiérrez-
Arriola, D. Osés. V. Osma. Evolution of efficiency and success rate for Electrical and Electronic 
Engineering students at EUITT from Universidad Politécnica de Madrid. Proceedings of 
INTED2009 Conference. 9-11 March 2009, Valencia, Spain. [8] J. C. G. de Sande, L. Arriero, C. Benavente R. Fraile, J. I. Godino-Llorente, J. M. Gutiérrez-
Arriola, D. Osés. V. Osma. A case study: final exam versus continuous assessment marks for 
Electrical and Electronic Engineering students. Proceedings of ICERI2008 conference. 
November 2008 Madrid, Spain. 
005009


Cheating and learning through web based tests

Archivo Digital UPM

http://oa.upm.es/9580/1/INVE_MEM_2010_88686.pdf

Cheating and learning through web based tests

Abstract

Similar works

Full text

Available Versions

Servicio de Coordinación de Bibliotecas de la Universidad Politécnica de Madrid

Archivo Digital UPM