Borysova, N. V.

Melnyk, K. V.

Melnyk, V. I.

English

Electronic National Technical University &quot;Kharkiv Polytechnic Institute&quot; Institutional Repository (eNTUKhPIIR)

Секція I. ІНФОРМАЦІЙНІ СИСТЕМИ І ТЕХНОЛОГІЇ  IV Всеукраїнська науково-практична конференція «ПЕРСПЕКТИВНІ НАПРЯМКИ СУЧАСНОЇ ЕЛЕКТРОНІКИ, ІНФОРМАЦІЙНИХ І КОМП'ЮТЕРНИХ СИСТЕМ» MEICS-2019 27-29 листопада 2019 р., Дніпро, Україна  27DEVELOPMENT OF COMPUTER SYSTEM FOR ADAPTIVE KNOWLEDGE TESTING IN HUMANITARIAN DISCIPLINES  N. Borysova1, K. Melnyk1, V. Melnyk2 1National Technical University "Kharkiv Polytechnic Institute" borysova.n.v@gmail.com 2Kharkiv general education school № 145 v13121423@gmail.com  Considering the opportunities offered to the education system by the current level of development of information technology and computer techniques there is no doubt the need for implementation them as much as possible, particularly in the form of computer systems for students’ and schoolchildren’s knowledge testing. Such systems are already widely used for knowledge testing in the natural and technical disciplines, which cannot be said for knowledge testing in the humanitarian disciplines. This is due to the specificity of the material and the teaching of the humanitarian disciplines, and the fact that specific forms of knowledge control are used in these disciplines such as essay, narration, etc., that are not typical for other disciplines and require free thoughts expression. It cannot be said that computer testing of knowledge in the humanitarian disciplines is not carried out at all. The analysis of publications and researches showed that either ready-made test programs or researchers’ own projects were used for this purpose. But in both cases, a limited number of questions types are used, mostly closed ones such as to choose the right answer from the several suggested ones, to agree or not with the statement, to find the matches, etc. If, for example, such questions types are used where the missing words are to be inserted as an answer, then the correctness of the answer is checked by comparing it to the exact match of the teacher’s answer without taking into account possible synonyms, paraphrases, etc.Thus, the identified disadvantages of the existing approaches to computer testing of knowledge in the humanitarian disciplines have determined the direction and purpose of our research, which resulted in a computer information system for the adaptive testing of 7-11 grades schoolchildren’ knowledge in the subjects "Foreign Literature" and "Arts". Let’s consider developed information system’s functionality in more detail way. The information system can be operated by an administrator or a user. The administrator is a teacher who can add, correct and delete questions, set points for the right answers, manage users’ profile, view the results of all users. The users are schoolchildren who study the disciplines and will be tested. For the user, the information system can operate in two modes: traditional control mode and self-control mode. In traditional control mode, a user’s knowledge in a discipline is tested using an adaptive test model, in self-control mode the user can also test their knowledge, but this mode is designed to train the system usage. The questions for the tests are separated into four difficulty levels according to their type: 1 – easy (choose the right answer(s); agree or refute; find the matches); 2 – medium (insert missing words; formulate definitions; make a list; give an example); 3 – difficult (classify; characterize; compare; justify; conclude; determine the main opinion; describe); 4 – very difficult (create; optimize information in your own interpretation). The number of points will be given for the correct answer depends on the difficulty level. All questions are stored in the database. The user must authorize to get started with the system. This procedure, first, eliminates the possibility of passing the test by another person, and secondly, gives the user access to only those tests that he can pass according to his profile. User information is stored in the database. In traditional control mode, the user gets the first question of the first difficulty level. If Секція I. ІНФОРМАЦІЙНІ СИСТЕМИ І ТЕХНОЛОГІЇ  IV Всеукраїнська науково-практична конференція «ПЕРСПЕКТИВНІ НАПРЯМКИ СУЧАСНОЇ ЕЛЕКТРОНІКИ, ІНФОРМАЦІЙНИХ І КОМП'ЮТЕРНИХ СИСТЕМ» MEICS-2019 27-29 листопада 2019 р., Дніпро, Україна  28 the user answers the previous question correctly, the next question will be more difficult, or will be of the same difficulty, if he answers incorrectly. During passing the test, the user can return to the previous question and change their answer. Testing continues either until the test is complete, or the user can interrupt the test by clicking the appropriate button and confirming its validity. The validity confirming is using to prevent accidental clicking the stop testing button. However, in this case, the answers to the remained unanswered questions will be counted like incorrect. After the test is complete, the user can view: the test results as a list of questions with his or her chosen answers; with marks "correctly", "incorrectly", "points for the answer were unaccrued – contact the teacher"; with the correct answers and with the number of points received for answering each question. He can also see the received points; his assessment on the traditional twelve-point scale and recommendations for further training. The total number of points is calculated using the Rasch model [1]. The user has the opportunity to take this test two more times. In this case, he will get other questions from the database and the previous test results will be canceled. Test results are stored in the database. In assessing the correctness of the answers to the first difficulty level questions, a comparison is made to the exact match of the answer given by the user with the correct answer stored in the database, due to the answers to these questions are unambiguous. In assessing the correctness of the answers to the second difficulty level questions, the type of question is taken into account: for the questions "make a list" – a comparison is made to the exact match of answer given by user with the correct answer stored in the database; for "insert missing words", "formulate definitions", "give an example" – a checking for the presence of keywords determined by the teacher when formulating the question answer, if keywords are absent, it is made a checking for their synonyms presence. Keywords and synonyms are stored in the database. In assessing the correctness of the answers to the third difficulty level questions, the neural network was used. It has been trained in the test collection of possible correct answers to the questions. Since the set of possible correct answers to a question can not be large at the beginning of use, which affects the performance of the network, it provides suggestions on the number of points for the answer to the question, but the final decision is made by the teacher. The test collection for the neural network training may be supplemented by the new answers given during the testing. The assessment of the correctness of the answers to the fourth difficulty level questions is carried out by the teacher, because the questions of this type belong to the creative level of cognition and automated assessment of the answers correctness is possible only if the intelligent methods of textual information processing are used. In self-control mode, the user’s knowledge of the discipline is also tested, but the adaptive model is not used. In this mode only questions of the first and second difficulty levels are used, questions are given in a random order, it is possible to use a hint (but points in this case will not be given); to check the correctness of the chosen answer immediately; to interrupt testing. At the end of the test, user can get the same results report as he would in traditional control mode. Thus, the use of computer-based testing for knowledge assessment allows teachers to systematize and simplify the process of knowledge assessment, while increasing schoolchildren’s motivation for learning and reducing their stress in the control process.  [1] Borysova N.V., Melnyk K.V. Rasch model usage for testing results assessment. XXVII Міжнародна науково-практична конференція “Інформаційні технології: наука, техніка, технологія, освіта, здоров’я” (MicroCAD-2019). Тези доповідей. Харків, 15-17травня 2019). Ч. 1. За ред. проф. Сокола Є.І. Харків: НТУ "ХПІ". (2019). – С. 14.  

Development of computer system for adaptive knowledge testing in humanitarian disciplines

Library of National Technical University "Kharkiv Polytechnic Institute"

Секція I. ІНФОРМАЦІЙНІ СИСТЕМИ І ТЕХНОЛОГІЇ 
 
IV Всеукраїнська науково-практична конференція 
«ПЕРСПЕКТИВНІ НАПРЯМКИ СУЧАСНОЇ ЕЛЕКТРОНІКИ, ІНФОРМАЦІЙНИХ І 
КОМП'ЮТЕРНИХ СИСТЕМ» MEICS-2019 
27-29 листопада 2019 р., Дніпро, Україна 
 
27
DEVELOPMENT OF COMPUTER SYSTEM FOR ADAPTIVE 
KNOWLEDGE TESTING IN HUMANITARIAN DISCIPLINES 
 
N. Borysova1, K. Melnyk1, V. Melnyk2 
1National Technical University "Kharkiv Polytechnic Institute" 
borysova.n.v@gmail.com 
2Kharkiv general education school № 145 
v13121423@gmail.com 
 
Considering the opportunities offered to the education system by the current level of 
development of information technology and computer techniques there is no doubt the need for 
implementation them as much as possible, particularly in the form of computer systems for 
students’ and schoolchildren’s knowledge testing. Such systems are already widely used for 
knowledge testing in the natural and technical disciplines, which cannot be said for knowledge 
testing in the humanitarian disciplines. This is due to the specificity of the material and the 
teaching of the humanitarian disciplines, and the fact that specific forms of knowledge control 
are used in these disciplines such as essay, narration, etc., that are not typical for other 
disciplines and require free thoughts expression. It cannot be said that computer testing of 
knowledge in the humanitarian disciplines is not carried out at all. The analysis of publications 
and researches showed that either ready-made test programs or researchers’ own projects were 
used for this purpose. But in both cases, a limited number of questions types are used, mostly 
closed ones such as to choose the right answer from the several suggested ones, to agree or not 
with the statement, to find the matches, etc. If, for example, such questions types are used where 
the missing words are to be inserted as an answer, then the correctness of the answer is checked 
by comparing it to the exact match of the teacher’s answer without taking into account possible 
synonyms, paraphrases, etc.Thus, the identified disadvantages of the existing approaches to 
computer testing of knowledge in the humanitarian disciplines have determined the direction 
and purpose of our research, which resulted in a computer information system for the adaptive 
testing of 7-11 grades schoolchildren’ knowledge in the subjects "Foreign Literature" and 
"Arts". Let’s consider developed information system’s functionality in more detail way. 
The information system can be operated by an administrator or a user. The administrator 
is a teacher who can add, correct and delete questions, set points for the right answers, manage 
users’ profile, view the results of all users. The users are schoolchildren who study the 
disciplines and will be tested. 
For the user, the information system can operate in two modes: traditional control mode 
and self-control mode. In traditional control mode, a user’s knowledge in a discipline is tested 
using an adaptive test model, in self-control mode the user can also test their knowledge, but 
this mode is designed to train the system usage. 
The questions for the tests are separated into four difficulty levels according to their 
type: 1 – easy (choose the right answer(s); agree or refute; find the matches); 2 – medium (insert 
missing words; formulate definitions; make a list; give an example); 3 – difficult (classify; 
characterize; compare; justify; conclude; determine the main opinion; describe); 4 – very 
difficult (create; optimize information in your own interpretation). The number of points will 
be given for the correct answer depends on the difficulty level. All questions are stored in the 
database. 
The user must authorize to get started with the system. This procedure, first, eliminates 
the possibility of passing the test by another person, and secondly, gives the user access to only 
those tests that he can pass according to his profile. User information is stored in the database. 
In traditional control mode, the user gets the first question of the first difficulty level. If 
Секція I. ІНФОРМАЦІЙНІ СИСТЕМИ І ТЕХНОЛОГІЇ 
 
IV Всеукраїнська науково-практична конференція 
«ПЕРСПЕКТИВНІ НАПРЯМКИ СУЧАСНОЇ ЕЛЕКТРОНІКИ, ІНФОРМАЦІЙНИХ І 
КОМП'ЮТЕРНИХ СИСТЕМ» MEICS-2019 
27-29 листопада 2019 р., Дніпро, Україна 
 
28 
the user answers the previous question correctly, the next question will be more difficult, or 
will be of the same difficulty, if he answers incorrectly. 
During passing the test, the user can return to the previous question and change their 
answer. Testing continues either until the test is complete, or the user can interrupt the test by 
clicking the appropriate button and confirming its validity. The validity confirming is using to 
prevent accidental clicking the stop testing button. However, in this case, the answers to the 
remained unanswered questions will be counted like incorrect. After the test is complete, the 
user can view: the test results as a list of questions with his or her chosen answers; with marks 
"correctly", "incorrectly", "points for the answer were unaccrued – contact the teacher"; with 
the correct answers and with the number of points received for answering each question. He 
can also see the received points; his assessment on the traditional twelve-point scale and 
recommendations for further training. The total number of points is calculated using the Rasch 
model [1]. The user has the opportunity to take this test two more times. In this case, he will 
get other questions from the database and the previous test results will be canceled. Test results 
are stored in the database. 
In assessing the correctness of the answers to the first difficulty level questions, a 
comparison is made to the exact match of the answer given by the user with the correct answer 
stored in the database, due to the answers to these questions are unambiguous. In assessing the 
correctness of the answers to the second difficulty level questions, the type of question is taken 
into account: for the questions "make a list" – a comparison is made to the exact match of answer 
given by user with the correct answer stored in the database; for "insert missing words", 
"formulate definitions", "give an example" – a checking for the presence of keywords determined 
by the teacher when formulating the question answer, if keywords are absent, it is made a 
checking for their synonyms presence. Keywords and synonyms are stored in the database. In 
assessing the correctness of the answers to the third difficulty level questions, the neural network 
was used. It has been trained in the test collection of possible correct answers to the questions. 
Since the set of possible correct answers to a question can not be large at the beginning of use, 
which affects the performance of the network, it provides suggestions on the number of points 
for the answer to the question, but the final decision is made by the teacher. The test collection 
for the neural network training may be supplemented by the new answers given during the testing. 
The assessment of the correctness of the answers to the fourth difficulty level questions is carried 
out by the teacher, because the questions of this type belong to the creative level of cognition and 
automated assessment of the answers correctness is possible only if the intelligent methods of 
textual information processing are used. 
In self-control mode, the user’s knowledge of the discipline is also tested, but the 
adaptive model is not used. In this mode only questions of the first and second difficulty levels 
are used, questions are given in a random order, it is possible to use a hint (but points in this 
case will not be given); to check the correctness of the chosen answer immediately; to interrupt 
testing. At the end of the test, user can get the same results report as he would in traditional 
control mode. 
Thus, the use of computer-based testing for knowledge assessment allows teachers to 
systematize and simplify the process of knowledge assessment, while increasing 
schoolchildren’s motivation for learning and reducing their stress in the control process. 
 
[1] Borysova N.V., Melnyk K.V. Rasch model usage for testing results assessment. 
XXVII Міжнародна науково-практична конференція “Інформаційні технології: 
наука, техніка, технологія, освіта, здоров’я” (MicroCAD-2019). Тези доповідей. 
Харків, 15-17травня 2019). Ч. 1. За ред. проф. Сокола Є.І. Харків: НТУ "ХПІ". (2019). 
– С. 14.  


http://repository.kpi.kharkov.ua/bitstream/KhPI-Press/44722/1/Borysova_Computer_system_2019.pdf

Development of computer system for adaptive knowledge testing in humanitarian disciplines

Abstract

Similar works

Full text

Available Versions

Electronic National Technical University "Kharkiv Polytechnic Institute" Institutional Repository (eNTUKhPIIR)

Library of National Technical University "Kharkiv Polytechnic Institute"

Development of computer system for adaptive knowledge testing in humanitarian disciplines

Abstract

Similar works

Full text

Available Versions

Electronic National Technical University &quot;Kharkiv Polytechnic Institute&quot; Institutional Repository (eNTUKhPIIR)

Library of National Technical University "Kharkiv Polytechnic Institute"

Electronic National Technical University "Kharkiv Polytechnic Institute" Institutional Repository (eNTUKhPIIR)