Application Of IRT And MIRT Models To Analysis Of Analytical Geometry Tests

The article conducts research on the construction of analytical methods of computer tests in higher mathematics. These computer tests contain test items of various types, including problems with "embedded answers" which include several related subitems. Also, on the basis this method quality analysis of the control work was conducted. The technic is based on the methods of Classical Test Theory (CTT) and Item Response Theory (IRT), which have proven their effectiveness in statistical analysis of tests. The focus of the work is on the use of Multidimensional Item Response Theory (MIRT) models, which allows to analyze simultaneously the whole vector of students' competencies and to scrutinize them more carefully. The study also uses one-dimensional IRT models, the results of which are compared with the use of MIRT. Among the one-dimensional models, the well-known Muraki and Birnbaum models were selected, and among the multidimensional models, the choices were the two-dimensional 2-PL and GPCM. The multidimensional models involved in the study are compensatory. The application of non-compensatory models was not considered. The comparison of compliance with the data of different models was carried out on the basis of special information criteria. One-dimensional models were discovered to be somewhat more effective. The R programming environment was chosen as the main toolkit, which provides a powerful set of software for the test statistical analysis. The mirt package is selected as the main software package. Tests results from a modular test in analytical geometry have been chosen as data for research. The test was written by 105 students from IT stream, specialty 121, at FIOT NTUU "Igor Sikorsky Kiev Polytechnic Institute". The test was placed on the MOODLE platform and was conducted remotely. The analysis of test results on the basis of selected models demonstrated the consistency between the results of both one-dimensional and multidimensional models. However, multidimensional models also allow to expound the analysis of various additional competences, in this case - knowledge of vector algebra and knowledge of lines, planes, and surfaces in space. The analysis showed that the test was, in general, composed correctly; also the analysis allowed to systematize the items by complexity; and for questions such as "embedded answers " to detail the complexity of the subitems. Evaluating the overall results of the application of one-dimensional and multidimensional models of IRT, it should be noted their effectiveness in the analysis of both tests in higher mathematics and in the control of knowledge in other disciplines

Simulation Of Distribution Of The Maximum Of The Chentsov Field On ‘Steps’

The paper extends the results obtained by Paranjape, Park, Klesov and Kruglova. A method of finding the distribution of the maximum of a Chentsov field restriction on a polyline is suggested. To do this, it was proposed to simulate a random process that corresponds to the restriction. The suggested algorithm has high computational efficiency which is crucial due to a large number of points of division of the process trajectory. The selection of the distributions of extreme functionals from the field was performed by means of the statistical programming language R. The Weibull distribution was chosen as the most suitable distribution, which was confirmed by the Kolmogorov criterion, as well as by Q-Q and P-P diagrams

Analysis of Conceptual Categorical Apparatus of Research of Higher Mathematics Tests Quality

In the article is presented the conceptual categorical apparatus, used at the study of the quality of tests in higher mathematics. The main notions, necessary for constructing the method of test quality assessment, are selected and defined, namely: Test it is a system of tasks of specific form, certain content for objective assessment of the level of students' readiness with the preliminary set methodology of results analysis;Test quality it is a test characteristic that indicates the correspondence to requirements, presented to test characteristics in whole and to separate test tasks (the mean parameters of test quality is validity, effectiveness of test tasks – complication and differentiation ability);Assessment of test quality it is a procedure of the determination of a correspondence degree of characteristics of separate test tasks and test in whole to quality criteria and the formation of a conclusion about test quality;Computer oriented assessment of test quality it is an assessment of test quality using ICT;Methodology of computer oriented assessment of test quality it is a theoretically grounded and logically regulated totality of methods of test quality assessment using ICT.The introduced notions distinctly outline research directions; determine assessment methods and ways of quality improvement of tests in higher mathematics that, in its turn, favors the improvement of students' knowledge contro

ОЦІНЮВАННЯ ЗМІСТОВОЇ ВАЛІДНОСТІ ТЕСТІВ З ВИЩОЇ МАТЕМАТИКИ З ВИКОРИСТАННЯМ GOOGLE DOCS ДОДАТКУ

У статті досліджено питання аналізу змістової валідності комп’ютерних тестів з вищої математики. Розглянуто технологію оцінки якості змісту тесту з використанням Google Docs додатку, яка передбачає організацію і проведення експертизи із встановлення відповідності між тестовими завданнями і змістовою галуззю тесту і приймання узагальнюючих висновків розробником. Експертиза включає три напрями роботи експертів: оцінювання завдань тесту, оцінювання тесту у цілому та оформлення узагальнюючих висновків і рекомендацій. Відзначено основні особливості та переваги використання Google Docs додатку як інструменту проведення експертизи змісту тесту

IMITATIONAL MODELING AND ANALYSIS OF MATRIXES CONTAINING PRIMARY GRADING OBTAINED IN EDUCATIONAL TESTING BY THE MEANS OF LANGUAGE R

The article researches methods of imitational modeling of matrixes containing primary grading obtained in educational testing by the means of statistical programming language R. Unique algorithms and functions were developed to allow generating of matrix of the primary grades according to corresponding test of the defined structure. The importance of this approach is defined by several reasons, specifically the needs to: create reference samples; analyze primary grades by means of CTT (Clasic Test Theory) and IRT; predict basic statistical test characteristics; clarify parameters for the calibrated tasks; model independent parameters for the test takers; increase and development educator’s competency. It should be noted that input parameters could be generated or set up manually. Comparable analysis was conducted for created functions against already existing function packages, such as eRm, ltm, mcIRT, as well as statistical analysis of the generated matrixes. This analysis took into consideration the following procedures: verification of hypothesis about compatibility between generated matrixes and set parameters for the testing tasks per criteria; verification of hypothesis about equivalence of average grades in entire matrix according Student criteriа; verification of hypothesis about equality for vectors of correct answers relative frequencies by columns according to Hotteling criteriа; comparison of theoretical characteristic curves with empirical probabilities; comparison of set parameters for task complexity and parameters graded by generated matrixes with consideration of errors in grading. An experimental system of imitational modeling and analysis for testing results was created. Such system combines contemporary methods of IRT and methods of Classical Testing Theory (CTT). It allows generating matrixes of primary testing grades and performing test results analysis; permits computation of basic statistical characteristics of the test, estimation of the latent parameters, construction of characteristic curves and informational functions. The system graphic shell was generated with the help of the package Shiny. The system utilizes modeling and analysis for testing results according to basic IRT models: Rasch, Birnbaum, Suh-Bolt, Rasch-Masters. A performance verification for algorithms and functions implemented into the system has been done by utilizing several noted statistical methods and procedures; and correct execution of these algorithms has been confirmed

DEVELOPMENT MODEL OF THE TEACHERS’ COMPETENCY CONCERNING THE QUALITY ASSESSMENT OF TESTS IN HIGHER MATHEMATICS

The paper studies a task of developing the teachers’ competency concerning the quality of tests in higher mathematics. This issue becomes especially critical in the context of finding ways to improve knowledge control and the introduction of computer testing to determine students' learning curve in higher mathematics. The concept of the teachers’ competency concerning the quality of tests in higher mathematics is substantiated. It is defined as the teacher’s ability to calculate the characteristics of the test, test items with the use of ICT and on the basis of these characteristics to evaluate the quality of individual items and objectively draw conclusions about the quality of the test in general, its improvement and the expediency of using in the learning process for knowledge control. The test analysis consists in analysis of distribution of the results’ sample, estimation of the test reliability, estimation of the test validity and estimation of the test's effectiveness. The analysis of test items consists in analysis of the correlation matrix of items, estimation of the items reliability and validity, and estimation of test efficiency; analysis of the ensemble of characteristic curves of items, the analysis of polytomic items, the analysis of the distractors of the multiple choice items. The development model of this competency is proposed. It consists of five components: motivational (includes goals and objectives of the educational process), content (defines the content of educational activities by the levels of teachers’ competency in evaluating the quality of tests in higher mathematics), technological (defines concretely the organizational forms and methods of competency development), diagnostic and resultative (determines the competency levels of teachers concerning the quality of tests in higher mathematics, examination and effectiveness analysis of this process). Prospects of research have been determined. They consist in both determining the criteria of grading the level of teachers’ competency against the quality of tests in higher mathematics as well as definition of competency levels for more precise characterization of the model content component. Development of the teachers’ competency as related to the quality of tests in higher mathematics will help improving controls of students' knowledge in higher mathematics and increase the professional aptitude of educators

Використання математичних моделей для аналізу результатів психологічного тесту Гілфорда

Results of the survey, conducted using Joy Guilford methodology, of 800 servicemen regarding their professional competence was analyzed with IRT models. In particular, Rasch model and 1-PL, 2-PL and 3-PL models were used. All computations were performed with R statistical software along with ltm package. Latent parameters for both respondents and corresponding indicators (survey questions) were obtained. It was found that there exists signifi cant discrepancy between the models and results of the survey. Analysis of this discrepancy has shown that the major cause of their existence are inadequate conditions under which the survey was conducted.У роботi дослiджено результати тестування за методикою Гiлфорда на професiйну придатнiсть 800 вiйськовослужбовцiв за допомогою вiдомих моделей IRT. Застосовано модель Раша, а також 1-PL, 2-PL та 3-PL моделi. Для комп’ютерного оброблення застосовано мову R, а саме, пакет ltm. Розраховано латентнi параметри як респондентiв, так i вiдповiдних iндикаторiв (запитань тесту). Виявлено суттєвi порушення адекватностi побудованих моделей результатам тестування. Проаналiзовано можливi причини таких невiдповiдностей, головною з яких є порушення об’єктивностi умов проведення тестування

Методи знаходження законів розподілів випадкових величин за даними статистичних вибірок засобами мови R

The following article discusses various methods for probability distribution fitting to simulated data by means of R statistical computing language. In particular, some graphical methods like plotting of histograms, empirical and theoretical density functions, P-P and Q-Q plots, were considered. Estimation functions for probability distribution parameters were investigated by applying method of moments, method of quantiles, method of maximum likelihood, and shortest distance method. Hypothesis about probability distribution were verifi ed with Kolmogorov–Smirnov, AIC, and BIC tests. The corresponding data set used to illustrate the above methods was taken from probability distribution of the maximum of Chenstov field restriction to a particular curve. The distribution was simulated with the special original algorithm in R statistical software.У статтi дослiджено методи пiдбору теоретичного ймовiрнiсного розподiлу для змодельованих статистичних даних засобами мови статистичного програмування R. Розглянуто графiчнi засоби пiдбору закону розподiлу: побудова гiстограм, емпiричних i теоретичних щiльностей i функцiй розподiлу, P-P i Q-Q дiаграм. Дослiджено функцiї оцiнювання параметрiв законiв розподiлу методами: моментiв, квантiлiв, найбiльшої вiрогiдностi та найменшої вiдстанi. Перевiрено гiпотези про закон розподiлу за допомогою критерiю Колмогорова — Смiрнова, а також критерiїв AIC, BIC. Вiдповiдний пiдбiр, як приклад застосування, проведено для зiмiтованого за спецiальним авторським алгоритмом розподiлу максимуму звуження поля Ченцова на певну криву засобами мови R