Some procedures for computerized ability testing

For computerized test systems to be operational, the use of item response theory is a prerequisite. As opposed to classical test theory, in item response models the abilities of the examinees and the properties of the items are parameterized separately. Hence, when measuring the abilities of examinees, the model implicitly corrects for the item properties, and measurement on an item-independent scale is possible. In addition, item response theory offers the use of test and item information as local reliability indices defined on the ability scale. In this chapter, it is shown how the main features of item response theory have given rise to the development of promising procedures for computerized testing. Among the topics discussed are procedures for item bank calibration, automated test construction, adaptive test administration, generating norm distributions, and diagnosing test scores

Comparing performance of statistical models for individual’s ability index and ranking

Efficient allocation of resources is the basic problem in economics. Firms, educational institutions, universities are faces problem of estimating true abilities and ranking of individuals to be selected for job, admissions and scholarship awards etc. This study will provide a guide line what technique should to be used for estimating true ability indices and ranking that reveals ability with maximum efficiency as well as it clearly has the advantage of differentiating among individuals having equal raw score. Two major theories Classical Testing Theory and Item Response Theory have been using in the literature. We design two different Monte Carlo studies to investigate which theory is better and which model perform more efficiently. By discussing the weaknesses of CTT this study proved that IRT is superior to CTT. Different IRT models have been used in literature; we measured the performance of these models and found that Logistic P2 model is best model. By using this best model we estimate the ability indices of the students on the basis of their entry test scores and then compare with their abilities obtained from final board examination result (used as proxy of true abilities). This is a reasonable because the final exam consists of various papers and chance variation in ability Index is a minimum. With real life application this study also proved that IRT estimate the true abilities more efficiently as compared to classical methodology.Ability Index, Monte Carlo study, Logistic and Probit models, Item Response Theory, Classical Test Theory, Ranking of Students

Happy software developers solve problems better: psychological measurements in empirical software engineering

For more than 30 years, it has been claimed that a way to improve software developers' productivity and software quality is to focus on people and to provide incentives to make developers satisfied and happy. This claim has rarely been verified in software engineering research, which faces an additional challenge in comparison to more traditional engineering fields: software development is an intellectual activity and is dominated by often-neglected human aspects. Among the skills required for software development, developers must possess high analytical problem-solving skills and creativity for the software construction process. According to psychology research, affects-emotions and moods-deeply influence the cognitive processing abilities and performance of workers, including creativity and analytical problem solving. Nonetheless, little research has investigated the correlation between the affective states, creativity, and analytical problem-solving performance of programmers. This article echoes the call to employ psychological measurements in software engineering research. We report a study with 42 participants to investigate the relationship between the affective states, creativity, and analytical problem-solving skills of software developers. The results offer support for the claim that happy developers are indeed better problem solvers in terms of their analytical abilities. The following contributions are made by this study: (1) providing a better understanding of the impact of affective states on the creativity and analytical problem-solving capacities of developers, (2) introducing and validating psychological measurements, theories, and concepts of affective states, creativity, and analytical-problem-solving skills in empirical software engineering, and (3) raising the need for studying the human factors of software engineering by employing a multidisciplinary viewpoint.Comment: 33 pages, 11 figures, published at Peer

Examining the relationship between abstract thinking ability and academic performance in object oriented programming : a psychometric and theoretical analysis of student capacity

Published ArticleAbstract reasoning involves flexible thinking as well as solving problems creatively and logically. Evidence suggest that especially first year students that undertake Information Technology studies at institutions of higher education, find it challenging to successfully comprehend concepts regarding object oriented programming (OOP). This study is based on the hypothetical premise that student performance regarding OOP can be linked to the abstract thinking ability of the particular student because of the highly abstract nature of OOP concepts. A comparison between psychometric test results gauging abstract thinking abilities of students and student performance in OOP are also presented and argued

Measuring Youth Program Quality: A Guide to Assessment Tools

Thanks to growing interest in the subject of youth program quality, many tools are now available to help organizations and systems assess and improve quality. Given the size and diversity of the youth-serving sector, it is unrealistic to expect that any one tool or process will fit all programs or circumstances. This report compares the purpose, history, structure, methodology, content and technical properties of nine different program observation tools

Achievement test construction using 0-1 linear programming

In educational testing the work of professional test agencies has shown a trend towards item banking. Achievement test construction is viewed as selecting items from a test item bank such that certain specifications are met. As the number of possible tests is large and practice usually imposes various constraints on the selection process, a mathematical programming approach is obvious. In this paper it is shown how to formulate achievement test construction as a 0¿1 linear programming problem. A heuristic for solving the problem is proposed and two examples are given. It is concluded that a 0¿1 linear programming approach fits the problem of test construction in an appropriate way and offers test agencies the possibility of computerizing their services

Development and Application of a Rasch Model Measure of Student Competency in University Introductory Computer Programming

University computer programming instruction nomenclature commonly uses the term Computer Science 1 (CS1) to describe introductory units of study. Success in CS1 is important as a pre-requisite for further study in programming and related disciplines. It is important to measure student progress and the antecedent influences. This study applied the Rasch Model and Messick’s Unified Theory of Validity to construct an interval level measure of CS1 competency with demonstrable suitability for this purpose

Towards a Theory of Software Development Expertise

Software development includes diverse tasks such as implementing new features, analyzing requirements, and fixing bugs. Being an expert in those tasks requires a certain set of skills, knowledge, and experience. Several studies investigated individual aspects of software development expertise, but what is missing is a comprehensive theory. We present a first conceptual theory of software development expertise that is grounded in data from a mixed-methods survey with 335 software developers and in literature on expertise and expert performance. Our theory currently focuses on programming, but already provides valuable insights for researchers, developers, and employers. The theory describes important properties of software development expertise and which factors foster or hinder its formation, including how developers' performance may decline over time. Moreover, our quantitative results show that developers' expertise self-assessments are context-dependent and that experience is not necessarily related to expertise.Comment: 14 pages, 5 figures, 26th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2018), ACM, 201