Psychometrics in Practice at RCEC

A broad range of topics is dealt with in this volume: from combining the psychometric generalizability and item response theories to the ideas for an integrated formative use of data-driven decision making, assessment for learning and diagnostic testing. A number of chapters pay attention to computerized (adaptive) and classification testing. Other chapters treat the quality of testing in a general sense, but for topics like maintaining standards or the testing of writing ability, the quality of testing is dealt with more specifically.\ud All authors are connected to RCEC as researchers. They present one of their current research topics and provide some insight into the focus of RCEC. The selection of the topics and the editing intends that the book should be of special interest to educational researchers, psychometricians and practitioners in educational assessment

Neuro-fuzzy knowledge processing in intelligent learning environments for improved student diagnosis

In this paper, a neural network implementation for a fuzzy logic-based model of the diagnostic process is proposed as a means to achieve accurate student diagnosis and updates of the student model in Intelligent Learning Environments. The neuro-fuzzy synergy allows the diagnostic model to some extent "imitate" teachers in diagnosing students' characteristics, and equips the intelligent learning environment with reasoning capabilities that can be further used to drive pedagogical decisions depending on the student learning style. The neuro-fuzzy implementation helps to encode both structured and non-structured teachers' knowledge: when teachers' reasoning is available and well defined, it can be encoded in the form of fuzzy rules; when teachers' reasoning is not well defined but is available through practical examples illustrating their experience, then the networks can be trained to represent this experience. The proposed approach has been tested in diagnosing aspects of student's learning style in a discovery-learning environment that aims to help students to construct the concepts of vectors in physics and mathematics. The diagnosis outcomes of the model have been compared against the recommendations of a group of five experienced teachers, and the results produced by two alternative soft computing methods. The results of our pilot study show that the neuro-fuzzy model successfully manages the inherent uncertainty of the diagnostic process; especially for marginal cases, i.e. where it is very difficult, even for human tutors, to diagnose and accurately evaluate students by directly synthesizing subjective and, some times, conflicting judgments

Towards an Integrative Formative Approach of Data-Driven Decision Making, Assessment for Learning, and Diagnostic Testing

This study concerns the comparison of three approaches to assessment: Data-Driven Decision Making, Assessment for Learning, and Diagnostic Testing. Although the three approaches claim to be beneficial with regard to student learning, no clear study into the relationships and distinctions between these approaches exists to date. The goal of this study was to investigate the extent to which the three approaches can be shaped into an integrative formative approach towards assessment. The three approaches were compared on nine characteristics of assessment. The results suggest that although the approaches seem to be contradictory with respect to some characteristics, it is argued that they could complement each other despite these differences. The researchers discuss how the three approaches can be shaped into an integrative formative approach towards assessmen

Success in tutoring electronic troubleshooting

Two years ago Dr. Sherrie Gott of the Air Force Human Resources Laboratory described an avionics troubleshooting tutor being developed under the Basic Job Skills Research Program. The tutor, known as Sherlock, is directed at teaching the diagnostic procedures necessary to investigate complex test equipment used to maintain F-15 fighter aircraft. Since Dr. Gott's presentation in 1987, the tutor has undergone field testing at two Air Force F-15 flying wings. The results of the field test showed that after an average of 20 hours on the tutor, the 16 airmen in the experimental group (who average 28 months of experience) showed significant performance gains when compared to a control group (having a mean experience level of 37 months) who continued participating in the existing on-the-job training program. Troubleshooting performance of the tutored group approached the level of proficiency of highly experienced airmen (averaging approximately 114 months of experience), and these performance gains were confirmed in delayed testing six months following the intervention. The tutor is currently undergoing a hardware and software conversion form a Xerox Lisp environment to a PC-based environment using an object-oriented programming language. Summarized here are the results of the successful field test. The focus is on: (1) the instructional features that contributed to Sherlock's success; and (2) the implementation of these features in the PC-based version of the avionics troubleshooting tutor

Mobile Robot Lab Project to Introduce Engineering Students to Fault Diagnosis in Mechatronic Systems

This document is a self-archiving copy of the accepted version of the paper. Please find the final published version in IEEEXplore: http://dx.doi.org/10.1109/TE.2014.2358551This paper proposes lab work for learning fault detection and diagnosis (FDD) in mechatronic systems. These skills are important for engineering education because FDD is a key capability of competitive processes and products. The intended outcome of the lab work is that students become aware of the importance of faulty conditions and learn to design FDD strategies for a real system. To this end, the paper proposes a lab project where students are requested to develop a discrete event dynamic system (DEDS) diagnosis to cope with two faulty conditions in an autonomous mobile robot task. A sample solution is discussed for LEGO Mindstorms NXT robots with LabVIEW. This innovative practice is relevant to higher education engineering courses related to mechatronics, robotics, or DEDS. Results are also given of the application of this strategy as part of a postgraduate course on fault-tolerant mechatronic systems.This work was supported in part by the Spanish CICYT under Project DPI2011-22443