16 research outputs found
Estimating measurement error on highly speeded tests
Despite many advances in test theory, estimating
measurement error which arises from temporary
changes in the person or test situation has remained
relatively unchanged. Unfortunately, not only
are short-term instabilities the most important
source of measurement error for many traits,
especially those measured from highly speeded
tests; but the classical test-retest formula for estimating
error is based on untenable assumptions
with respect to practice effects. The current paper
presents a method which gives maximum likelihood
estimates of measurement error within the context
of a simplex model for practice effects. The appropriateness
of the model is tested for five traits, and
error estimates are compared to the classical formula
estimates.Whitely, Susan E.. (1979). Estimating measurement error on highly speeded tests. Retrieved from the University Digital Conservancy, https://hdl.handle.net/11299/99574
Individual inconsistency: Implications for test reliability and behavioral predictability
The nature of individual inconsistency in performance
on trait measurements is an important
topic in psychometrics because of its direct relevance
to measurement reliability. Several studies
have supported short-term inconsistency as a systematic
source of variation among individuals by
finding some evidence for generalizability and relationship
to behavioral predictability. However, these
findings are questionable, since these studies confounded
change with short-term fluctuation in their
response inconsistency measure. The current research
separates these two sources of inconsistency
in a reanalysis of the data from one major study on
short-term consistency and finds little evidence for
generalizability or a relationship to behavioral predictability.
These results support the popular assumption
that measurement error from short-term
fluctuations is not due to systematic individual differences
in response consistency, as well as supporting
a more limited definition of the individual
inconsistency construct.Whitely, Susan E.. (1978). Individual inconsistency: Implications for test reliability and behavioral predictability. Retrieved from the University Digital Conservancy, https://hdl.handle.net/11299/99455
Information structure for geometric analogies: A test theory approach
Although geometric analogies are popular items
for measuring intelligence, the information processes
that are involved in their solution have not
been studied in a test theory context. In the current
study, processing is examined by testing alternative
models of information structure on geometric
analogies. In contrast to the treatment of models in
other studies that have appeared in the cognitive
literature, the models are tested jointly as mathematical
models of processing and as latent trait
models of individual differences. The joint modeling
was achieved by applying the one-parameter linear
logistic latent trait model to predict response
accuracy from information structure. The results
supported the model that distinguished between
spatial distortion and spatial displacement transformations,
which have opposite effects on item difficulty.
Further, no significant sex difference in overall
accuracy or processing were observed. Implications
of the results for processing mechanisms and
test design are discussed.Whitely, Susan E.; Schneider, Lisa M.. (1981). Information structure for geometric analogies: A test theory approach. Retrieved from the University Digital Conservancy, https://hdl.handle.net/11299/100415
Analyses and validation of central assessment instruments of the research program TEDS-M
The TEDS-Validate project has been carried out within the research program derived from Teacher Education and Development Study: Learning to Teach Mathematics (TEDS-M). In this chapter, we describe the aim of the study, which is related to the question of whether research findings brought forward by measurement instruments to test professional competence of mathematics teachers have predictive validity for the quality of their classroom instruction and the learning gains of their students. Based on this, we question whether situation-specific skills (measured via video-based assessments) contribute to explain instructional quality and learning gains of students – in addition to the effects of professional knowledge of teachers. To answer the research questions, data was collected in Thuringia, Saxony and Hesse from 2016 to 2019 with a survey of 113 in-service teachers. They were tested using web-based instruments to capture their mathematics, mathematics pedagogical and general pedagogical knowledge as well as their noticing competencies. TEDS-Validate points out the broad applicability of the instruments for the comprehensive measurement of mathematics teachers competencies. To evaluate the effects of prospective teachers’ practical activities during their school practicum within the master study of initial teacher education, these instruments will be applied in a follow-up transfer project