43 research outputs found
Construct Validity and Cognitive Diagnostic Assessment
Cognitive diagnostic assessment (CDA) is increasingly a major focus in psychological and educational measurement. Instead of inferring a general response tendency or behavior consistency of an examinee over
a target domain of measurement, diagnostic assessment results provide a detailed account of the underlying cognitive basis of the examinee's performance by mining the richer information that is afforded by
specific response patterns
Problem Restructuring Processes for Ill-Structured Verbal Analogies
Copyright 1981 Psychonomic Society, Inc.Published under the name Susan E. Whitely.The processes involved in analogy solving have been an important investigative area in
cognitive psychology. Although problem restructuring has been a central construct in problem solving theory, no restructuring processes have been proposed for analogical reasoning. Yet, the stimulus terms for analogies, as they appear on ability tests, are often ill-structured. That is, they are ordered in a way that does not permit direct problem comprehension. In the current study, both perceptual and semantic problem restructuring processes were hypothesized for analogy solving. The independence, stage of execution, and susceptibility to
strategic control of the two processes were examined. The results from two experiments
indicated that (1) ill-structured analogies are restructured during problem solving, (2) perceptual and semantic restructuring processes are independent and executed at different stages of analogy solving, and (3) both processes exhibited automaticity since repetition of analogy solution attenuated but did not eliminate either restructuring process. A model of analogical reasoning that incorporated both restructuring processes and their execution sequences was
proposed. The nature and automaticity of perceptual and semantic analogy restructuring
processes were discussed
Hyperarousal symptoms after traumatic and nontraumatic births
Background: Measurement is critical in postnatal posttraumatic stress disorder (PTSD) because symptoms may be influenced by normal postnatal phenomena such as physiological changes and fatigue. Objective: This study examined: (1) whether hyperarousal symptoms differ between women who have traumatic or nontraumatic births; (2) whether the construct of hyperarousal is coherent in postnatal women; and (3) whether hyperarousal symptoms are useful for identifying women who have traumatic births or PTSD. Methods: A survey of PTSD symptoms in 1,078 women recruited via the community or Internet who completed an online or paper questionnaire measuring childbirth-related PTSD symptoms between 1 and 36 months after birth. Women who had a traumatic birth as defined by DSM-IV criterion A (n = 458) were compared with women who did not have a traumatic birth (n = 591). Results: A one-factor dimension of hyperarousal was identified that included all five hyperarousal items. Diagnostic criteria of two or more hyperarousal symptoms in the previous week were reported by 75.3% of women with traumatic birth and 50.5% of women with nontraumatic births. The difference in mean hyperarousal symptoms between groups was substantial at 0.76 of a standard deviation (Hedge’s g, CI = 0.64, 0.89). A larger difference was observed between women with and without diagnostic PTSD (g = 1.64, CI 1.46, 1.81). However, receiver operating characteristic analyses showed hyperarousal symptoms have poor specificity and alternative ways of calculating symptoms did not improve this. Comparison with other PTSD symptoms found re-experiencing symptoms were most accurate at identifying women with traumatic births. Conclusions: Results suggest hyperarousal symptoms are associated with traumatic birth and are a coherent construct in postnatal women. However, they have poor specificity and should only be used as part of diagnostic criteria, not as a sole indicator
Cognitive Design Principles and the Successful Performer: A Study on Spatial Ability
An important trend in educational measurement is the use of principles of
cognitive psychology to design achievement and ability test items. Many studies
show that manipulating the stimulus features of items influences the processes,
strategies, and knowledge structures that are involved in solution. However,
little is known about how cognitive design influences individual differences.
That is, does applying cognitive design principles change the background skills
and abilities that are associated with successful performance? This study compared
the correlates of two spatial ability tests that used the same item type but different test design principles (cognitive design versus psychometric design). The results indicated differences in factorial complexity in the two tests; specifically, the impact of verbal abilities was substantially reduced by applying the cognitive design principles
Measuring Human Intelligence with Artificial Intelligence
Adaptive item generation may be the next innovation in intelligence testing.
In adaptive item generation, the optimally informative item is developed anew for the examinee during the test. Reminiscent of computer versus person chess games, the computer generates the next item based on the
previous pattern of the examinee's responses. Adaptive item generation
requires the merger of two lines of research, psychometric methods for
adaptive testing and a cognitive analysis of items
Measuring and Validating Cognitive Modifiability as an Ability: A Study in the Spatial Domain
Measuring cognitive modifiability from the responsiveness of an individual's
performance to intervention has long been viewed (e.g., Dearborne, 1921) as an
alternative to traditional (static) ability measurement. Currently, dynamic testing
in which cues or instruction are presented with ability test items, is a popular
method for assessing cognitive modifiability. Despite the long-standing interest, however, little data exists to support the validity of cognitive modifiability measures in any ability domain. Several special methodological difficulties have limited validity studies, including psychometric problems in measuring modifiability (i.e., as change), lack of appropriate validation criteria, and difficulty in linking modifiability to cognitive theory. In this article, relatively new developments for solving the validation problems are applied to measuring and validating spatial modifiability. Criterion-related validity for predicting learning in an applied knowledge domain, as well as construct validity, is supported
Models, Meanings and Misunderstandings: Some Issues in Applying Rasch's Theory
Published under the name Susan E. Whitely.Wright (1977) shows that a debate is developing between those who strongly advocate use of the Rasch model and those who have certain reservations about the extent to which the model meets some traditional concerns in trait measurement. In an earlier article, Whitely and Dawis (1974) presented the Rasch model in the context of least squares estimation, and noted some features that may limit the utility of the model in test development. Wright (1977) questions several of the specific interpretations and conclusions that were given in the earlier article. The current article is a response to those questions
The Second Century of Ability Testing: Some Predictions and Speculations
The seventh annual William H. Angoff Memorial Lecture was presented at Educational Testing Service, Princeton, New Jersey, on January 11, 2001. Copyright © 2003 by Educational Testing Service. All rights reserved. Educational Testing
Service is an Affirmative Action/Equal Opportunity Employer. Educational Testing Service, ETS, and the ETS logos are
registered trademarks of Educational
Testing Service.Dazzling changes in many areas, such as technology and communications, marked the 20th century—the first century of ability testing. Predictions about the second century of testing seem difficult in such a context. Yet, looking back to the turn of the last century, Kirkpatrick (1900) in his APA presidential address presented fundamental desiderata for ability testing (normative age standards, emphasis on culture-general tasks, simultaneous measurement of many persons, and adult ability measurement) that, in fact, guides major testing research even today. An overview of the last century shows that most fundamental principles in psychometrics and testing were available by 1930. With a few
notable exceptions, the remainder of the last century of testing was devoted to applying or refining these principles.
I predict that the same pattern will occur in this century of testing. Further developments in model-based measurement
and cognitive psychology principles in testing, intermingled with technology, will guide ability testing throughout
the next century. These changes, which I will elaborate in detail, include fundamental changes in test development
procedures, the nature of the measuring tasks, aspects of ability that are measured, and types of interpretations given
to ability
A general latent trait model for response processes
latent trait models, item response theory, aptitude process model,