Putting theory oriented evaluation into practice

Evaluations of gaming simulations and business games as teaching devices are typically end-state driven. This emphasis fails to detect how the simulation being evaluated does or does not bring about its desired consequences. This paper advances the use of a logic model approach which possesses a holistic perspective that aims at including all elements associated with the situation created by a game. The use of the logic model approach is illustrated as applied to Simgame, a board game created for secondary school level business education in six European Union countries

Reliability and validity of two widely-used worry questionnaires: self-report and self-peer convergence

The reliability and validity of the Penn State Worry Questionnaire (PSWQ) and the Worry Domains Questionnaire (WDQ) were examined with self-ratings from a non-clinical sample of 148 students in a test-retest design across four weeks. Ratings from three well-acquainted peers were also obtained. With internal consistencies and test-retest correlations of at least 0.85, the present study confirmed the high reliability of the questionnaires. Moreover, both measures demonstrated substantial convergent validity: Average agreement among peers was 0.42 (PSWQ) and 0.47 (WDQ) and aggregated self-peer agreement was 0.55 (PSWQ) and 0.49 (WDQ). Self-peer agreement was not biased by social desirability. These findings challenge views that worry is an unreliable and unobservable phenomenon

Churchmanship and personality among clergymen in the church in Wales : are Anglo-Catholic priests more feminine?

The aim of the present study is to develop and test a new measure of Anglo-Catholic orientation capable of assessing the extent of the continuing influence of the Anglican-Catholic movement among Anglican clergy and useful for testing theories regarding the association between Anglo-Catholic orientation and personality. Data provided by a sample of 232 clergymen serving in the Church in Wales support the internal consistency reliability of the 21-item Francis-Littler Anglo-Catholic Orientation Scale, and, in terms of the Eysenckian dimensional model of personality, demonstrate that Anglo-Catholic orientation is associated with higher levels of psychological femininity as assessed by the neuroticism scale, but not as assessed by the psychoticism scale

Psychometric properties of the Pelvic Organ Prolapse Symptom Score

Objective To assess the internal consistency, construct validity and sensitivity to change of a pelvic organ prolapse symptom score (POP-SS). Design Analysis of data from three prolapse studies, including symptomatic and asymptomatic women who completed the POP-SS. Setting (1) A community setting in New Zealand, (2) two gynaecology outpatient departments in Scotland and (3) a gynaecological surgery department in Scotland. Population or sample (1) Participants from a survey of postnatal women at 12-year follow up, invited to complete a prolapse questionnaire and have prolapse assessment, (2) new gynaecology outpatients presenting with prolapse symptoms, randomised to pelvic floor muscle training (PFMT) or control and (3) women having anterior and/or posterior prolapse surgery, randomised to mesh insert or no mesh. Method Data were analysed to assess internal consistency, construct validity and sensitivity to change of the POP-SS. Main outcome measures Cronbach's alpha, significance of differences in POP-SS scores between studies and significance of difference in POP-SS scores pre- to post-intervention. Results For internal consistency, Cronbach's alpha ranged from 0.723 to 0.828. Women having surgery had higher POP-SS scores than those having conservative management (mean difference 5.0, 95% CI 3.1–6.9), who in turn had higher scores than the asymptomatic women (mean difference 5.9, 95% CI 4.4–7.4). Significant differences in POP-SS score were detected after surgery and PFMT. The improvement due to surgery was significantly greater than that associated with PFMT (z =−3.006, P = 0.003). Conclusion The POP-SS has good internal consistency and construct validity and is sensitive to change.1 ProLong: University of Otago Postgraduate Scholarship in Obstetrics and Gynaecology. 2 POPPY: Health Services Research Committee grant, Chief Scientist Office, Scottish Government. 3 IMPRESS: None

Relational discrepancies and emotion : The moderating roles of relationship type and relational discrepancy valence

Peer reviewedPostprin

Implicit theories of a desire for fame

The aim of the present studies was to generate implicit theories of a desire for fame among the general population. In Study 1, we were able to develop a nine-factor analytic model of conceptions of the desire to be famous that initially comprised nine separate factors; ambition, meaning derived through comparison with others, psychologically vulnerable, attention seeking, conceitedness, social access, altruistic, positive affect, and glamour. Analysis that sought to examine replicability among these factors suggested that three factors (altruistic, positive affect, and glamour) neither display factor congruence nor display adequate internal reliability. A second study examined the validity of these factors in predicting profiles of individuals who may desire fame. The findings from this study suggested that two of the nine factors (positive affect and altruism) could not be considered strong factors within the model. Overall, the findings suggest that implicit theories of a desire for fame comprise six factors. The discussion focuses on how an implicit model of a desire for fame might progress into formal theories of a desire for fame

Measuring pregnancy planning: An assessment of the London Measure of Unplanned Pregnancy among urban, south Indian women

Copyright © 2010 Corinne H. Rocca et al. This open-access work is published under the terms of the Creative Commons Attribution NonCommercial License 2.0 Germany, which permits use, reproduction & distribution in any medium for non-commercial purposes, provided the original author(s) and source are given credit. See http:// creativecommons.org/licenses/by-nc/2.0/de/.We evaluated the psychometric properties of the London Measure of Unplanned Pregnancy among Indian women using classical methods and Item Response Modeling. The scale exhibited good internal consistency and internal structure, with overall scores correlating well with each item’s response categories. Items performed similarly for pregnant and non-pregnant women, and scores decreased with increasing parity, providing evidence for validity. Analyses detected small disadvantages, including low endorsement of middle response categories and some evidence of differential item functioning by parity. We conclude that the LMUP is suitable for use in India and recommend steps for improving scale performance for this cultural context.National Institute of Child Health and Human Development and the Levis Strauss Foundation

Achievement goals and motivational responses in tennis: Does the context matter?

Objectives: This study examined: (a) whether athletes’ goal orientations differ across training and competition; (b) whether goal orientations predict effort, enjoyment, and psychological skill use differently in training and competition; and (c) whether goal orientations predict perceived improvement in training and perceived performance in competition. Method: Participants were 116 competitive tennis players (mean age = 19.99, SD = 5.82), who completed questionnaires measuring goal orientations, effort, enjoyment, and psychological skill use in training and competition, perceived improvement in training, and perceived performance in competition. Results: Dependent t-tests revealed that athletes reported higher task orientation in training than in competition and higher ego orientation in competition than in training, while Pearson product-moment correlations revealed a high cross-contextual consistency for both task and ego goal orientations between training and competition. Regression analyses indicated that task orientation predicted positively effort, enjoyment, self-talk, and goal setting in both contexts, perceived improvement in training, and perceived performance in competition. An interaction effect also emerged whereby ego orientation predicted positively effort in competition only when task orientation was low or average. Conclusions: The findings suggest that goal orientations may differ between training and competition; task orientation is the goal that should be promoted in both contexts; and the context may affect the relationship between goal orientations and effort, enjoyment, and goal setting

The standard error of measurement is a more appropriate measure of quality for postgraduate medical assessments than is reliability: an analysis of MRCP(UK) examinations

Background: Cronbach's alpha is widely used as the preferred index of reliability for medical postgraduate examinations. A value of 0.8-0.9 is seen by providers and regulators alike as an adequate demonstration of acceptable reliability for any assessment. Of the other statistical parameters, Standard Error of Measurement (SEM) is mainly seen as useful only in determining the accuracy of a pass mark. However the alpha coefficient depends both on SEM and on the ability range (standard deviation, SD) of candidates taking an exam. This study investigated the extent to which the necessarily narrower ability range in candidates taking the second of the three part MRCP(UK) diploma examinations, biases assessment of reliability and SEM.Methods: a) The interrelationships of standard deviation (SD), SEM and reliability were investigated in a Monte Carlo simulation of 10,000 candidates taking a postgraduate examination. b) Reliability and SEM were studied in the MRCP(UK) Part 1 and Part 2 Written Examinations from 2002 to 2008. c) Reliability and SEM were studied in eight Specialty Certificate Examinations introduced in 2008-9.Results: The Monte Carlo simulation showed, as expected, that restricting the range of an assessment only to those who had already passed it, dramatically reduced the reliability but did not affect the SEM of a simulated assessment. The analysis of the MRCP(UK) Part 1 and Part 2 written examinations showed that the MRCP(UK) Part 2 written examination had a lower reliability than the Part 1 examination, but, despite that lower reliability, the Part 2 examination also had a smaller SEM (indicating a more accurate assessment). The Specialty Certificate Examinations had small Ns, and as a result, wide variability in their reliabilities, but SEMs were comparable with MRCP(UK) Part 2.Conclusions: An emphasis upon assessing the quality of assessments primarily in terms of reliability alone can produce a paradoxical and distorted picture, particularly in the situation where a narrower range of candidate ability is an inevitable consequence of being able to take a second part examination only after passing the first part examination. Reliability also shows problems when numbers of candidates in examinations are low and sampling error affects the range of candidate ability. SEM is not subject to such problems; it is therefore a better measure of the quality of an assessment and is recommended for routine use

The Perils of Positivity

The passion and productivity that characterizes research on positive organizational behavior (POB) is impressive. Yet POB research is accumulating so rapidly that it may exceed what the field's conceptual, methodological, and ideological foundation can bear. I discuss here six concerns prompted by the articles in this special issue. These concerns are (1) the emphasis of positive organizational scholarship on individual-level phenomena, (12) the ahistorical character of POB research and writing, (3) the construct validity of key concepts, (4) over-reliance on a particular research strategy, (5) implicit acceptance of fundamental flaws in how work and organizations are designed, and (6) the seductiveness of new research paradigms.Psycholog