New approaches to measuring emotional intelligence

New scoring and test construction methods for emotional intelligence (EI) are suggested as alternatives for current practice, where most tests are scored by group judgment and are in ratings-based format. Both the ratings-based format and the proportion-based scores resulting from group judgments may act as method effects, obscuring relationships between EI tests, and between EI and intelligence. In addition, scoring based on standards rather than group judgments add clarity to the meaning of test scores. For these reasons, two new measures of emotional intelligence (EI) are constructed: (1) the Situational Test of Emotional Understanding (STEU); and (2) the Situational Test of Emotion Management (STEM). Following test construction, validity evidence is collected from four multi-variate studies. The STEU’s items and a standards-based scoring system are developed according to empirically derived appraisal theory concerning the structure of emotion [Roseman, 2001]. The STEM is developed as a Situational Judgment Test (SJT) with situations representing sadness, fear and anger in work life and personal life settings. Two qualitative studies form the basis for the STEM’s item development: (1) content analysis of responses to semi-structured interviews with 31 psychology undergraduates and 19 community volunteers; and (2) content analysis of free responses to targeted vignettes created from these semi-structured interviews (N = 99). The STEM may be scored according to two expert panels of emotions researchers, psychologists, therapists and life coaches (N = 12 and N = 6). In the first multi-variate study (N = 207 psychology undergraduates), both STEU and STEM scores relate strongly to vocabulary test scores and moderately to Agreeableness but no other dimension from the five-factor model of personality. STEU scores predict psychology grade and an emotionally-oriented thinking style after controlling vocabulary and personality test scores (ΔR2 = .08 and .06 respectively). STEM scores did not predict academic achievement but did predict emotionally-oriented thinking and life satisfaction (ΔR2 = .07 and .05 for emotionally-oriented thinking and .04 for life satisfaction). In the second multi-variate study, STEU scores predict lower levels of state anxiety, and STEM scores predict lower levels of state anxiety, depression, and stress among 149 community volunteers from Sydney, Australia. In the third multi-variate study (N = 181 psychology undergraduates), Strategic EI, fluid intelligence (Gf) and crystallized intelligence (Gc) were each measured with three indicators, allowing these constructs to be assessed at the latent variable level. Nested structural equation models show that Strategic EI and Gc form separate latent factors (Δχ2(1) = 12.44, p < .001). However, these factors relate very strongly (r = .73), indicating that Strategic EI may be a primary mental ability underlying Gc. In this study, STEM scores relate to emotionally-oriented thinking but not loneliness, life satisfaction or state stress, and STEU scores do not relate to any of these. STEM scores are significantly and meaningfully higher for females (d = .80), irrespective of gender differences in verbal ability or personality, or whether expert scores are derived from male or female experts. The fourth multi-variate study (N = 118 psychology undergraduates) distinguishes an EI latent factor (indicated by scores on the STEU, STEM and two emotion recognition ability measures) from a general cognitive ability factor (indicated by three intelligence measures; Δχ2(1) = 10.49, p < .001), although again cognitive ability and EI factors were strongly related (r = .66). Again, STEM scores were significantly higher for females (d = .44) and both STEU and STEM relate to Agreeableness but not to any other dimension from the five-factor model of personality. Taken together, results suggest that: (1) STEU and STEM scores are reasonably reliable and valid tests of EI; (2) EI tests assess slightly different constructs to existing measures of Gc, but more likely form a new primary mental ability within Gc than an entirely separate construct; and (3) the female superiority for EI tests may prove useful for addressing adverse impact in applied settings (e.g., selection for employment, promotion or educational opportunities), particularly given that many current assessment tools result in a male advantage

New approaches to measuring emotional intelligence

New scoring and test construction methods for emotional intelligence (EI) are suggested as alternatives for current practice, where most tests are scored by group judgment and are in ratings-based format. Both the ratings-based format and the proportion-based scores resulting from group judgments may act as method effects, obscuring relationships between EI tests, and between EI and intelligence. In addition, scoring based on standards rather than group judgments add clarity to the meaning of test scores. For these reasons, two new measures of emotional intelligence (EI) are constructed: (1) the Situational Test of Emotional Understanding (STEU); and (2) the Situational Test of Emotion Management (STEM). Following test construction, validity evidence is collected from four multi-variate studies. The STEU’s items and a standards-based scoring system are developed according to empirically derived appraisal theory concerning the structure of emotion [Roseman, 2001]. The STEM is developed as a Situational Judgment Test (SJT) with situations representing sadness, fear and anger in work life and personal life settings. Two qualitative studies form the basis for the STEM’s item development: (1) content analysis of responses to semi-structured interviews with 31 psychology undergraduates and 19 community volunteers; and (2) content analysis of free responses to targeted vignettes created from these semi-structured interviews (N = 99). The STEM may be scored according to two expert panels of emotions researchers, psychologists, therapists and life coaches (N = 12 and N = 6). In the first multi-variate study (N = 207 psychology undergraduates), both STEU and STEM scores relate strongly to vocabulary test scores and moderately to Agreeableness but no other dimension from the five-factor model of personality. STEU scores predict psychology grade and an emotionally-oriented thinking style after controlling vocabulary and personality test scores (ΔR2 = .08 and .06 respectively). STEM scores did not predict academic achievement but did predict emotionally-oriented thinking and life satisfaction (ΔR2 = .07 and .05 for emotionally-oriented thinking and .04 for life satisfaction). In the second multi-variate study, STEU scores predict lower levels of state anxiety, and STEM scores predict lower levels of state anxiety, depression, and stress among 149 community volunteers from Sydney, Australia. In the third multi-variate study (N = 181 psychology undergraduates), Strategic EI, fluid intelligence (Gf) and crystallized intelligence (Gc) were each measured with three indicators, allowing these constructs to be assessed at the latent variable level. Nested structural equation models show that Strategic EI and Gc form separate latent factors (Δχ2(1) = 12.44, p < .001). However, these factors relate very strongly (r = .73), indicating that Strategic EI may be a primary mental ability underlying Gc. In this study, STEM scores relate to emotionally-oriented thinking but not loneliness, life satisfaction or state stress, and STEU scores do not relate to any of these. STEM scores are significantly and meaningfully higher for females (d = .80), irrespective of gender differences in verbal ability or personality, or whether expert scores are derived from male or female experts. The fourth multi-variate study (N = 118 psychology undergraduates) distinguishes an EI latent factor (indicated by scores on the STEU, STEM and two emotion recognition ability measures) from a general cognitive ability factor (indicated by three intelligence measures; Δχ2(1) = 10.49, p < .001), although again cognitive ability and EI factors were strongly related (r = .66). Again, STEM scores were significantly higher for females (d = .44) and both STEU and STEM relate to Agreeableness but not to any other dimension from the five-factor model of personality. Taken together, results suggest that: (1) STEU and STEM scores are reasonably reliable and valid tests of EI; (2) EI tests assess slightly different constructs to existing measures of Gc, but more likely form a new primary mental ability within Gc than an entirely separate construct; and (3) the female superiority for EI tests may prove useful for addressing adverse impact in applied settings (e.g., selection for employment, promotion or educational opportunities), particularly given that many current assessment tools result in a male advantage

The Use of Rasch Modeling To Improve Standard Setting

As a wider variety of meaningful assessment strategies come into more prominent classroom use, teachers are called upon to craft scoring rubrics which validly and reliably assess students\u27 knowledge and abilities. The creation of instructionally sound rubrics can be time consuming, and many teachers feeling the pinch of time pressures are turning to rubric resources from the World Wide Web for assistance. The purposes of this paper are to review the issues surrounding the creation of instructionally sound rubrics, to examine how those issues apply to online rubric banks and rubric generators, and to offer guidelines for how educators can use online resources to best support the creation of meaningful and effective rubrics. Accessed 20,478 times on https://pareonline.net from January 24, 2006 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right

Item Banking with Embedded Standards

An item banking method that does not use Item Response Theory (IRT) is described. This method provides a comparable grading system across schools that would be suitable for low-stakes testing. It uses the Angoff standard-setting method to obtain item ratings that are stored with each item. An example of such a grading system is given, showing how a grade and a scaled score could be calculated for a particular student. Accessed 10,825 times on https://pareonline.net from October 14, 2009 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right

Faking good and bad on self‐reports versus informant‐reports of Dark Triad personality

Research consistently demonstrates that people can distort their responses on self‐report personality tests. Informant‐reports (where a knowledgeable informant rates a target's personality) can be used as an alternative to self‐ratings. However, there has been little research on the extent to which informants can distort their responses on personality tests (or their motives for response distortion). The current study examines the effects of experimentally induced response distortion on self‐ and informant‐reports of the Dark Triad. The participants (N = 834 undergraduates) completed Dark Triad measures in a 2 × 3 between‐person design crossing format (self‐ vs. informant‐report [imagined friend]) with instruction condition (answer honestly, look good, or look bad). “Look good” effects were significant for both self‐reports (d = −1.22 to 1.42) and informant‐reports (d = −1.35 to 0.62). “Look bad” effects were also significant for both self‐reports (d = −0.56 to 3.58) and informant‐reports (d = −0.55 to 3.70). The Five Factor Machiavellianism Inventory results were opposite to hypotheses, but Dirty Dozen Machiavellianism results were as expected. We conclude that people can distort Dark Triad scores for themselves (self‐report) and on behalf of someone else (informant‐report). We discuss the relevance of our findings for self‐ and informant‐report assessment in applied contexts

Estimating the Standard Error of the Judging in a modified-Angoff Standards Setting Procedure

For a modified Angoff standards setting procedure, two methods of calculating the standard error of the judging were compared. The Central Limit Theorem (CLT) method is easy to calculate and uses readily available data. It estimates the variance of mean cut scores as a function of the variance of cut scores within a judging group, based on the independent judgements at Stage 1 of the process. Its theoretical drawback is that it is unable to take account of the effects of collaboration among the judges at Stages 2 and 3. The second method, an application of equipercentile (EQP) equating, relies on the selection of very large stable candidatures and the standardisation of the raw score distributions to remove effects associated with test difficulty. The standard error estimates were then empirically obtained from the mean cut score variation observed over a five year period. For practical purposes, the two methods gave reasonable agreement, with the CLT method working well for the top band, the band that attracts most public attention. For some bands in English and Mathematics, the CLT standard error was smaller than the EQP estimate, suggesting the CLT method be used with caution as an approximate guide only. Accessed 31,793 times on https://pareonline.net from March 01, 2004 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right

Teacher personality and teacher effectiveness in secondary school: Personality predicts teacher support and student self-efficacy but not academic achievement

Students' educational outcomes are predicted by their noncognitive characteristics, including Big Five personality domains. Although theories of teaching and learning suggest that teacher noncognitive characteristics also impact student outcomes, such characteristics are rarely studied systematically. We propose that the Big Five personality domains of teachers are associated with teacher effectiveness. Furthermore, we test two potential moderators of these relationships: (1) source of teacher personality report (student reports may show stronger effects than teacher self-reports) and (2) frame of reference (contextualized "at school" personality items for teacher self-reports may show stronger effects than noncontextualized standard personality items). Multilevel regressions were conducted on the data collected from secondary school students (N = 2,082) and their mathematics and English teachers (N = 75). We statistically controlled for student and teacher gender, student previous academic achievement, and student personality. Teacher personality predicted the subjective measures of teacher effectiveness- the strongest predictors were conscientiousness for teacher academic support, agreeableness for teacher personal support, and neuroticism for student performance self-efficacy. Teacher personality did not predict the objective measure (student academic achievement). These effects were moderated by source of personality report but not by frame of reference. The possibility of including personality as part of the initial teacher trainee selection procedure in the future is briefly discussed

Predicting breakfast consumption: A comparison of the theory of planned behaviour and the health action process approach

Purpose: Breakfast consumption is associated with a range of beneficial health outcomes including improved overall diet quality, lower BMI, decreased risk of chronic disease, and improved cognitive function. Although there are many models of health and social behaviour, there is a paucity of research utilising these in breakfast consumption and very few studies that directly compare these models. This study aims to compare the theory of planned behaviour (TPB) and the health action process approach (HAPA) in predicting breakfast consumption. Design/methodology/approach: University students (N=102; M=19.5 years) completed a questionnaire measuring demographics, TPB and HAPA motivational variables, and intentions. Behaviour and HAPA volitional variables were measured four weeks later. Findings: Using structural equation modelling, it was found that the TPB model was a superior fit to the data across a range of model indices compared to the HAPA. Both models significantly predicted both intentions and behaviour at follow up; however, the TPB predicted a higher proportion of the variance in breakfast consumption (47.6 per cent) than the HAPA (44.8 per cent). Further, the volitional variables did not mediate the intention-behaviour gap, and the data were not an adequate statistical fit to the model compared to the TPB.Research limitations/implications: The results support the use of the TPB and show that some aspects of the HAPA are useful in predicting breakfast consumption, suggesting that risk perception and self-efficacy be targeted in interventions to increase behaviour. The volitional variables did not appear to mediate breakfast consumption indicating that intention is still the strongest predictor, at least in this behaviour. Originality/value: The current study is the first to compare the TPB and HAPA in predicting breakfast consumption

The Role of Body Image on Psychosocial Outcomes in People With Diabetes and People With an Amputation

IntroductionResearch indicates that body image disturbance is associated with poorer psychosocial outcomes for individuals with physical health conditions, with poorest body image reported for individuals with visible bodily changes. Using White's (2000) theoretical model of body image the present paper aimed to examine the nature of these relationships in two distinct groups: individuals with an amputation and individuals with diabetes. It was hypothesized that body image disturbance would be associated with psychosocial outcomes and would mediate the relationships between self-ideal discrepancy and personal investment in psychosocial outcomes.MethodsIndividuals with diabetes (N = 212) and individuals with an amputation (N = 227) provided details regarding their medical condition, and completed measures assessing body image, investment, self-ideal discrepancy, depression, anxiety, and quality of life. Structural equation and invariance modeling were used to test the model paths and the invariance of the model.ResultsAs hypothesized, body image disturbance was found to mediate the relationships between personal investment and psychosocial outcome, and between self-ideal discrepancy and psychosocial outcome. The predicted paths were invariant across groups, although the model accounted for more variance in people with an amputation than people with diabetes.ConclusionBody image disturbance, personal investment, and self-ideal discrepancy are important factors contributing to psychosocial outcome for individuals with diabetes and individuals with an amputation. These findings not only confirm the validity of the model in these two groups, but they emphasize the importance of targeting body image in future psychological interventions for individuals with a health condition

Using the health action process approach to predict and improve health outcomesin individuals with type 2 diabetes mellitus

Background: The purpose of this study was to explore the predictive utility of the Health Action Process Approach (HAPA) and test a HAPA-based healthy eating intervention, in adults with type 2 diabetes mellitus. Materials and methods: The study employed a prospective, randomized, controlled trial design. The 4-month intervention consisted of self-guided HAPA-based workbooks in addition to two telephone calls to assist participants with the program implementation, and was compared to “treatment as usual”. Participants (n=87) completed health measures (diet, body mass index [BMI], waist circumference, blood pressure, blood glucose levels, lipid levels, and diabetes distress) and HAPA measures prior to the intervention and again upon completion 4 months later. Results: The overall HAPA model predicted BMI, although only risk awareness and recovery self-efficacy were significant independent contributors. Risk awareness, intentions, and self-efficacy were also independent predictors of health outcomes; however, the HAPA did not predict healthy eating. No significant time × condition interaction effects were found for diet or any HAPA outcome measures. Conclusion: Despite the success of HAPA in predicting health outcomes for those with type 2 diabetes mellitus, the intervention was unsuccessful in changing healthy eating or any of the other measured variables, and alternative low-cost health interventions for those with type 2 diabetes mellitus should be explored