New approaches to measuring emotional intelligence

New scoring and test construction methods for emotional intelligence (EI) are suggested as alternatives for current practice, where most tests are scored by group judgment and are in ratings-based format. Both the ratings-based format and the proportion-based scores resulting from group judgments may act as method effects, obscuring relationships between EI tests, and between EI and intelligence. In addition, scoring based on standards rather than group judgments add clarity to the meaning of test scores. For these reasons, two new measures of emotional intelligence (EI) are constructed: (1) the Situational Test of Emotional Understanding (STEU); and (2) the Situational Test of Emotion Management (STEM). Following test construction, validity evidence is collected from four multi-variate studies. The STEU’s items and a standards-based scoring system are developed according to empirically derived appraisal theory concerning the structure of emotion [Roseman, 2001]. The STEM is developed as a Situational Judgment Test (SJT) with situations representing sadness, fear and anger in work life and personal life settings. Two qualitative studies form the basis for the STEM’s item development: (1) content analysis of responses to semi-structured interviews with 31 psychology undergraduates and 19 community volunteers; and (2) content analysis of free responses to targeted vignettes created from these semi-structured interviews (N = 99). The STEM may be scored according to two expert panels of emotions researchers, psychologists, therapists and life coaches (N = 12 and N = 6). In the first multi-variate study (N = 207 psychology undergraduates), both STEU and STEM scores relate strongly to vocabulary test scores and moderately to Agreeableness but no other dimension from the five-factor model of personality. STEU scores predict psychology grade and an emotionally-oriented thinking style after controlling vocabulary and personality test scores (ΔR2 = .08 and .06 respectively). STEM scores did not predict academic achievement but did predict emotionally-oriented thinking and life satisfaction (ΔR2 = .07 and .05 for emotionally-oriented thinking and .04 for life satisfaction). In the second multi-variate study, STEU scores predict lower levels of state anxiety, and STEM scores predict lower levels of state anxiety, depression, and stress among 149 community volunteers from Sydney, Australia. In the third multi-variate study (N = 181 psychology undergraduates), Strategic EI, fluid intelligence (Gf) and crystallized intelligence (Gc) were each measured with three indicators, allowing these constructs to be assessed at the latent variable level. Nested structural equation models show that Strategic EI and Gc form separate latent factors (Δχ2(1) = 12.44, p < .001). However, these factors relate very strongly (r = .73), indicating that Strategic EI may be a primary mental ability underlying Gc. In this study, STEM scores relate to emotionally-oriented thinking but not loneliness, life satisfaction or state stress, and STEU scores do not relate to any of these. STEM scores are significantly and meaningfully higher for females (d = .80), irrespective of gender differences in verbal ability or personality, or whether expert scores are derived from male or female experts. The fourth multi-variate study (N = 118 psychology undergraduates) distinguishes an EI latent factor (indicated by scores on the STEU, STEM and two emotion recognition ability measures) from a general cognitive ability factor (indicated by three intelligence measures; Δχ2(1) = 10.49, p < .001), although again cognitive ability and EI factors were strongly related (r = .66). Again, STEM scores were significantly higher for females (d = .44) and both STEU and STEM relate to Agreeableness but not to any other dimension from the five-factor model of personality. Taken together, results suggest that: (1) STEU and STEM scores are reasonably reliable and valid tests of EI; (2) EI tests assess slightly different constructs to existing measures of Gc, but more likely form a new primary mental ability within Gc than an entirely separate construct; and (3) the female superiority for EI tests may prove useful for addressing adverse impact in applied settings (e.g., selection for employment, promotion or educational opportunities), particularly given that many current assessment tools result in a male advantage

New approaches to measuring emotional intelligence

New scoring and test construction methods for emotional intelligence (EI) are suggested as alternatives for current practice, where most tests are scored by group judgment and are in ratings-based format. Both the ratings-based format and the proportion-based scores resulting from group judgments may act as method effects, obscuring relationships between EI tests, and between EI and intelligence. In addition, scoring based on standards rather than group judgments add clarity to the meaning of test scores. For these reasons, two new measures of emotional intelligence (EI) are constructed: (1) the Situational Test of Emotional Understanding (STEU); and (2) the Situational Test of Emotion Management (STEM). Following test construction, validity evidence is collected from four multi-variate studies. The STEU’s items and a standards-based scoring system are developed according to empirically derived appraisal theory concerning the structure of emotion [Roseman, 2001]. The STEM is developed as a Situational Judgment Test (SJT) with situations representing sadness, fear and anger in work life and personal life settings. Two qualitative studies form the basis for the STEM’s item development: (1) content analysis of responses to semi-structured interviews with 31 psychology undergraduates and 19 community volunteers; and (2) content analysis of free responses to targeted vignettes created from these semi-structured interviews (N = 99). The STEM may be scored according to two expert panels of emotions researchers, psychologists, therapists and life coaches (N = 12 and N = 6). In the first multi-variate study (N = 207 psychology undergraduates), both STEU and STEM scores relate strongly to vocabulary test scores and moderately to Agreeableness but no other dimension from the five-factor model of personality. STEU scores predict psychology grade and an emotionally-oriented thinking style after controlling vocabulary and personality test scores (ΔR2 = .08 and .06 respectively). STEM scores did not predict academic achievement but did predict emotionally-oriented thinking and life satisfaction (ΔR2 = .07 and .05 for emotionally-oriented thinking and .04 for life satisfaction). In the second multi-variate study, STEU scores predict lower levels of state anxiety, and STEM scores predict lower levels of state anxiety, depression, and stress among 149 community volunteers from Sydney, Australia. In the third multi-variate study (N = 181 psychology undergraduates), Strategic EI, fluid intelligence (Gf) and crystallized intelligence (Gc) were each measured with three indicators, allowing these constructs to be assessed at the latent variable level. Nested structural equation models show that Strategic EI and Gc form separate latent factors (Δχ2(1) = 12.44, p < .001). However, these factors relate very strongly (r = .73), indicating that Strategic EI may be a primary mental ability underlying Gc. In this study, STEM scores relate to emotionally-oriented thinking but not loneliness, life satisfaction or state stress, and STEU scores do not relate to any of these. STEM scores are significantly and meaningfully higher for females (d = .80), irrespective of gender differences in verbal ability or personality, or whether expert scores are derived from male or female experts. The fourth multi-variate study (N = 118 psychology undergraduates) distinguishes an EI latent factor (indicated by scores on the STEU, STEM and two emotion recognition ability measures) from a general cognitive ability factor (indicated by three intelligence measures; Δχ2(1) = 10.49, p < .001), although again cognitive ability and EI factors were strongly related (r = .66). Again, STEM scores were significantly higher for females (d = .44) and both STEU and STEM relate to Agreeableness but not to any other dimension from the five-factor model of personality. Taken together, results suggest that: (1) STEU and STEM scores are reasonably reliable and valid tests of EI; (2) EI tests assess slightly different constructs to existing measures of Gc, but more likely form a new primary mental ability within Gc than an entirely separate construct; and (3) the female superiority for EI tests may prove useful for addressing adverse impact in applied settings (e.g., selection for employment, promotion or educational opportunities), particularly given that many current assessment tools result in a male advantage

Faking good and bad on self‐reports versus informant‐reports of Dark Triad personality

Research consistently demonstrates that people can distort their responses on self‐report personality tests. Informant‐reports (where a knowledgeable informant rates a target's personality) can be used as an alternative to self‐ratings. However, there has been little research on the extent to which informants can distort their responses on personality tests (or their motives for response distortion). The current study examines the effects of experimentally induced response distortion on self‐ and informant‐reports of the Dark Triad. The participants (N = 834 undergraduates) completed Dark Triad measures in a 2 × 3 between‐person design crossing format (self‐ vs. informant‐report [imagined friend]) with instruction condition (answer honestly, look good, or look bad). “Look good” effects were significant for both self‐reports (d = −1.22 to 1.42) and informant‐reports (d = −1.35 to 0.62). “Look bad” effects were also significant for both self‐reports (d = −0.56 to 3.58) and informant‐reports (d = −0.55 to 3.70). The Five Factor Machiavellianism Inventory results were opposite to hypotheses, but Dirty Dozen Machiavellianism results were as expected. We conclude that people can distort Dark Triad scores for themselves (self‐report) and on behalf of someone else (informant‐report). We discuss the relevance of our findings for self‐ and informant‐report assessment in applied contexts

Teacher personality and teacher effectiveness in secondary school: Personality predicts teacher support and student self-efficacy but not academic achievement

Students' educational outcomes are predicted by their noncognitive characteristics, including Big Five personality domains. Although theories of teaching and learning suggest that teacher noncognitive characteristics also impact student outcomes, such characteristics are rarely studied systematically. We propose that the Big Five personality domains of teachers are associated with teacher effectiveness. Furthermore, we test two potential moderators of these relationships: (1) source of teacher personality report (student reports may show stronger effects than teacher self-reports) and (2) frame of reference (contextualized "at school" personality items for teacher self-reports may show stronger effects than noncontextualized standard personality items). Multilevel regressions were conducted on the data collected from secondary school students (N = 2,082) and their mathematics and English teachers (N = 75). We statistically controlled for student and teacher gender, student previous academic achievement, and student personality. Teacher personality predicted the subjective measures of teacher effectiveness- the strongest predictors were conscientiousness for teacher academic support, agreeableness for teacher personal support, and neuroticism for student performance self-efficacy. Teacher personality did not predict the objective measure (student academic achievement). These effects were moderated by source of personality report but not by frame of reference. The possibility of including personality as part of the initial teacher trainee selection procedure in the future is briefly discussed

The link between mathematics engagement and personality

BACKGROUND Prior educational research has often focused on how environmental factors impact student educational outcomes such as classroom engagement. However, it has yet to be considered in educational research, how the personality of the student and the student’s perception of their teacher’s personality may influence these outcomes. AIMS Using the dominant personality model (the Big-Five Model), the current study investigated whether student personality and student-rated teacher personality predicted students’ course grades, motivation at university, and classroom engagement. SAMPLE Data were collected from 137 Australian university students from first year undergraduate mathematics courses. Of these, 80 were female and the mean age of the total sample was 19.85 (SD = 4.27). METHOD Students completed four assessments: Analogies Test to measure cognitive ability; Big-Five personality inventory (Saucier, 1994) to measure the personality of themselves and the same Big-Five personality inventory to assess the personality of their mathematics tutor; Student Evaluations of Educational Quality (Marsh, 1982) to assess aspects of student engagement in the mathematics class; and Motivation and Engagement (Martin, 2007) to assess student motivation and engagement in university in general. Their demographics and overall mathematics course grades were also collected. RESULTS Hierarchical regressions revealed that both student personality and student ratings of their mathematics tutor’s personality predicted different domains of student engagement (beyond demographics and intelligence). Here, different domains of personality were important in predicting different aspects of student engagement. However, student ratings of their tutor’s personality predicted more domains and a higher percentage of variance. Furthermore, course grade and motivation and engagement in university in general were not predicted by teacher personality but only student personality. CONCLUSIONS The current study highlights the importance of understanding both student and teacher personality as it can determine students’ level of engagement in the classroom. REFERENCES Marsh, H. W. (1982). SEEQ: A reliable, valid, and useful instrument for collecting students ‘evaluations of university teaching. British Journal of Educational Psychology, 52(1), 77-95. Martin, A. J. (2007). The Motivation and Engagement Scale. Sydney: Lifelong Achievement Group. Saucier, G. (1994). Mini-markers: A brief version of Goldberg's unipolar Big-Five markers. Journal of Personality Assessment, 63(3), 506-516

What is students' ideal university instructor personality? An investigation of absolute and relative personality preferences

Despite intuitions that the ideal teacher has a particular set of non-cognitive characteristics, there is little research investigating such issues. The current two studies investigate students' descriptions of “ideal” in- structor personality using the Five-Factor Model of personality. Both absolute personality preferences (certain traits are universally desired) and relative personality preferences (certain traits are desired relative to students' own level of the trait) are examined among 137 first year mathematics students (Study 1) and 378 first year psychology students (Study 2). Students provided Big Five personality ratings for themselves, their actual instructor, and their ideal instructor. Supporting the absolute preference hypothesis, students rated their ideal instructor as having significantly higher levels than both themselves and the general population on all five personality domains (except for openness in Study 1), with particularly large effect sizes for emotional stability and conscientiousness. Supporting the relative preference hypothesis, students also rated their ideal instructor as having a similar Big Five profile to themselves. Moreover, if their actual instructor's personality was similar to their ideal instructor's personality, students showed greater educational satisfaction (but not higher performance self-efficacy nor academic achievement). The extent to which institutions should consider student preferences is discussed

Emotional Intelligence as Assessed by Situational Judgment and Emotion Recognition Tests: Building the Nomological Net

Recent research on emotion recognition ability (ERA) suggests that the capacity to process emotional information may differ for disparate emotions. However, little research has examined whether this findings holds for emotional understanding and emotion management, as well as emotion recognition. Moreover, little research has examined whether the abilities to recognize emotions, understand emotions, and manage emotions form a distinct emotional intelligence (EI) construct that is independent from traditional cognitive ability factors. The current study addressed these issues. Participants (N=118) completed two ERA measures, two situational judgment tests assessing emotional understanding and emotion management, and three cognitive ability tests. Exploratory and confirmatory factor analyses of both the understanding and management item parcels showed that a three-factor model relating to fear, sadness, and anger content was a better fit than a one-factor model, supporting an emotion-specific view of EI. In addition, an EI factor composed of emotion recognition, emotional understanding, and emotion management was distinct from a cognitive ability factor composed of a matrices task, general knowledge test, and reading comprehension task. Results are discussed in terms of their potential implications for theory and practice, as well as the integration of EI research with known models of cognitive ability

Facets of conscientiousness and their differential relationships with cognitive ability factors

This study examined relationships between conscientiousness facets and both broad factors of cognitive ability and collegiate GPA. Students responded to 117 Conscientiousness items and 15 cognitive tests demarcating fluid intelligence, crystallized intelligence, quantitative reasoning, visual processing, and broad retrieval ability. Confirmatory factor analysis replicated the eight-factor model found in MacCann, Duckworth, and Roberts (2009). Conscientiousness facet correlations with Cognitive Ability and GPA revealed that Cautiousness exhibited the highest correlation with Cognitive Ability, while Industriousness showed the strongest relationship with GPA. Procrastination Refrainment was the only facet negatively related to Cognitive Ability. Implications of these results are discussed in light of previ- ous research and the potentially moderating effect of high- versus low-stakes testing on the relationship between conscientiousness and cognitive ability

Differences between multimedia and text-based assessments of emotion management: An exploration with the multimedia emotion management assessment (MEMA)

People process emotional information using visual, vocal, and verbal cues. However, emotion management is typically assessed with text based rather than multimedia stimuli. This study (N=427) presents the new multimedia emotion management assessment (MEMA) and compares it to the text-based assessment of emotion management used in the MSCEIT. The text-based and multimedia assessment showed similar levels of cognitive saturation and similar prediction of relevant criteria. Results demonstrate that the MEMA scores have equivalent evidence of validity to the text-based MSCEIT test scores, demonstrating that multimedia assessment of emotion management is viable. Furthermore, our results inform the debate as to whether cognitive saturation in emotional intelligence (EI) measures represents noise or substance. We find that cognitive ability associations with EI represent substantive variance rather than construct-irrelevant shared variance due to reading comprehension ability required for text-based items

Assessing Emotional Intelligence Abilities, Acquiescent and Extreme Responding in Situational Judgment Tests Using Principal Component Metrics.

Principal Component Metrics is a novel theoretically-based and data-driven methodology that enables the evaluation of the internal structure at item level of maximum emotional intelligence tests. This method disentangles interindividual differences in emotional ability from acquiescent and extreme responding. Principal Component Metrics are applied to existing (Mayer-Salovey-Caruso Emotional Intelligence Test) and assembled (specifically, the Situational Test of Emotion Understanding, the Situational Test of Emotion Management, and the Geneva Emotion Recognition Test) emotional intelligence test batteries in an analysis of three samples (total N = 2,303 participants). In undertaking these analyses important aspects of the nomological network of emotional intelligence, acquiescent, and extreme responding are investigated. The current study adds a central piece of empirical validity evidence to the emotional intelligence domain. In the three different samples, theoretically predicted internal structures at item level were found using raw item scores. The validity of the indicators for emotional intelligence, acquiescent, and extreme responding was confirmed by their relationships across emotional intelligence tests and by their nomological networks. The current findings contribute to evaluating the efficacy of the emotional intelligence construct as well as the validity evidence surrounding the instruments that are currently designed for its assessment, in the process opening new perspectives for analyzing existing and constructing new emotional intelligence tests