Construct validity of multi-source performance ratings: An examination of the relationship of self-, supervisor-, and peer-ratings with cognitive and personality measures

Although more and more organizations prefer using multi-source performance ratings or 3601 feedback over traditional performance appraisals, researchers have been rather skeptical regarding the reliability and validity of such ratings. The present study examined the validity of self-, supervisor-, and peer-ratings of 195 employees in a Dutch public organization, using scores on an In-Basket exercise, an intelligence test, and a personality questionnaire as external criterion measures. Interrater agreement ranged from .28 to .38. Variance in the ratings was explained by both method and content factors. Support for the external construct validity was rather weak. Supervisor-ratings were not found to be superior to self- and peer-ratings in predicting the scores on the external measures

A Triadic Approach to the Construct Validity of the Assessment Center: The Effect of Categorizing Dimensions into a Feeling, Thinking, and Power Taxonomy

This study examined the influence on construct validity of implementing the triad Feeling, Thinking, and Power as a taxonomy for behavioral dimensions in assessment center (AC) exercises. A sample of 1567 job applicants participated in an AC specifically developed according to this taxonomy. Each exercise tapped three dimensions, one dimension from each cluster of the taxonomy. Confirmatory Factor Analysis of the multitrait-multimethod matrix showed evidence for construct validity. Thus, the ratings matched the a priori triadic grouping to a good extent. Practical implications are discussed

Impact of common rater variance on construct validity of assessment center dimension judgements

In an assessment center (AC), assessors generally rate an applicant's performance on multiple dimensions in just 1 exercise. This rating procedure introduces common rater variance within exercises but not between exercises. This article hypothesizes that this phenomenon is partly responsible for the consistently reported result that the AC lacks construct validity. Therefore, in this article, the rater effect is standardized on discriminant and convergent validity via a multitrait-multimethod design in which each matrix cell is based on ratings of different assessors. Two independent studies (N = 200, N = 52) showed that, within exercises, correlations decrease when common rater variance is excluded both across exercises (by having assessors rate only 1 exercise) and within exercises (by having assessors rate only 1 dimension per exercise). Implications are discussed in the context of the recent discussion around the appropriateness of the within-exercise versus the within-dimension evaluation method

Latent-trait latent-class analysis of selfdisclosure in the work environment

Based on the literature about self-disclosure, it was hypothesized that different groups of subjects differ in their pattern of self-disclosure with respect to different areas of social interaction. An extended latent-trait latent-class model was proposed to describe these general patterns of self-disclosure. The model was used to analyze the data of 1,113 subjects, tested on extraversion and with respect to their degree of self-disclosure toward different categories of people in the work environment. A model with one latent trait and a latent class variable with three categories was identified. Subjects belonging to the different latent classes differ in their general tendency to self-disclose, in their choice to whom they will show self-disclosure and in the degree to which they are selective in their self-disclosure. The collateral variable extraversion was associated with both latent variables. The association of extraversion with selectivity in self-disclosure was not significant. Copyright © 2005, Lawrence Erlbaum Associates, Inc