Online self-assessments to measure study-related competencies

An der Universität Frankfurt entwickelte Online-Self-Assessment-Verfahren für die Studiengänge Psychologie und Informatik sollen Studieninteressierten noch vor Studienbeginn auf der Basis von Selbsterkundungsmaßnahmen und Tests eine Rückmeldung über ihre eigenen Fähigkeiten, Motive, personalen Kompetenzen und Interessen mit Blick auf den jeweiligen Studiengang geben. Sowohl die Befunde zur psychometrischen Güte der Verfahren als auch jene zur prognostischen Validität lassen ihren Einsatz zur Feststellung studienrelevanter Kompetenzen als geeignet erscheinen. Da die erfassten Kompetenzen und Merkmale substanzielle Beziehun-gen zu Studienleistungen aufweisen, könnten die Informationen über individuelle Stärken zur Wahl eines geeigneten Studienganges genutzt werden; Schwächen hingegen könnten frühzeitig Hinweise für geeignete Fördermaßnahmen liefern.Online self-assessments are offered as a means to inform prospective students about their personal appropriateness in the degree programs of psychology and computer science at the University of Frankfurt. Based on online tests a feedback regarding their skills, motives, personal competencies and orientations with respect to the particular study course is provided. Psychometric properties and predictive validity suggest the suitability as an instrument to assess study-related competencies. Since the measures applied as part of the study show substantial relationships to course achievement, one could utilize the Self-Assessments information for decisions concerning the selection of an appropriate course of studies. Furthermore, noted weaknesses could serve as early hints for appropriate affirmative actions

Speed Effect Analysis Using the CFA Framework

The paper outlines a method for investigating the speed effect due to a time limit in testing. It is assumed that the time limit enables latent processing speed to influence responses by causing omissions in the case of insufficient speed. Because of processing speed as additional latent source, the customary confirmatory factor model is enlarged by a second latent variable representing latent processing speed. For distinguishing this effect from other method effects, the factor loadings are fixed according to the cumulative normal distribution. With the second latent variable added, confirmatory factor analysis of reasoning data (N=518) including omissions because of a time limit yielded good model fit and discriminated the speed effect from other possible effects due to the item difficulty, the homogeneity of an item subset and the item positions. Because of the crucial role of the cumulative normal distribution for fixing the factor loadings a check of the normality assumption is also reported

Speed Effect Analysis Using the CFA Framework

The paper outlines a method for investigating the speed effect due to a time limit in testing. It is assumed that the time limit enables latent processing speed to influence responses by causing omissions in the case of insufficient speed. Because of processing speed as additional latent source, the customary confirmatory factor model is enlarged by a second latent variable representing latent processing speed. For distinguishing this effect from other method effects, the factor loadings are fixed according to the cumulative normal distribution. With the second latent variable added, confirmatory factor analysis of reasoning data (N=518) including omissions because of a time limit yielded good model fit and discriminated the speed effect from other possible effects due to the item difficulty, the homogeneity of an item subset and the item positions. Because of the crucial role of the cumulative normal distribution for fixing the factor loadings a check of the normality assumption is also reported

Does the effect of a time limit for testing impair structural investigations by means of confirmatory factor models?

The article reports three simulation studies conducted to find out whether the effect of a time limit for testing impairs model fit in investigations of structural validity, whether the representation of the assumed source of the effect prevents impairment of model fit and whether it is possible to identify and discriminate this method effect from another method effect. Omissions due to the time limit for testing were not considered as missing data but as information on the participants’ processing speed. In simulated data the presence of a time-limit effect impaired comparative fit index and nonnormed fit index whereas normed chi-square, root mean square error of approximation, and standardized root mean square residual indicated good model fit. The explicit consideration of the effect due to the time limit by an additional component of the model improved model fit. Effect-specific assumptions included in the model of measurement enabled the discrimination of the effect due to the time limit from another possible method effect

Can variances of latent variables be scaled in such a way that they correspond to eigenvalues?

The paper reports an investigation of whether sums of squared factor loadings obtained in confirmatory factor analysis correspond to eigenvalues of exploratory factor analysis. The sum of squared factor loadings reflects the variance of the corresponding latent variable if the variance parameter of the confirmatory factor model is set equal to one. Hence, the computation of the sum implies a specific type of scaling of the variance. While the investigation of the theoretical foundations suggested the expected correspondence between sums of squared factor loadings and eigenvalues, the necessity of procedural specifications in the application, as for example the estimation method, revealed external influences on the outcome. A simulation study was conducted that demonstrated the possibility of exact correspondence if the same estimation method was applied. However, in the majority of realized specifications the estimates showed similar sizes but no correspondence