Method effects associated with negatively and positively worded items on the 12-item General Health Questionnaire (GHQ-12) : results from a cross-sectional survey with a representative sample of Catalonian workers

This study focused on the examination of the latent structure underlying the responses to the GHQ-12, considering the role of method effects associated with both, PW and NW items, and using two alternative parameterisations of the CFA measurement models. What should first be noted is that the studies that have included method effects in the measurement model of the GHQ-12 have been more the exception than the rule in previous research into the factor structure of this questionnaire. According to the results of the present study, we conclude that the GHQ-12 factor structure is best characterised by introducing latent method factors that capture both the method effects associated with NW and PW items (model 7). These results support the conclusion from previous research that the good fit obtained by multidimensional models (mainly the two-factor model and the three-factor Graetz's model) could simply be explained by the artificial grouping of PW and NW items. However, the interpretation of the latent (method) factors as purely integrating method bias due to wording is not straightforward. It is obvious that NW and PW items share the wording. It is also clear that this three bifactor model (one trait and two method factors) fitted the data best. And finally, there is a lot of empirical evidence on these wording effects. However, it is also relevant to discuss the large loadings of many items on the method factors, being these loadings sometimes larger than their loadings in the trait factor. The general factor explains a 52% of the shared variance, but there are some items that deserve careful attention. For example, items 3 ('playing useful part in things') and 4 ('capable of making decisions') had very low loadings on the trait factor. If we understand PW method factor as the only method bias, then it follows that these two items are purely method effects, but surely they must share some trait variance. In the same vein, items 10 ('losing confidence in yourself') and 11 ('thinking of yourself as a worthless person') load very high in the NW method factor and, as a reviewer pointed out, a likely (post-hoc) explanation is that wording bias are still confounded with a confidence/self-image factor. Therefore, the interpretation of these effects as purely method and, accordingly, the interpretation of an overall score for the scale difficult may be compromised. The second aim of this study was to examine the relationship between the method factors associated with both NW and PW items and three demographic variables, namely sex, age and educational level of the respondents. Regarding the sex, we found a statistically significant, but weak, relationship between PW and sex, so that men were more likely than women to endorse PW items. These results are in line with previous works that, in the context of RSES, have found sex differences in wording effects. 56 57 As for the explanatory role of age on method effects, we found that the relationship between age and the NW effect was not statistically significant, which supports previous research using other questionnaires (eg, self-esteem scales, 50 Hospital Anxiety & Depression Scale 64). Moreover, our results give support to previous studies which had stated that, in older adults, the strongest method effects would be associated with PW items, rather than NW items. 55 58 As to the educational level, we found that there was not a significant correlation of this variable on the two method factors. This result supports and extends the evidence obtained in Tomás et al 50 who found that the educational level of the respondents had no effect on the negative method factor using self-esteem questionnaires. This results contradicts previous research on the relationship of the NW factor and the educational level/verbal ability with different questionnaires and samples. 41 64-69 Overall, the significant effects of sex and age on trait and method factors point out that women have a worse well-being, but this effect is partly modified by a method effect on the PW items, whereas the results for age suggest that older respondents have worse well-being and this effect is magnified by a method effect on the PW factor. The results on the individual differences related to the demographic variables considered in this study cannot only help to understand the presence of wording method effects but also to identify respondents who are prone to answering PW and NW items differently. In this sense, the relationship that appears as more evident is for the age and sex variables. Another practical consequence of our study concerns the relationship between the intended measure of the GHQ-12 (ie, the GPH factor) and other constructs of interest. Several studies have shown that method effects can inflate, deflate or have no effect at all on estimates of the relationship between two constructs (see Podsakoff et al 70 for a further review of the effects that method biases have on individual measures and on the covariation between different constructs). Thus, it is desirable that both the constructs of interest as well as the effects of method factors, like PW and NW, are considered in SEM models as a means of controlling these systematic sources of bias, and thus avoiding the drawing of inaccurate conclusions about the relationship between the substantive factors. Previous research on the GHQ-12 31 36 has outlined the asymmetry in the participants' responses as a function of the wording of the items, as well as the different responses scales for the PW and NW items. This asymmetry in the participants' responses as a function of the wording of the items is consistent with results from previous research into wording effects for contrastive survey questions. 71 The extent to which the presence of method effects is linked to the asymmetric pattern of responses and/or to the different response scales for the PW and NW items in the GHQ-12 should be examined in future research. Comparing the current work with previous studies into the factorial structure of the GHQ-12, to our knowledge, this is the first study that tests a comprehensive set of models including method effects associated with both PW and NW items and also explores some demographic correlates of these method effects. Another strength of this work was the fact that it used a large representative sample of workers, but the results might not be generalised to other specific populations, for example, adolescents and elderly retired people. Twitter @jmlosilla, @VivesJ_Research Contributors All authors meet the criteria recommended by the International Committee of Medical Journal Editors (ICMJE). All authors made substantial contributions to conception and design, acquisition of data or analysis and interpretation of data. MFR and JGM: drafted the article. JV and JML: critically revised the draft for important intellectual content. JMT: worked in the statistical analysis and interpretation of data. All authors agreed on the final version. Funding This work was supported by the Grant PGC2018-100675-B-I00, Spanish Ministry of Science, Innovation and Universities (Spain). The funders had no role in the study design, data collection and analysis, decision to publish or preparation of the manuscript. Disclaimer All authors have agreed to authorship in the indicated order. All authors declare that this paper is an original unpublished work and it is not being submitted elsewhere. All authors do not have any financial interests that might be interpreted as influencing the research, and APA ethical standard were followed in the conduct of the study. Competing interests None declared. Patient consent for publication Not required. Ethics approval The research was not submitted to approval by an institutional review board since this is not a requirement at our universities for this type of study. Ethics approval was not sought for this study since this was a secondary analysis of anonymised data. Provenance and peer review Not commissioned; externally peer reviewed. Data availability statement Data are available upon reasonable request.Objective Recent studies into the factorial structure of the 12-item version of the General Health Questionnaire (GHQ-12) have shown that it was best represented by a single substantive factor when method effects associated with negatively worded (NW) items are considered. The purpose of the present study was to examine the presence of method effects, and their relationships with demographic covariates, associated with positively worded (PW) and/or NW items. Design A cross-sectional, observational study to compare a comprehensive set of confirmatory factor models, including method effects associated with PW and/or NW items with GHQ-12 responses. Setting Representative sample of all employees living in Catalonia (Spain). Participants 3050 participants (44.6% women) who responded the Second Catalonian Survey of Working Conditions. Results A confirmatory factor analysis showed that the best fitting model was a unidimensional model with two additional uncorrelated method factors associated with PW and NW items. Furthermore, structural equation modelling (SEM) revealed that method effects were differentially related to both the sex and age of the respondents. Conclusion Individual differences related to sex and age can help to identify respondents who are prone to answering PW and NW items differently. Consequently, it is desirable that both the constructs of interest as well as the effects of method factors are considered in SEM models as a means of avoiding the drawing of inaccurate conclusions about the relationships between the substantive factors

Convergent and discriminant validity of the structured diagnostic interview for children and adolescents (DICA-R)

This paper examines the convergent validity of the Diagnostic Interview for Children and Adolescents - Revised (DICA-R), using the Child Behavior Checklist of T. M. Achenbach as a criterion. The ability of the instrument to detect subjects with psychopathology in the general population is also studied. The study was made through 196 interviews with psychiatric patients and 130 interviews with subjects from schools and Paediatric Services. The results obtained with the DICA-R are comparable with those obtained from other instruments that are widely accepted and considered as valid. The interview is a valid instrument for use in epidemiological research, as it can distinguish correctly between controls and casesEl artículo examina la validez convergente de la Entrevista Diagnóstica Estructurada para Niños y Adolescentes (DICA-R), utilizando como criterio la CBCL de T. M. Achenbach. Asimismo se describe la capacidad del instrumento para detectar sujetos con psicopatología en la población general. El estudio se llevó a cabo con un total de 196 entrevistas realizadas con pacientes psiquiátricos externos y 130 entrevistas realizadas con sujetos reclutados en escuelas y Servicios de Pediatría. Los resultados obtenidos con lo DICA-R convergen con los que se obtienen a partir de otros instrumentos ampliamente utilizados y considerados como válidos. La entrevista es asimismo válida para el estudio epidemiológico ya que diferencia adecuadamente entre grupos controles y psiquiátrico

Fiabilidad test-retest de la adaptación española de la Diagnostic Interview for Children and Adolescents (DICA-R)

El artículo examina la fiabilidad test-retest de la adaptación española de la Diagnostic Interview for Children and adolescents-Revised (DICA-R) en una muestra de 110 niños y adolescentes pacientes psiquiátricos externos. Niños y padres fueron entrevistados con la DICA-R en dos ocasiones separadas por un intervalo medio de 11 días. Los resultados del estudio indican que hay estabilidad en la evaluación de los trastornos con la DICA-R, como lo demuestra la ausencia de diferencias significativas entre el test y el retest y el rango de la mayoría de los valores kappa que oscilaron entre buenos y excelentes. Los más estables informando sobre los trastornos fueron los padres y los menos fiables fueron los adolescentes. Los trastornos mostraron cierta atenuación en el retest, aunque las diferencias no fueron significativas. La fiabilidad de los respondientes individuales dependió del tipo de trastorno y de la edad. La entrevista fue más fiable en las categorías más generales que en los trastornos específicosThe paper examines the test-retest reliability of the Diagnostic Interview for Children and Adolescents-Revised (DICA-R). 110 children and adolescent outpatients and their parents were interviewed using the DICA-R on two occasions, separated by a mean interval of 11 days. The results of our study indicate that there is stability in the assessment of the disorders with DICA-R, as shown by the absence of significant differences between test and retest and the majority of kappa values which ranged from good to excellent. The parents were the most stable in reporting disorders and adolescents were the least. Test-retest reliability of DICA-R in the sample was very good. The disorders appeared attenuated in the retest. The reliability of respondent assessment depended on the type of disorder and on age. The interview was more reliable in higher rank of major categories than in specific disorder

La Diagnostic Interview for Children and Adolescents-Revisada (DICAR-R) : acuerdo diagnóstico entre niños/adolescentes y sus padres

El artículo examina el acuerdo diagnóstico entre 145 niños pacientes psicológicos externos y sus padres al responder al Diagnostic Interview for Children and Adolescents-Revised (DICA-R) adaptada al castellano. Los resultados indican que existe acuerdo, en general bajo, entre ambas fuentes respecto a la mayoría de las categorías diagnósticas. El grado de acuerdo aumentó con la edad. Niños y adolescentes informaron de más síntomas que sus padres. Las áreas donde se encontraron mayores diferencias entre las fuentes fueron los trastornos por conductas perturbadoras y trastornos por ansiedad, y también en depresión mayor para los adolescentes. Para la evaluación de la psicopatología de niños y adolescentes es necesaria la información de ambosThe paper examines the diagnostic agreement in 145 outpatient children and their parents answering the Diagnostic Interview for Children and Adolescents-Revised (DICA-R) adapted to the Spanish population. The results indicate that there exist diagnostic concordance between the two sources, but the level of agreement wast mostly low. The level of agreement improved with the age. Children and adolescent reported more symptoms than their parents. Significant differences were found between the child/adolescent and parent information in the areas of disruptive disorders and anxiety disorders, and in major depression for adolescents. Multiple informants are needed in the assessment of child and adolescent psychopatholog

Fiabilidad entre entrevistadores de la Entrevista Diagnóstica Estructurada para Niños y Adolescentes (DICA-R)

El artículo examina la fiabilidad entre entrevistadores de la Diagnostic Interview for Children and Adolescents-Revised (DICA-R). El estudio se llevó a cabo con un total de 45 entrevistas, en las cuales participaron un examinador y un observador. Se describe de manera exhaustiva el prodecimiento de entrenamiento. Los resultados muestran que la DICA-R es un instrumento objetivo que puede ser utilizado por personas legas si se les proporciona un entrenamiento adecuado. Se pone énfasis en la necesidad de que los diseños para estudiar la fiabilidad entre examinadores sean estrictosInterrater reliability of the Diagnostic Interview for Children and Adolescents (DICA-R). The paper examines the interrater reliability of the Diagnostic Interview for Children and Adolescents-Revised (DICA-R). The study was held over a total of 45 interviews in which an interviewer and an observer participated. The trainning program is exhaustively described. The results show that DICA-R is an objective instrument that can be used by lay persons if a proper training takes place. Strictness in interrater reliability designs is encourage

Fiabilidad test-retest de la adaptación española de la Diagnostic Interview for Children and Adolescents (DICA-R)

El artículo examina la fiabilidad test-retest de la adaptación española de la Diagnostic Interview for Children and adolescents-Revised (DICA-R) en una muestra de 110 niños y adolescentes pacientes psiquiátricos externos. Niños y padres fueron entrevistados con la DICA-R en dos ocasiones separadas por un intervalo medio de 11 días. Los resultados del estudio indican que hay estabilidad en la evaluación de los trastornos con la DICA-R, como lo demuestra la ausencia de diferencias significativas entre el test y el retest y el rango de la mayoría de los valores kappa que oscilaron entre buenos y excelentes. Los más estables informando sobre los trastornos fueron los padres y los menos fiables fueron los adolescentes. Los trastornos mostraron cierta atenuación en el retest, aunque las diferencias no fueron significativas. La fiabilidad de los respondientes individuales dependió del tipo de trastorno y de la edad. La entrevista fue más fiable en las categorías más generales que en los trastornos específicosThe paper examines the test-retest reliability of the Diagnostic Interview for Children and Adolescents-Revised (DICA-R). 110 children and adolescent outpatients and their parents were interviewed using the DICA-R on two occasions, separated by a mean interval of 11 days. The results of our study indicate that there is stability in the assessment of the disorders with DICA-R, as shown by the absence of significant differences between test and retest and the majority of kappa values which ranged from good to excellent. The parents were the most stable in reporting disorders and adolescents were the least. Test-retest reliability of DICA-R in the sample was very good. The disorders appeared attenuated in the retest. The reliability of respondent assessment depended on the type of disorder and on age. The interview was more reliable in higher rank of major categories than in specific disorder

Method effects associated with negatively and positively worded items on the 12-item General Health Questionnaire (GHQ-12) : results from a cross-sectional survey with a representative sample of Catalonian workers

This study focused on the examination of the latent structure underlying the responses to the GHQ-12, considering the role of method effects associated with both, PW and NW items, and using two alternative parameterisations of the CFA measurement models. What should first be noted is that the studies that have included method effects in the measurement model of the GHQ-12 have been more the exception than the rule in previous research into the factor structure of this questionnaire. According to the results of the present study, we conclude that the GHQ-12 factor structure is best characterised by introducing latent method factors that capture both the method effects associated with NW and PW items (model 7). These results support the conclusion from previous research that the good fit obtained by multidimensional models (mainly the two-factor model and the three-factor Graetz's model) could simply be explained by the artificial grouping of PW and NW items. However, the interpretation of the latent (method) factors as purely integrating method bias due to wording is not straightforward. It is obvious that NW and PW items share the wording. It is also clear that this three bifactor model (one trait and two method factors) fitted the data best. And finally, there is a lot of empirical evidence on these wording effects. However, it is also relevant to discuss the large loadings of many items on the method factors, being these loadings sometimes larger than their loadings in the trait factor. The general factor explains a 52% of the shared variance, but there are some items that deserve careful attention. For example, items 3 ('playing useful part in things') and 4 ('capable of making decisions') had very low loadings on the trait factor. If we understand PW method factor as the only method bias, then it follows that these two items are purely method effects, but surely they must share some trait variance. In the same vein, items 10 ('losing confidence in yourself') and 11 ('thinking of yourself as a worthless person') load very high in the NW method factor and, as a reviewer pointed out, a likely (post-hoc) explanation is that wording bias are still confounded with a confidence/self-image factor. Therefore, the interpretation of these effects as purely method and, accordingly, the interpretation of an overall score for the scale difficult may be compromised. The second aim of this study was to examine the relationship between the method factors associated with both NW and PW items and three demographic variables, namely sex, age and educational level of the respondents. Regarding the sex, we found a statistically significant, but weak, relationship between PW and sex, so that men were more likely than women to endorse PW items. These results are in line with previous works that, in the context of RSES, have found sex differences in wording effects. 56 57 As for the explanatory role of age on method effects, we found that the relationship between age and the NW effect was not statistically significant, which supports previous research using other questionnaires (eg, self-esteem scales, 50 Hospital Anxiety & Depression Scale 64). Moreover, our results give support to previous studies which had stated that, in older adults, the strongest method effects would be associated with PW items, rather than NW items. 55 58 As to the educational level, we found that there was not a significant correlation of this variable on the two method factors. This result supports and extends the evidence obtained in Tomás et al 50 who found that the educational level of the respondents had no effect on the negative method factor using self-esteem questionnaires. This results contradicts previous research on the relationship of the NW factor and the educational level/verbal ability with different questionnaires and samples. 41 64-69 Overall, the significant effects of sex and age on trait and method factors point out that women have a worse well-being, but this effect is partly modified by a method effect on the PW items, whereas the results for age suggest that older respondents have worse well-being and this effect is magnified by a method effect on the PW factor. The results on the individual differences related to the demographic variables considered in this study cannot only help to understand the presence of wording method effects but also to identify respondents who are prone to answering PW and NW items differently. In this sense, the relationship that appears as more evident is for the age and sex variables. Another practical consequence of our study concerns the relationship between the intended measure of the GHQ-12 (ie, the GPH factor) and other constructs of interest. Several studies have shown that method effects can inflate, deflate or have no effect at all on estimates of the relationship between two constructs (see Podsakoff et al 70 for a further review of the effects that method biases have on individual measures and on the covariation between different constructs). Thus, it is desirable that both the constructs of interest as well as the effects of method factors, like PW and NW, are considered in SEM models as a means of controlling these systematic sources of bias, and thus avoiding the drawing of inaccurate conclusions about the relationship between the substantive factors. Previous research on the GHQ-12 31 36 has outlined the asymmetry in the participants' responses as a function of the wording of the items, as well as the different responses scales for the PW and NW items. This asymmetry in the participants' responses as a function of the wording of the items is consistent with results from previous research into wording effects for contrastive survey questions. 71 The extent to which the presence of method effects is linked to the asymmetric pattern of responses and/or to the different response scales for the PW and NW items in the GHQ-12 should be examined in future research. Comparing the current work with previous studies into the factorial structure of the GHQ-12, to our knowledge, this is the first study that tests a comprehensive set of models including method effects associated with both PW and NW items and also explores some demographic correlates of these method effects. Another strength of this work was the fact that it used a large representative sample of workers, but the results might not be generalised to other specific populations, for example, adolescents and elderly retired people. Twitter @jmlosilla, @VivesJ_Research Contributors All authors meet the criteria recommended by the International Committee of Medical Journal Editors (ICMJE). All authors made substantial contributions to conception and design, acquisition of data or analysis and interpretation of data. MFR and JGM: drafted the article. JV and JML: critically revised the draft for important intellectual content. JMT: worked in the statistical analysis and interpretation of data. All authors agreed on the final version. Funding This work was supported by the Grant PGC2018-100675-B-I00, Spanish Ministry of Science, Innovation and Universities (Spain). The funders had no role in the study design, data collection and analysis, decision to publish or preparation of the manuscript. Disclaimer All authors have agreed to authorship in the indicated order. All authors declare that this paper is an original unpublished work and it is not being submitted elsewhere. All authors do not have any financial interests that might be interpreted as influencing the research, and APA ethical standard were followed in the conduct of the study. Competing interests None declared. Patient consent for publication Not required. Ethics approval The research was not submitted to approval by an institutional review board since this is not a requirement at our universities for this type of study. Ethics approval was not sought for this study since this was a secondary analysis of anonymised data. Provenance and peer review Not commissioned; externally peer reviewed. Data availability statement Data are available upon reasonable request.Objective Recent studies into the factorial structure of the 12-item version of the General Health Questionnaire (GHQ-12) have shown that it was best represented by a single substantive factor when method effects associated with negatively worded (NW) items are considered. The purpose of the present study was to examine the presence of method effects, and their relationships with demographic covariates, associated with positively worded (PW) and/or NW items. Design A cross-sectional, observational study to compare a comprehensive set of confirmatory factor models, including method effects associated with PW and/or NW items with GHQ-12 responses. Setting Representative sample of all employees living in Catalonia (Spain). Participants 3050 participants (44.6% women) who responded the Second Catalonian Survey of Working Conditions. Results A confirmatory factor analysis showed that the best fitting model was a unidimensional model with two additional uncorrelated method factors associated with PW and NW items. Furthermore, structural equation modelling (SEM) revealed that method effects were differentially related to both the sex and age of the respondents. Conclusion Individual differences related to sex and age can help to identify respondents who are prone to answering PW and NW items differently. Consequently, it is desirable that both the constructs of interest as well as the effects of method factors are considered in SEM models as a means of avoiding the drawing of inaccurate conclusions about the relationships between the substantive factors

La Diagnostic Interview for Children and Adolescents-Revisada (DICAR-R) : acuerdo diagnóstico entre niños/adolescentes y sus padres

El artículo examina el acuerdo diagnóstico entre 145 niños pacientes psicológicos externos y sus padres al responder al Diagnostic Interview for Children and Adolescents-Revised (DICA-R) adaptada al castellano. Los resultados indican que existe acuerdo, en general bajo, entre ambas fuentes respecto a la mayoría de las categorías diagnósticas. El grado de acuerdo aumentó con la edad. Niños y adolescentes informaron de más síntomas que sus padres. Las áreas donde se encontraron mayores diferencias entre las fuentes fueron los trastornos por conductas perturbadoras y trastornos por ansiedad, y también en depresión mayor para los adolescentes. Para la evaluación de la psicopatología de niños y adolescentes es necesaria la información de ambosThe paper examines the diagnostic agreement in 145 outpatient children and their parents answering the Diagnostic Interview for Children and Adolescents-Revised (DICA-R) adapted to the Spanish population. The results indicate that there exist diagnostic concordance between the two sources, but the level of agreement wast mostly low. The level of agreement improved with the age. Children and adolescent reported more symptoms than their parents. Significant differences were found between the child/adolescent and parent information in the areas of disruptive disorders and anxiety disorders, and in major depression for adolescents. Multiple informants are needed in the assessment of child and adolescent psychopatholog

Fiabilidad entre entrevistadores de la Entrevista Diagnóstica Estructurada para Niños y Adolescentes (DICA-R)

El artículo examina la fiabilidad entre entrevistadores de la Diagnostic Interview for Children and Adolescents-Revised (DICA-R). El estudio se llevó a cabo con un total de 45 entrevistas, en las cuales participaron un examinador y un observador. Se describe de manera exhaustiva el prodecimiento de entrenamiento. Los resultados muestran que la DICA-R es un instrumento objetivo que puede ser utilizado por personas legas si se les proporciona un entrenamiento adecuado. Se pone énfasis en la necesidad de que los diseños para estudiar la fiabilidad entre examinadores sean estrictosInterrater reliability of the Diagnostic Interview for Children and Adolescents (DICA-R). The paper examines the interrater reliability of the Diagnostic Interview for Children and Adolescents-Revised (DICA-R). The study was held over a total of 45 interviews in which an interviewer and an observer participated. The trainning program is exhaustively described. The results show that DICA-R is an objective instrument that can be used by lay persons if a proper training takes place. Strictness in interrater reliability designs is encourage