Percentile ranks and benchmark estimates of change for the Health Education Impact Questionnaire: normative data from an Australian sample

Objective: Participant self-report data play an essential role in the evaluation of health education activities, programmes and policies. When questionnaire items do not have a clear mapping to a performance-based continuum, percentile norms are useful for communicating individual test results to users. Similarly, when assessing programme impact, the comparison of effect sizes for group differences or baseline to follow-up change with effect sizes observed in relevant normative data provides more directly useful information compared with statistical tests of mean differences and the evaluation of effect sizes for substantive significance using universal rule-of-thumb such as those for Cohen’s ‘d’. This article aims to assist managers, programme staff and clinicians of healthcare organisations who use the Health Education Impact Questionnaire interpret their results using percentile norms for individual baseline and follow-up scores together with group effect sizes for change across the duration of typical chronic disease self-management and support programme. Methods: Percentile norms for individual Health Education Impact Questionnaire scale scores and effect sizes for group change were calculated using freely available software for each of the eight Health Education Impact Questionnaire scales. Data used were archived responses of 2157 participants of chronic disease self-management programmes conducted by a wide range of organisations in Australia between July 2007 and March 2013. Results: Tables of percentile norms and three possible effect size benchmarks for baseline to follow-up change are provided together with two worked examples to assist interpretation. Conclusion: While the norms and benchmarks presented will be particularly relevant for Australian organisations and others using the English-language version of the Health Education Impact Questionnaire, they will also be useful for translated versions as a guide to the sensitivity of the scales and the extent of the changes that might be anticipated from attendance at a typical chronic disease self-management or health education programme

The grounded psychometric development and initial validation of the Health Literacy Questionnaire (HLQ)

BACKGROUND: Health literacy has become an increasingly important concept in public health. We sought to develop a comprehensive measure of health literacy capable of diagnosing health literacy needs across individuals and organisations by utilizing perspectives from the general population, patients, practitioners and policymakers. METHODS: Using a validity-driven approach we undertook grounded consultations (workshops and interviews) to identify broad conceptually distinct domains. Questionnaire items were developed directly from the consultation data following a strict process aiming to capture the full range of experiences of people currently engaged in healthcare through to people in the general population. Psychometric analyses included confirmatory factor analysis (CFA) and item response theory. Cognitive interviews were used to ensure questions were understood as intended. Items were initially tested in a calibration sample from community health, home care and hospital settings (N=634) and then in a replication sample (N=405) comprising recent emergency department attendees. RESULTS: Initially 91 items were generated across 6 scales with agree/disagree response options and 5 scales with difficulty in undertaking tasks response options. Cognitive testing revealed that most items were well understood and only some minor re-wording was required. Psychometric testing of the calibration sample identified 34 poorly performing or conceptually redundant items and they were removed resulting in 10 scales. These were then tested in a replication sample and refined to yield 9 final scales comprising 44 items. A 9-factor CFA model was fitted to these items with no cross-loadings or correlated residuals allowed. Given the very restricted nature of the model, the fit was quite satisfactory: χ(2)(WLSMV)(866 d.f.) = 2927, p<0.000, CFI = 0.936, TLI = 0.930, RMSEA = 0.076, and WRMR = 1.698. Final scales included: Feeling understood and supported by healthcare providers; Having sufficient information to manage my health; Actively managing my health; Social support for health; Appraisal of health information; Ability to actively engage with healthcare providers; Navigating the healthcare system; Ability to find good health information; and Understand health information well enough to know what to do. CONCLUSIONS: The HLQ covers 9 conceptually distinct areas of health literacy to assess the needs and challenges of a wide range of people and organisations. Given the validity-driven approach, the HLQ is likely to be useful in surveys, intervention evaluation, and studies of the needs and capabilities of individuals

Measuring health literacy in community agencies: a Bayesian study of the factor structure and measurement invariance of the Health Literacy Questionnaire (HLQ)

Background The development of the Health Literacy Questionnaire (HLQ), reported in 2013, attracted widespread international interest. While the original study samples were drawn from clinical and home-based aged-care settings, the HLQ was designed for the full range of healthcare contexts including community-based health promotion and support services. We report a follow-up study of the psychometric properties of the HLQ with respondents from a diverse range of community-based organisations with the principal goal of contributing to the development of a soundly validated evidence base for its use in community health settings.Methods Data were provided by 813 clients of 8 community agencies in Victoria, Australia who were administered the HLQ during the needs assessment stage of the Ophelia project, a health literacy-based intervention. Most analyses were conducted using Bayesian structural equation modelling that enables rigorous analysis of data but with some relaxation of the restrictive requirements for zero cross-loadings and residual correlations of &lsquo;classical&rsquo; confirmatory factor analysis. Scale homogeneity was investigated with one-factor models that allowed for the presence of small item residual correlations while discriminant validity was studied using the inter-factor correlations and factor loadings from a full 9-factor model with similar allowance for small residual correlations and cross-loadings. Measurement invariance was investigated scale-by-scale using a model that required strict invariance of item factor loadings, thresholds, residual variances and co-variances.Results All HLQ scales were found to be homogenous with composite reliability ranging from 0.80 to 0.89. The factor structure of the HLQ was replicated and 6 of the 9 scales were found to exhibit clear-cut discriminant validity. With a small number of exceptions involving non-invariance of factor loadings, strict measurement invariance was established across the participating organisations and the gender, language background, age and educational level of respondents.Conclusions The HLQ is highly reliable, even with only 4 to 6 items per scale. It provides unbiased mean estimates of group differences across key demographic indicators. While measuring relatively narrow constructs, the 9 dimensions are clearly separate and therefore provide fine-grained data on the multidimensional area of health literacy. These analyses provide researchers, program managers and policymakers with a range of robust evidence by which they can make judgements about the appropriate use of the HLQ for their community-based setting

Absence of social desirability bias in the evaluation of chronic disease self-management interventions

BACKGROUND: Bias due to social desirability has long been of concern to evaluators relying on self-report data. It is conceivable that health program evaluation is particularly susceptible to social desirability bias as individuals may be inclined to present themselves or certain health behaviors in a more positive light and/or appease the course leader. Thus, the influence of social desirability bias on self-report outcomes was explored in the present study. METHODS: Data were collected from 331 participants of group-based chronic disease self-management interventions using the highly robust eight-scale Health Education Impact Questionnaire (heiQ) and the 13-item short form Marlowe-Crowne Social Desirability Scale (MC-C). The majority of self-management courses were run by community-based organizations across Australia between February 2005 and December 2006 where 6 to 12 individuals have the opportunity to develop considerable rapport with course leaders and each other over about six weeks. Pre-test data were collected on the first day of courses, while post-test and social desirability scores were assessed at the end of courses. A model of partial mediation within the framework of structural equation modeling was developed with social desirability as the mediating variable between pre-test and post-test. RESULTS: The \u27Defensiveness\u27 factor of the MC-C showed clear association with heiQ pre-test data, a prerequisite for investigating mediation; however, when investigating the eight full pre-test/post-test models \u27Defensiveness\u27 was only associated with one heiQ scale. This effect was small, explaining 8% of the variance in the model. No other meditational effects through social desirability were observed. CONCLUSIONS: The overall lack of association of social desirability with heiQ outcomes was surprising as it had been expected that it would explain at least some of the variance observed between pre-test and post-test. With the assumption that the MC-C captures the propensity for an individual to provide socially desirable answers, this study concludes that change scores in chronic disease self-management program evaluation are not biased by social desirability

Cultural adaptation and validation of the Health Literacy Questionnaire (HLQ): robust nine-dimension Danish language confirmatory factor model

Health literacy is an important construct in population health and healthcare requiring rigorous measurement. The Health Literacy Questionnaire (HLQ), with nine scales, measures a broad perception of health literacy. This study aimed to adapt the HLQ to the Danish setting, and to examine the factor structure, homogeneity, reliability and discriminant validity. The HLQ was adapted using forward-backward translation, consensus conference and cognitive interviews (n&nbsp;=&nbsp;15). Psychometric properties were examined based on data collected by face-to-face interview (n&nbsp;=&nbsp;481). Tests included difficulty level, composite scale reliability and confirmatory factor analysis (CFA). Cognitive testing revealed that only minor re-wording was required. The easiest scale to respond to positively was \u27Social support for health\u27, and the hardest were \u27Navigating the healthcare system\u27 and \u27Appraisal of health information\u27. CFA of the individual scales showed acceptably high loadings (range 0.49-0.93). CFA fit statistics after including correlated residuals were good for seven scales, acceptable for one. Composite reliability and Cronbach\u27s &alpha; were &gt;0.8 for all but one scale. A nine-factor CFA model was fitted to items with no cross-loadings or correlated residuals allowed. Given this restricted model, the fit was satisfactory. The HLQ appears robust for its intended application of assessing health literacy in a range of settings. Further work is required to demonstrate sensitivity to measure changes

Construction of the descriptive system for the assessment of quality of life AQoL-6D utility instrument

BackgroundMulti attribute utility (MAU) instruments are used to include the health related quality of life (HRQoL) in economic evaluations of health programs. Comparative studies suggest different MAU instruments measure related but different constructs. The objective of this paper is to describe the methods employed to achieve content validity in the descriptive system of the Assessment of Quality of Life (AQoL)-6D, MAU instrument.MethodsThe AQoL program introduced the use of psychometric methods in the construction of health related MAU instruments. To develop the AQoL-6D we selected 112 items from previous research, focus groups and expert judgment and administered them to 316 members of the public and 302 hospital patients. The search for content validity across a broad spectrum of health states required both formative and reflective modelling. We employed Exploratory Factor Analysis and Structural Equation Modelling (SEM) to meet these dual requirements.Results and DiscussionThe resulting instrument employs 20 items in a multi-tier descriptive system. Latent dimension variables achieve sensitive descriptions of 6 dimensions which, in turn, combine to form a single latent QoL variable. Diagnostic statistics from the SEM analysis are exceptionally good and confirm the hypothesised structure of the model.ConclusionsThe AQoL-6D descriptive system has good psychometric properties. They imply that the instrument has achieved construct validity and provides a sensitive description of HRQoL. This means that it may be used with confidence for measuring health related quality of life and that it is a suitable basis for modelling utilities for inclusion in the economic evaluation of health programs.<br /

The Health Literacy Questionnaire (HLQ) at the patient-clinician interface: a qualitative study of what patients and clinicians mean by their HLQ scores

BackgroundThe Health Literacy Questionnaire (HLQ) has nine scales that each measure an aspect of the multidimensional construct of health literacy. All scales have good psychometric properties. However, it is the interpretations of data within contexts that must be proven valid, not just the psychometric properties of a measurement instrument. The purpose of this study was to establish the extent of concordance and discordance between individual patient and clinician interpretations of HLQ data in the context of complex case management.MethodsSixteen patients with complex needs completed the HLQ and were interviewed to discuss the reasons for their answers. Also, the clinicians of each of these patients completed the HLQ about their patient, and were interviewed to discuss the reasons for their answers. Thematic analysis of HLQ scores and interview data determined the extent of concordance between patient and clinician HLQ responses, and the reasons for discordance.ResultsHighest concordance (80%) between patient and clinician item-response pairs was seen in Scale 1 and highest discordance (56%) was seen in Scale 6. Four themes were identified to explain discordance: 1) Technical or literal meaning of specific words; 2) Patients&rsquo; changing or evolving circumstances; 3) Different expectations and criteria for assigning HLQ scores; and 4) Different perspectives about a patient&rsquo;s reliance on healthcare providers.ConclusionThis study shows that the HLQ can act as an adjunct to clinical practice to help clinicians understand a patient&rsquo;s health literacy challenges and strengths early in a clinical encounter. Importantly, clinicians can use the HLQ to detect differences between their own perspectives about a patient&rsquo;s health literacy and the patient&rsquo;s perspective, and to initiate discussion to explore this. Provision of training to better detect these differences may assist clinicians to provide improved care.The outcomes of this study contribute to the growing body of international validation evidence about the use of the HLQ in different contexts. More specifically, this study has shown that the HLQ has measurement veracity at the patient and clinician level and may support clinicians to understand patients&rsquo; health literacy and enable a deeper engagement with healthcare services.<br /

Validation of the Dutch version of the health education impact questionnaire (HEIQ) and comparison of the Dutch translation with the English, German and French HEIQ

BACKGROUND: The Health Education Impact Questionnaire (heiQ) evaluates the effectiveness of health education and self-management programs provided to people dealing with a wide range of conditions. Aim of this study was to translate, culturally adapt and validate the Dutch translation of the heiQ and to compare the results with the English, German and French translations. METHODS: A systematic translation process was undertaken. Psychometric properties were studied among patients with arthritis, atopic dermatitis, food allergy and asthma (n&thinsp;=&thinsp;286). Factorial validity using confirmatory factor analysis, item difficulty (D), item remainder correlation and composite reliability were conducted. Stability was tested using the intra-class correlation coefficient (ICC). RESULTS: Items were well understood and only minor language adjustments were required. Confirmatory fit indices were &gt;0.95 and item difficulty was D&thinsp;&ge;&thinsp;0.65 for all items in scales showing acceptable fit indices, except for the reversed Emotional distress scale. Composite reliability ranged between 0.67 and 0.85. Test-retest reliability (n&thinsp;=&thinsp;93) ICC varied between 0.61 and 0.84. Comparisons with other translations showed comparable fit indices. A lower ICC on Self-monitoring and insight scale was observed. CONCLUSIONS: The Dutch translation of the heiQ was found to be well understood and user friendly by patients with Rheumatoid Arthritis, Atopic Dermatitis, Food allergy and asthma and to have robust psychometric properties for evaluating the impact of health education and self-management programs. Given the wide applications of the heiQ and the comparability of the Dutch results with the English, German and French version, the heiQ is a practical and useful questionnaire to evaluate the impact of self-management support programs in different countries and populations with different diseases

Distribution of health literacy strengths and weaknesses across socio-demographic groups: a cross-sectional survey using the Health Literacy Questionnaire (HLQ)

BACKGROUND: Recent advances in the measurement of health literacy allow description of a broad range of personal and social dimensions of the concept. Identifying differences in patterns of health literacy between population sub-groups will increase understanding of how health literacy contributes to health inequities and inform intervention development. The aim of this study was to use a multi-dimensional measurement tool to describe the health literacy of adults in urban and rural Victoria, Australia. METHODS: Data were collected from clients (n = 813) of 8 health and community care organisations, using the Health Literacy Questionnaire (HLQ). Demographic and health service data were also collected. Data were analysed using descriptive statistics. Effect sizes (ES) for standardised differences in means were used to describe the magnitude of difference between demographic sub-groups. RESULTS: Mean age of respondents was 72.1 (range 19-99) years. Females comprised 63% of the sample, 48% had not completed secondary education, and 96% reported at least one existing health condition. Small to large ES were seen for mean differences in HLQ scales between most demographic groups. Compared with participants who spoke English at home, those not speaking English at home had much lower scores for most HLQ scales including the scales 'Understanding health information well enough to know what to do' (ES -1.09 [95% confidence interval (CI) -1.33 to -0.84]), 'Ability to actively engage with healthcare providers' (ES -1.00 [95% CI -1.24, -0.75]), and 'Navigating the healthcare system' (ES -0.72 [95% CI -0.97, -0.48]). Similar patterns and ES were seen for participants born overseas compared with those born in Australia. Smaller ES were seen for sex, age group, private health insurance status, number of chronic conditions, and living alone. CONCLUSIONS: This study has revealed some large health literacy differences across nine domains of health literacy in adults using health services in Victoria. These findings provide insights into the relationship between health literacy and socioeconomic position in vulnerable groups and, given the focus of the HLQ, provide guidance for the development of equitable interventions