Current Knowledge and Considerations Regarding Survey Refusals: Executive Summary of the AAPOR Task Force Report on Survey Refusals

The landscape of survey research has arguably changed more significantly in the past decade than at any other time in its relatively brief history. In that short time, landline telephone ownership has dropped from some 98 percent of all households to less than 60 percent; cell-phone interviewing went from a novelty to a mainstay; address-based designs quickly became an accepted method of sampling the general population; and surveys via Internet panels became ubiquitous in many sectors of social and market research, even as they continue to raise concerns given their lack of random selection. Among these widespread changes, it is perhaps not surprising that the substantial increase in refusal rates has received comparatively little attention. As we will detail, it was not uncommon for a study conducted 20 years ago to have encountered one refusal for every one or two completed interviews, while today experiencing three or more refusals for every one completed interview is commonplace. This trend has led to several concerns that motivate this Task Force. As refusal rates have increased, refusal bias (as a component of nonresponse bias) is an increased threat to the validity of survey results. Of practical concern are the efficacy and cost implications of enhanced efforts to avert initial refusals and convert refusals that do occur. Finally, though no less significant, are the ethical concerns raised by the possibility that efforts to minimize refusals can be perceived as coercive or harassing potential respondents. Indeed, perhaps the most important goal of this document is to foster greater consideration by the reader of the rights of respondents in survey research

Emotional Exhaustion in Primary Care During Early Implementation of the VA's Medical Home Transformation Patient-aligned Care Team (PACT)

Objective: Transformation of primary care to new patient-centered models requires major changes in healthcare organizations, including interprofessional expectations and organizational policies. Emotional exhaustion (EE) among workers can accompany major organizational change, threatening its success. Yet little guidance exists about the magnitude of associations with EE during primary care transformation. We assessed EE during the initial phase of national primary care transformation in the Veterans Health Administration. Research Design: Cross-sectional online surveys of primary care clinicians (PCCs) and staff in 23 primary care clinics within 5 healthcare systems in 1 veterans administration administrative region. We used descriptive, bivariate, and multivariable analyses adjusted for clinic membership and weighted for nonresponse. Participants: 515 veterans administration employees (191 PCCs and 324 other primary care staff). Measures: Outcome is the EE subscale of the Maslach Burnout Inventory. Predictors include clinic characteristics (from administrative data) and self-reported efficacy for change, experiences with transformation, and perspectives about the organization. Results: The overall response rate was 64% (515/811). In total, 53% of PCCs and 43% of staff had high EE. PCCs (vs. other primary care staff), female (vs. male), and non-Latino (vs. Latino) respondents reported higher EE. Respondents reporting higher efficacy for change and participatory decision making had lower EE scores, adjusting for sex and race. Conclusions: Recognition by healthcare organizations of the potential for clinician and staff EE during primary care transformation is critical. Methods for reducing EE by increasing clinician and staff change efficacy and opportunities to participate in decision making should be considered, with attention to PCCs, and women

Self-administered Web-Based Tests of Executive Functioning and Perceptual Speed: Measurement Development Study With a Large Probability-Based Survey Panel

BackgroundCognitive testing in large population surveys is frequently used to describe cognitive aging and determine the incidence rates, risk factors, and long-term trajectories of the development of cognitive impairment. As these surveys are increasingly administered on internet-based platforms, web-based and self-administered cognitive testing calls for close investigation. ObjectiveWeb-based, self-administered versions of 2 age-sensitive cognitive tests, the Stop and Go Switching Task for executive functioning and the Figure Identification test for perceptual speed, were developed and administered to adult participants in the Understanding America Study. We examined differences in cognitive test scores across internet device types and the extent to which the scores were associated with self-reported distractions in everyday environments in which the participants took the tests. In addition, national norms were provided for the US population. MethodsData were collected from a probability-based internet panel representative of the US adult population—the Understanding America Study. Participants with access to both a keyboard- and mouse-based device and a touch screen–based device were asked to complete the cognitive tests twice in a randomized order across device types, whereas participants with access to only 1 type of device were asked to complete the tests twice on the same device. At the end of each test, the participants answered questions about interruptions and potential distractions that occurred during the test. ResultsOf the 7410 (Stop and Go) and 7216 (Figure Identification) participants who completed the device ownership survey, 6129 (82.71% for Stop and Go) and 6717 (93.08% for Figure Identification) participants completed the first session and correctly responded to at least 70% of the trials. On average, the standardized differences across device types were small, with the absolute value of Cohen d ranging from 0.05 (for the switch score in Stop and Go and the Figure Identification score) to 0.13 (for the nonswitch score in Stop and Go). Poorer cognitive performance was moderately associated with older age (the absolute value of r ranged from 0.32 to 0.61), and this relationship was comparable across device types (the absolute value of Cohen q ranged from 0.01 to 0.17). Approximately 12.72% (779/6123 for Stop and Go) and 12.32% (828/6721 for Figure Identification) of participants were interrupted during the test. Interruptions predicted poorer cognitive performance (P<.01 for all scores). Specific distractions (eg, watching television and listening to music) were inconsistently related to cognitive performance. National norms, calculated as weighted average scores using sampling weights, suggested poorer cognitive performance as age increased. ConclusionsCognitive scores assessed by self-administered web-based tests were sensitive to age differences in cognitive performance and were comparable across the keyboard- and touch screen–based internet devices. Distraction in everyday environments, especially when interrupted during the test, may result in a nontrivial bias in cognitive testing

Long-term impact of evidence-based quality improvement for facilitating medical home implementation on primary care health professional morale

Abstract Background Poor morale among primary care providers (PCPs) and staff can undermine the success of patient-centered care models such as the patient-centered medical home that rely on highly coordinated inter-professional care teams. Medical home literature hypothesizes that participation in quality improvement can ease medical home transformation. No studies, however, have assessed the impact of quality improvement participation on morale (e.g., burnout or dissatisfaction) during transformation. The objective of this study is to examine whether primary care practices participating in evidence-based quality improvement (EBQI) during medical home transformation reduced burnout and increased satisfaction over time compared to non-participating practices. Methods We used a longitudinal quasi-experimental design to examine the impact of EBQI (vs. no EBQI), a multi-level, interdisciplinary approach for engaging frontline primary care practices in developing evidence-based improvement innovations and tools for spread on PCP and staff morale following the 2010 national implementation of the medical home model in the Veterans Health Administration. The sample included 356 primary care employees (107 primary care providers and 249 staff) from 23 primary care practices (6 intervention and 17 comparison) within one Veterans Health Administration region. Three intervention practices began EBQI in 2011 (early) and three more began EBQI in 2012 (late). Three waves of surveys were administered across 42 months beginning in November 2011 and ending in January 2016 approximately 2 years 18 months apart. We used repeated measures analysis of the survey data on medical home teams. Main outcome measures were the emotional exhaustion subscale from the Maslach Burnout Inventory, and job satisfaction. Results Six of 26 approved EBQI innovations directly addressed provider and staff morale; all 26 addressed medical home implementation challenges. Survey rates were 63% for baseline and 48% for both follow-up waves. Age was associated with lower burnout among PCPs (p = .039) and male PCPs had higher satisfaction (p = .037). Controlling for practice and PCP/staff characteristics, burnout increased by 5 points for PCPs in comparison practices (p = .024) and decreased by 1.4 points for early and 6.8 points (p = .039) for the late EBQI practices. Conclusions Engaging PCPs and staff in EBQI reduced burnout over time during medical home transformation

Cluster randomized trial of a multilevel evidence-based quality improvement approach to tailoring VA Patient Aligned Care Teams to the needs of women Veterans.

BackgroundThe Veterans Health Administration (VA) has undertaken a major initiative to transform care through implementation of Patient Aligned Care Teams (PACTs). Based on the patient-centered medical home (PCMH) concept, PACT aims to improve access, continuity, coordination, and comprehensiveness using team-based care that is patient-driven and patient-centered. However, how VA should adapt PACT to meet the needs of special populations, such as women Veterans (WVs), was not considered in initial implementation guidance. WVs' numerical minority in VA healthcare settings (approximately 7-8&nbsp;% of users) creates logistical challenges to delivering gender-sensitive comprehensive care. The main goal of this study is to test an evidence-based quality improvement approach (EBQI) to tailoring PACT to meet the needs of WVs, incorporating comprehensive primary care services and gender-specific care in gender-sensitive environments, thereby accelerating achievement of PACT tenets for women (Women's Health (WH)-PACT).Methods/designEBQI is a systematic approach to developing a multilevel research-clinical partnership that engages senior organizational leaders and local quality improvement (QI) teams in adapting and implementing new care models in the context of prior evidence and local practice conditions, with researchers providing technical support, formative feedback, and practice facilitation. In a 12-site cluster randomized trial, we will evaluate WH-PACT model achievement using patient, provider, staff, and practice surveys, in addition to analyses of secondary administrative and chart-based data. We will explore impacts of receipt of WH-PACT care on quality of chronic disease care and prevention, health status, patient satisfaction and experience of care, provider experience, utilization, and costs. Using mixed methods, we will assess pre-post practice contexts; document EBQI activities undertaken in participating facilities and their relationship to provider/staff and team actions/attitudes; document WH-PACT implementation; and examine barriers/facilitators to EBQI-supported WH-PACT implementation through a combination of semi-structured interviews and monthly formative progress narratives and administrative data.DiscussionLack of gender-sensitive comprehensive care has demonstrated consequences for the technical quality and ratings of care among WVs and may contribute to decisions to continue use or seek care elsewhere under the US Affordable Care Act. We hypothesize that tailoring PACT implementation through EBQI may improve the experience and quality of care at many levels.Trial registrationClinicalTrials.gov, NCT02039856

Patient-Rated Access to Needed Care: Patient-Centered Medical Home Principles Intertwined

BACKGROUND:Primary care teams can facilitate access to care by helping patients to determine whether and when care is needed, and coordinating care across multiple clinicians and settings. Appointment availability metrics may or may not capture these contributions, but patients' own ratings of their access to care provide an important alternative view of access that may be more closely related to these key functions of care teams. PROCEDURES:We used a 2015 telephone survey of 1,395 women veterans to examine associations between key care team functions and patient-rated access to needed care. The care team functions were care coordination, in-person communication (between patient and care team), and phone communication (timely answers to health questions). We controlled for sociodemographics, health status, care settings, and other experience of care measures. KEY FINDINGS:Overall, 74% of participants reported always or usually being able to see a provider for routine care, and 68% for urgent care. In adjusted analyses, phone communication was associated with better ratings of access to routine care (odds ratio [OR], 4.31; 95% CI, 2.65-6.98) and urgent care (OR, 2.26; 95% CI, 1.23-4.18). Care coordination was also associated with better ratings of access to routine care (OR, 1.66; 95% CI, 1.01-2.74) and urgent care (OR, 2.26; 95% CI, 1.23-4.18). Associations with in-person communication were not significant. CONCLUSIONS:Access, communication, and care coordination are interrelated. Approaches to improving access may prove counterproductive if they compromise the team's ability to coordinate care, or diminish the team's role as a primary point of contact for patients

Mental Health Screening Results Associated with Women Veterans’ Ratings of Provider Communication, Trust and Care Quality

BACKGROUND: Identifying factors influencing patient experience and communication with their providers is crucial for tailoring comprehensive primary care for women veterans within the Veterans Health Administration. In particular, the impact of mental health (MH) conditions that are highly prevalent among women veterans is unknown. METHODS: From January to March 2015, we conducted a cross-sectional survey of women veterans with three or more primary care and/or women's health visits in the prior year at 12 Veterans Health Administration sites. Patient measures included ratings of provider communication, trust in provider, and care quality; demographics, health status, health care use; and brief screeners for symptoms of depression, anxiety, and posttraumatic stress disorder. We used multivariate models to analyze associations of patient ratings and characteristics. RESULTS: Among the 1,395 participants, overall communication ratings were high, but significant variations were observed among women screening positive for MH conditions. In multivariate models, high communication ratings were less likely among women screening positive for multiple MH conditions compared with patients screening negative (odds ratio, 0.43; p < .001). High trust in their provider and high care ratings were significantly less likely among women with positive MH screens. Controlling for communication, the effect of MH on trust and care ratings became less significant, whereas the effect of communication remained highly significant. CONCLUSIONS: Women veterans screening positive for MH conditions were less likely to give high ratings for provider communication, trust, and care quality. Given the high prevalence of MH comorbidity among women veterans, it is important to raise provider awareness about these differences, and to enhance communication with patients with MH symptoms in primary care