Recent years have seen an increased focus on measuring how people experience health services to ensure that care and treatment is of the highest quality and safety.1 2 Better patient care experiences are associated with better adherence, clinical outcomes and patient safety, and with lower health care utilisation.3 4 In England, the national GP Patient Survey (GPPS) measures patients’ experience of Primary Care.5 As part of an NHS initiative to improve patient experience and facilitate patient-centred care, GPPS scores are currently reported at the level of general practices, Clinical Commissioning Groups (CCGs) and nationally.Not externally funde

Abel, Gary

Burt, Jenni

Campbell, John L

Elliott, Marc

Fletcher, Emily

Gomez-Cano, Mayam

English

Apollo (Cambridge)

Confidential: For Review OnlyThe role of practices and Clinical Commissioning Groups in measures of patient experience: analysis of routine data.Journal: BMJ Quality & SafetyManuscript ID bmjqs-2020-011701.R1Article Type: Short reportKeywords: Patient satisfaction, General practice, Healthcare quality improvement, Health services research, Patient-centred care https://mc.manuscriptcentral.com/bmjqsBMJ Quality & SafetyConfidential: For Review OnlyI, the Submitting Author has the right to grant and does grant on behalf of all authors of the Work (as defined in the below author licence), an exclusive licence and/or a non-exclusive licence for contributions from authors who are: i) UK Crown employees; ii) where BMJ has agreed a CC-BY licence shall apply, and/or iii) in accordance with the terms applicable for US Federal Government officers or employees acting as part of their official duties; on a worldwide, perpetual, irrevocable, royalty-free basis to BMJ Publishing Group Ltd (“BMJ”) its licensees and where the relevant Journal is co-owned by BMJ to the co-owners of the Journal, to publish the Work in this journal and any other BMJ products and to exploit all rights, as set out in our licence.The Submitting Author accepts and understands that any supply made under these terms is made by BMJ to the Submitting Author unless you are acting as an employee on behalf of your employer or a postgraduate student of an affiliated institution which is paying any applicable article publishing charge (“APC”) for Open Access articles. Where the Submitting Author wishes to make the Work available on an Open Access basis (and intends to pay the relevant APC), the terms of reuse of such Open Access shall be governed by a Creative Commons licence – details of these licences and which Creative Commons licence will apply to this Work are set out in our licence referred to above. Other than as permitted in any relevant BMJ Author’s Self Archiving Policies, I confirm this Work has not been accepted for publication elsewhere, is not being considered for publication elsewhere and does not duplicate material already published. I confirm all authors consent to publication of this Work and authorise the granting of this licence. Page 1 of 4https://mc.manuscriptcentral.com/bmjqsBMJ Quality & Safety123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960Confidential: For Review OnlyThe role of practices and Clinical Commissioning Groups in measures of patient experience: analysis of routine dataGomez-Cano, Fletcher, Campbell, Elliott, Burt, AbelWord count excluding table and refs: 1223IntroductionRecent years have seen an increased focus on measuring how people experience health services to ensure that care and treatment is of the highest quality and safety.1 2 Better patient care experiences are associated with better adherence, clinical outcomes and patient safety, and with lower health care utilisation.3 4 In England, the national GP Patient Survey (GPPS) measures patients’ experience of Primary Care.5 As part of an NHS initiative to improve patient experience and facilitate patient-centred care, GPPS scores are currently reported at the level of general practices, Clinical Commissioning Groups (CCGs) and nationally.English CCGs are clinically-led NHS bodies that commission local health care services. There were 209 CCGs in 2016, though there are now fewer, on account of recent practice-mergers. Although GPPS scores are reported for CCGs, little is known about the influence CCGs have on patient experience scores. We have previously shown that patient experience scores vary considerably between GPs within a practice, and that measures reported at practice level can mask this variation.6 The quality of care or patient experience delivered at any one practice may be influenced by drivers from higher organisations such as CCGs. Recently, CCGs have been shown to influence the rate and accuracy of “fast-track” cancer referrals from English primary care.7 Here we explore whether there was systematic variation in the quality of patient experience between CCGs compared with variability between practices themselves.MethodsData were analysed from 836,172 GPPS respondents in 2015/16 (response rate 38.9%). The GPPS survey was conducted in two waves (July-September 2015 and January-March 2016). Full details of the survey development and methodology are published elsewhere.8 Patient experience measuresWe focused on seven patient experience measures which have been commonly used in previous research, and which reflect key aspects of patient experience6 9-13. Six were based on single survey items: (i) access (difficulty making an appointment), (ii) continuity of care (how often it is possible to see a preferred GP), (iii) helpfulness of receptionists, (iv) overall experience, (v) out-of-hours speed of advice and (vi) out-of-hours overall experience. For the seventh measure, a composite GP communication score was calculated from the mean rating across five linked communication items among patients providing three or more informative responses. Responses to all items were rescaled linearly from 0 to 100 (most favourable), following previous research6 9-11. Statistical AnalysisFor each outcome measure, a 3-level mixed-effect linear regression model was fitted (patients nested within practices nested within CCGs). Patient age, gender, ethnicity, deprivation, and presence of a long-term condition were included as fixed effects. Each model estimated three variance components: residual, practice and CCG. We focused on variances associated with CCGs and practices, and express them as percentages of their sum. From this we can quantify the percentage of the variation in practice GPPS scores attributable to either practices or CCGs. Where the percentage associated with Page 2 of 4https://mc.manuscriptcentral.com/bmjqsBMJ Quality & Safety123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960Confidential: For Review OnlyCCGs is high, the implication is that much of the differences in practice scores is being driven by factors aligned with CCGs. Where the percentage associated with CCGs is low, this implies that practice-level factors are dominant in predicting practice scores. As some variability between practices and CCGs may reflect structural differences between organisations, rather than organisational policies, we ran an additional set of models adjusting for region (fourteen large regions covering England) rurality (rural/urban based on practice location) and the interaction between them. Comparison of the variance associated with CCGs between models with and without rurality provides some insight into the extent to which differences between CCGs reflect the geography covered by CCGs and regional differences in staffing (e.g. recruitment).Analyses were performed using R version 3.4.4.ResultsTable 1 shows the estimated variance components. Before adjusting for region and rurality, the variation in practice scores for the out-of-hours items was mostly associated with the CCG (57% for out-of-hours speed of advice and 56% for out-of-hours overall experience), an area for which CCGs have statutory responsibility. For all other items, practice score variation was mainly associated with the practice, though GP communication and overall experience had notable CCG contributions (17% and 14% of variance associated with CCGs, respectively). A considerable proportion of between-CCG variance can be explained by region and rurality. This was largest for GP communication (43%), helpfulness of receptionists and overall experience (40% for both), and smallest for continuity of care (4%).Table 1 Variance associated with CCGs and practices for the seven GPPS outcome measures estimated with the hierarchical models.Percentage of practice score variance associated with*Outcome CCG PracticePercentage of between CCG variance associated with region and Rurality†Access 11.8 88.2 30.3Continuity of care 4.9 95.2 4.3Overall experience 14.2 85.8 40.4Helpful receptionists 8.1 91.9 40.4GP Communication 16.7 83.3 42.6Out of hours speed of access 56.5 43.5 36.0Out of hours overall experience 57.1 42.9 38.3*Estimated from a model without region and rurality.†Estimated from a model including region and rurality. The effects of rurality are inconsistent in direction and magnitude across both region and patient experience items; similar inconsistencies exist by region as permitted by the interaction term in the model.DiscussionOur findings suggest that CCGs may have a greater influence than individual practices on patient experience scores relating to aspects of care that are under the control of CCGs, such as out-of-hours care. Although CCGs were observed to contribute to other aspects of patient experience, the contribution from practices was much greater – in agreement with expectations, given that these are generally under the direct control of practices.  Similar patterns of variation have previously been seen Page 3 of 4https://mc.manuscriptcentral.com/bmjqsBMJ Quality & Safety123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960Confidential: For Review Onlyfor US health plans.14 Various mechanisms exist through which CCGs may exert influence; for example, local enhanced services (LESs) may have focused efforts on improving access as well as introduction of the extended hours directed enhanced service. Furthermore, at the time this data was collected, important transitional changes were taking place in the commissioning of primary care services, with transfer of responsibility progressing from centralised (NHS England) to local (CCG) commissioning, and thus with a potential local influence on patient experience. Certainly, some of the variation associated with CCGs can be explained by macro-scale structural differences such as larger region and rurality, and the larger-scale approach adopted, for example, at regional rather than at practice-level in relation to configuring the delivery of out-of-hours GP services. The contribution of region and rurality is largest for measures related directly to staffing and overall experience, potentially reflecting geographic variation in the availability of high-quality staff, which has led to schemes aimed at attracting GPs to underserved areas.15This study is not without limitations. The GPPS response rate is 39%. Although this is typical of such surveys16 research suggests that non-response does not meaningfully affect relative performance after accounting for case-mix.16 We also note that our analysis addresses the data which are publicly reported - any biases in our analysis will also be present in those publicly reported measures. Also, we have only accounted for macro-scale structural influences using large-scale proxy measures for region and rurality. There may be other factors of influence which we have not accounted for, for example the quality of secondary care or access to particular services.To improve patient experience, or indeed any aspect of quality, it is essential to recognise and understand which parts of the health system are responsible for, and influence, different domains of this experience.17 Just because a measure of patient experience is reported at a certain level of organisational structure does not mean that that level is the most relevant when considering impact on the actual experience of patients.  This study builds on previous work and suggests that individual GPs, practices and CCGs (and newly emerging Primary Care Networks) all have a role to play, and that the magnitude of that role may be very different for different aspects of patient experience. As such quality improvement efforts may be targeted at all levels of the system, tailored to the scope for improvement that exists. In this study, practices were shown to be associated with substantial variation in out-of-hours care experience and on this account, improvement efforts at practice-level rather than CCG level may be warranted.References1. The Health Foundation. Measuring patient experience. Evidence scan, 2013.2. Coulter A, Locock L, Ziebland S, et al. Collecting data on patient experience is not enough: they must be used to improve care. BMJ (Clinical research ed) 2014;348(mar26 1):g2225.3. Doyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open 2013;3(1) doi: 10.1136/bmjopen-2012-0015704. Anhang Price R, Elliott MN, Zaslavsky AM, et al. Examining the role of patient experience surveys in measuring health care quality. Med Care Res Rev 2014;71(5):522-54. doi: 10.1177/1077558714541480 [published Online First: 2014/07/17]5. Ipsos MORI. GP Patient Survey – Technical Annex 2017 annual report, 2017.6. Roberts MJ, Campbell JL, Abel GA, et al. Understanding high and low patient experience scores in primary care: analysis of patients’ survey data for general practices and individual doctors. BMJ : British Medical Journal 2014;3497. Burton C, O'Neill L, Oliver P, et al. Contribution of primary care organisation and specialist care provider to variation in GP referrals for suspected cancer: ecological analysis of national data. BMJ Quality &amp; Safety 2020;29(4):296-303. doi: 10.1136/bmjqs-2019-0094698. Campbell J, Smith P, Nissen S, et al. The GP Patient Survey for use in primary care in the National Health Service in the UK--development and psychometric characteristics. BMC Fam Pract 2009;10:57. doi: 10.1186/1471-2296-10-57 [published Online First: 2009/08/25]Page 4 of 4https://mc.manuscriptcentral.com/bmjqsBMJ Quality & Safety123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960Confidential: For Review Only9. Lyratzopoulos G, Elliott M, Barbiere JM, et al. Understanding ethnic and other socio-demographic differences in patient experience of primary care: evidence from the English General Practice Patient Survey. BMJ Quality &amp; Safety 2012;21(1):21-29. doi: 10.1136/bmjqs-2011-00008810. Lyratzopoulos G, Elliott MN, Barbiere JM, et al. How can health care organizations be reliably compared?: Lessons from a national survey of patient experience. Medical care 2011;49(8):724-33. doi: 10.1097/MLR.0b013e31821b3482 [published Online First: 2011/05/26]11. Burt J, Lloyd C, Campbell J, et al. Variations in GP-patient communication by ethnicity, age, and gender: evidence from a national primary care patient survey. Br J Gen Pract 2016;66(642):e47-52. doi: 10.3399/bjgp15X687637 [published Online First: 2015/11/07]12. Warren FC, Abel G, Lyratzopoulos G, et al. Characteristics of service users and provider organisations associated with experience of out of hours general practitioner care in England: population based cross sectional postal questionnaire survey. BMJ : British Medical Journal 2015;350:h2040. doi: 10.1136/bmj.h204013. Paddison CA, Abel GA, Roland MO, et al. Drivers of overall satisfaction with primary care: evidence from the English General Practice Patient Survey. Health expectations : an international journal of public participation in health care and health policy 2015;18(5):1081-92. doi: 10.1111/hex.12081 [published Online First: 2013/06/01]14. Zaslavsky AM, Zaborski LB, Cleary PD. Plan, geographical, and temporal variation of consumer assessments of ambulatory health care. Health Serv Res 2004;39(5):1467-85. doi: 10.1111/j.1475-6773.2004.00299.x15. National Health Executive. Workforce and training 2016 [Available from: http://www.nationalhealthexecutive.com/Health-Care-News/over-100-gps-to-be-sent-to-unpopular-regions-with-20000-bursary accessed 21/02/2019.16. Elliott MN, Zaslavsky AM, Goldstein E, et al. Effects of survey mode, patient mix, and nonresponse on CAHPS hospital survey scores. Health services research 2009;44(2 Pt 1):501-18. doi: 10.1111/j.1475-6773.2008.00914.x [published Online First: 2009/03/26]17. Round T, Abel G. Seeing the wood and the trees: the impact of the healthcare system on variation in primary care referrals. BMJ Quality &amp; Safety 2020;29(4):274-76. doi: 10.1136/bmjqs-2019-010356Page 5 of 4https://mc.manuscriptcentral.com/bmjqsBMJ Quality & Safety123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

Role of practices and Clinical Commissioning Groups in measures of patient experience: analysis of routine data.

https://www.repository.cam.ac.uk/bitstream/1810/310066/1/Submitted%20BMJQS%20Short%20report%20reviewed.pdf

Role of practices and Clinical Commissioning Groups in measures of patient experience: analysis of routine data.

Abstract

Similar works

Full text

Available Versions

Apollo (Cambridge)