The NHS regularly asks its patients to complete surveys reporting on the quality of care they have received. These surveys include opportunities for patients to submit feedback in their own words. Carol Rivas describes how computational and digital methods can be used to analyse and report patient feedback in an efficient and timely manner. However, it is important to recognise and resolve some of the issues arising from use of these methods; from the need to avoid reducing patients’ voices to data points, to the risks of outputs being viewed as incontestable “truths”

Rivas, Carol

LSE Research Online

3/20/2017Patient experience feedback: we need to engage with the issuesof using Big Data methods to capture the human voiceblogs.lse.ac.uk/impactofsocialsciences/2017/03/20/patient-experience-feedback-we-need-to-engage-with-the-issues-of-using-big-data-methods-to-capture-the-human-voice/The NHS regularly asks its patients to complete surveys reporting on the quality of care they havereceived. These surveys include opportunities for patients to submit feedback in their own words.Carol Rivas describes how computational and digital methods can be used to analyse and reportpatient feedback in an efficient and timely manner. However, it is important to recognise andresolve some of the issues arising from use of these methods; from the need to avoid reducingpatients’ voices to data points, to the risks of outputs being viewed as incontestable “truths”.This post is part of our digital methodologies series, arising from a stream at the ISA Logic andMethodologies conference, September 2016, organised by Mark Carrigan, Christian Bokhove, Sarah Lewthwaiteand Richard Wiggins.The NHS is said to have the most comprehensive patient experience data collection in the world. Much of thiscomes from annual surveys, responses to which suggest that as a nation we are generally pretty satisfied with thehealth care we receive at an individual level. This might seem like good news. But it turns out that what people sayin answer to the closed questions in patient experience surveys (the type that are answered by multiple choicequestions, drop-down menus or measurements) does not always agree with the comments those same patientsmake when answering associated freetext questions (the type that ask “Have you any other comments? Pleasewrite them in this box”, for example). The reason, of course, is that these comments are free of the constraints andspecificities of a closed question. As such they often provide a more complete picture, and rich detail on carefailings, and the way they are written can make them more persuasive vehicles for change than simple yes/noanswers.Image credit: LEDs by Shaun Dunmall. This work is licensed under a CC BY-SA 2.0 license.1/4The challenge of large-volume qualitative data on healthcareUnfortunately it has proved a challenge for health service staff to make sense of freetext feedback from patients inany systematic way. So while it is important to ask patients about the care they receive in order to changehealthcare practice, we cannot be certain how effective this approach actually is.As a way of tackling the problem, we are applying a two-stage approach. First we are analysing the patientexperience survey freetext comments using knowledge engineering, a type of text mining. This involves constructinggazetteers and writing “rules” or algorithms so words in the freetext comments can be tagged and grouped accordingto healthcare topics or themes chosen by patients and healthcare staff. We are then outputting the results as web-based summary visualisations that can be mined down to the original data. Our summaries have been designed toenable frontline staff to make better, more timely use of survey freetext responses to drive healthcare improvements.Each year new survey data could be put through our system and be reported to frontline staff within hours.We have begun with the Cancer Patient Experience Survey, since this is widely acknowledged as the mostsuccessful national patient experience survey in enabling service improvement through summaries of responses toits closed questions. Each year CPES is sent out over a three-month period to all patients treated for cancer as in-patients or day cases in NHS trusts in England. The 70,000+ freetext comments it generates annually, which are ourfocus, are provided to NHS trusts as unstructured raw data.While strictly speaking these are not “Big Data”, their volume is too great for useful and systematic analysis orreporting without using computational, Big Data-style approaches such as ours. For example, when healthcare staffin London tried to analyse the 2013 London CPES freetext comments on top of their other duties, it took them 18months to do so and the analysis was not published until two years after the data had been released. Services werelikely to have changed from the time the data were collected, reducing the impact and direct usefulness to localservice improvement of any analyses. Clearly, then, our approach fulfils a need for staff wishing to use CPES.Using Big and Small Data methodsIt is important to remember that these data represent feedback from people living through often serious healthevents. Computational approaches risk removing the patient from the data. When we began our study, our patientadvisory group was concerned that our results would be mechanistic and reductionist, with survey respondents andthe people and healthcare processes they wrote about reduced to mere numbers, summary generalisations anddata points developed from our own perspectives. So, when developing our process, we interlaced ourcomputational (Big Data-style) work with Small Data inputs, to keep the patient’s voice present throughout, in whatHalford and Savage call a symphonic approach.What do we mean by “Small Data”? Well, in spring 2016, for example, we invited patients, carers and healthcareprofessionals to complete an online survey which prompted them to brainstorm terms and phrases they might usewhen writing about their experiences as a patient or potential patient. From this we discovered not only terms andphrases for our knowledge engineering, but also that patients and professionals had very different perspectives onwhat good care meant. This divergence of opinion was also apparent when we held discussion workshops inautumn 2016 and is something we have tried to take into account in the presentation of our outputs. Ultimately wewant these to drive healthcare improvements that matter to the patients themselves.Data qualitySeveral other issues affect our final outputs. In a second survey sent to patients and healthcare professionals,asking them what they thought were the benefits and drawbacks of dashboard data visualisations, data quality wasconsidered the most critical issue; something we discussed in our workshops.There is a danger our outputs, being based on a national survey, will be seen as absolute and incontestable “truth”by some users of our system, when in fact they can only provide a rough indication of care quality. There are many2/4reasons for this, not least the fact that survey data themselves are typically flawed; for example, ethnic minoritygroups are not well represented in CPES and there are other sampling and reporting biases. Biases also arisebecause we are channelling and filtering the data in particular ways; for example, we are making conscious choicesto select for display on a dashboard only those parts of the data we consider to be significant to healthcareimprovement.As another issue, some trusts may have many comments on a particular aspect of their care, such as waiting times,and other trusts may only have one or two, but this does not necessarily mean that one is better than the other.There may also be errors in the original data entry; Quality Health, commissioned by the NHS to run CPES, had topublish a revision shortly after release of the latest (2015) data because some responses had been assigned to thewrong tumour group. There may be other errors that have not yet been spotted.Issues with data quality will not necessarily be apparent to end users who need to know not just the limits of the dataset, but also the limits of which questions can be asked of it and what interpretations are appropriate.There are two ways we are tackling this. The first is simply to make the limitations clear in an article thataccompanies our visualisations. The second is to develop indicators within our data displays that alert the user tothe relative strength or weakness of that particular bit of data.Good design principles limit the number of themes we can show together on a webpage and users will be able tochoose which ones to look at. In doing so they may find spurious patterns in the data; for example, if staffing levelsand hospital food quality both appear to be rated poorly by patients, the service provider may think they are linkedwhen the correlation may have simply occurred by chance. Such issues may lead us to limit some of theindividualisations that can be made.Opening up the debateMany of the issues we have encountered have been specified by others as the reason the health sciences lagbehind other disciplines in their use of computational Big Data approaches. We hope we have gone some way tobridge the chasm between person-centred care and the depersonalised approach of computing, using ourcombination of Big and Small Data approaches. Our aim has been to produce actionable insights into healthcarethat lead to its improvement in ways that are meaningful to both patients and healthcare staff. But we also hope toopen up a debate on how to deal with the challenges of data dissemination in this area, as more and more of thesepatient experience data become reconstructed into what Boyd and Crawford call a “garden hose” trickle of digestiblebytes designed for professional and public consumption.This blog post is based on the work of the PRESENT project. PRESENT is funded by the National Institute forHealth Research Delivery Research Programme (project number 14/156/15). The views and opinions expressedherein are those of the author and do not necessarily reflect those of the HS&DR, NIHR, NHS or the Department ofHealth.Note: This article gives the views of the author, and not the position of the LSE Impact Blog, nor of the LondonSchool of Economics. Please review our comments policy if you have any concerns on posting a comment below.About the authorCarol Rivas is a Senior Research Fellow in Health Sciences at the University of Southampton, interested in patientexperience, health informatics and digital technologies. She leads on the study described in this post. Her ORCID is0000-0002-0316-8090.Copyright © The Author (or The Authors) - Unless otherwise stated, this work is licensed under a CreativeCommons Attribution Unported 3.0 License.3/44/4

Patient experience feedback: we need to engage with the issues of using big data methods to capture the human voice

Abstract

Similar works

Full text

Available Versions

LSE Research Online