The third in a series of three blogs produced by the MAX project team which summarises the preliminary findings from an online survey conducted with local authority staff associated with the Adult Social Care Survey (ASCS) and Personal Social Services Survey of Adult Carers in England (PSS SACE or Carers’ Survey)

Caiels, James

Fox, Diane

Heath, Clara

Malley, Juliette

Kent Academic Repository

Kent Academic RepositoryFull text document (pdf)Copyright & reuseContent in the Kent Academic Repository is made available for research purposes. Unless otherwise stated allcontent is protected by copyright and in the absence of an open licence (eg Creative Commons), permissions for further reuse of content should be sought from the publisher, author or other copyright holder. Versions of researchThe version in the Kent Academic Repository may differ from the final published version. Users are advised to check http://kar.kent.ac.uk for the status of the paper. Users should always cite the published version of record.EnquiriesFor any further enquiries regarding the licence status of this document, please contact: researchsupport@kent.ac.ukIf you believe this document infringes copyright then please contact the KAR admin team with the take-down information provided at http://kar.kent.ac.uk/contact.htmlCitation for published versionHeath, Clara and Malley, Juliette and Fox, Diane and Caiels, James  (2013) Emerging themesand provisional ideas for how the toolkits developed during the MAX project can help local authoritiesmaximise the use of data in adult social care.   .   Blog.DOILink to record in KARhttp://kar.kent.ac.uk/59038/Document VersionUNSPECIFIEDBLOG 3: EMERGING THEMES AND PROVISIONAL IDEAS FOR HOW THE TOOLKITS DEVELOPED DURING THE MAX PROJECT CAN HELP LOCAL AUTHORITIES MAXIMISE THE USE OF DATA IN ADULT SOCIAL CARE  In the third and final blog on the online survey results, the MAX project team describe the range of additional support requested by local authorities, and the emerging themes and issues to be explored further during the remainder of the project.  All of these will feed into the developing toolkits and, it is hoped, will facilitate the maximal use of Adult Social Care Survey (ASCS) and Personal Social Services Survey of Adult Carers in England (PSS SACE) data.  We would love to know what you think so please provide feedback via this blog or by email to maxproject@kent.ac.uk.  WHAT THEMES ARE EMERGING FROM THE ONLINE SURVEY RESPONSES?  The feedback from the online survey has highlighted the need for a wide range of training and guidance (e.g. on data analysis, interpretation, communicating the findings), and several significant issues concerning the ways in which LA staff currently view and use the ASCS and PSS SACE.   It is clear that these views are affecting how the data from these surveys are being utilised and that the toolkits developed as part of the MAX project will need to go beyond addressing specific training needs to also challenge underlying perceptions in three areas. Current perceptions about the ASCS and PSS SACE x The ASCS and PSS SACE are designed to inform national policy and practice and do not meet local needs. x The survey questions are too vague and the data is too general to inform policy and practice. x The survey data are primarily for populating performance indicators rather than informing strategy in the longer-term.  MANY LOCAL AUTHORITIES FEEL THAT THE ASCS AND PSS SACE FEED INTO A NATIONAL DATA SET AND DO NOT ADDRESS LOCAL NEEDS  Whilst nearly all of the online survey respondents agreed that the ASCS and PSS SACE provides useful information about the views of services users and carers (N = 100; 72 agree; 21 strongly agree), a considerable number ƌĞũĞĐƚĞĚƚŚĞƐƚĂƚĞŵĞŶƚ “ƚŚĞƐƵƌǀĞǇis a good fit ǁŝƚŚůŽĐĂůƌĞƐĞĂƌĐŚƉƌŝŽƌŝƚŝĞƐ ? ?^^EA? ? ? ? ? ? ? ?ƐƚƌŽŶŐůǇĚŝƐĂŐƌĞĞ ? ? ?ĚŝƐĂŐƌĞĞ ? ?W^^^E= 35/91 [3 strongly disagree; 32 disagree]) and maintained  “ƚŚĞ ƐƵƌǀĞǇ ƋƵĞƐƚŝŽŶƐ ĂƌĞ ŶŽƚƵƐĞĨƵůĨŽƌ ŝŶĨŽƌŵŝŶŐƉŽůŝĐǇĂŶĚƉƌĂĐƚŝĐĞ ?  ?^^EA? ? ?  ? ?  ? ?ƐƚƌŽŶŐůǇĂŐƌĞĞ ? ? ?ĂŐƌĞĞ ? ?W^^SACE N = 27/91 [2 strongly agree; 25 agree]). In other words, although the data from the surveys is seen as informative by the majority of respondents, it is often the case that it does not completely fulfil local research needs and priorities.  As one respondent commented:  Some of the questions are deemed not useful at local level, sample is taken to satisfy survey results not to inform on local initiatives  There is no doubt that the perception that local needs are not addressed by the ASCS and PSS SACE has a direct impact on how LAs apply the survey data to policy and practice.  We will work with LAs to generate ideas for making the survey data a better fit to local needs.  Options that could be explored within the toolkit include: (1) highlighting the option to include local questions and additional comments boxes to the surveys to address local priorities, and providing appropriate practical and methodological guidance; (2) developing or co-ordinating the development of a collection of additional questions; (3) producing training materials and worked examples on how to drill down into the survey data. For instance, step by step guides to conduct more complex analysis, such as cross tabulations to explore the relationships between different measures ?ĂŶĚ ? ? ?ŐƵŝĚĞůŝŶĞƐŽŶŚŽǁƚŽ “ĨŝůůƚŚĞŐĂƉƐ ? Žƌ ĐŽŵƉůĞŵĞŶƚ ƚŚĞ ƐƵƌǀĞǇ ĚĂƚĂsets, by drawing on administrative data from their own systems and making ƵƐĞŽĨŽƚŚĞƌĚĂƚĂƐŽƵƌĐĞƐ ?Ğ ?Ő ? ?E,^ĐŚŽŝĐĞƐ ?ĂŶĚsmall area data, existing local and national research).  THE SURVEY QUESTIONS ARE VAGUE AND THE RESULTING DATA IS TOO GENERAL TO INFORM POLICY AND PRACTICE  It is also apparent that many local authority staff feel that the ASCS and PSS SACE questions are too generic and vague, and as a consequence are not sufficiently specific enough to inform changes in policy or practice.  This viewpoint is discussed in our previous blog and is an area that we would like to address through the toolkit.    One reason behind this viewpoint may be the focus of the ASCS and PSS SACE on the outcomes of social care (e.g. the social care related quality of life of the service user)  ? rather than the processes of care (e.g. did the care worker do what was required or expected?) as in the previous user experience surveys  ? and the more complex analysis required to make sense of the data.  As one respondent noted,  In the old user surveys, there was a clear and unambiguous link between the question ("do homecare workers turn up at times that suit you?") and service delivery.  It was easy to take action based on the responses.  With the ASCOF measures and ASCOT methodology, it's not.  This has weakened the perceived value of the survey, even though the outcomes data captured is arguably even more important than knowing that home care workers do or don't turn up on time...  Help is needed with how to present this to management in a way that will interest them from a policy/service development perspective.  As this respondent highlights, multiple approaches are likely to help challenge this perception.  The toolkit could provide more information about the rationale for individual ƋƵĞƐƚŝŽŶƐ  ?ĂƐǁĞĚŝƐĐƵƐƐ ůĂƚĞƌ ŝŶ ƚŚŝƐďůŽŐ ? ?ďƵƚǁĞĐŽƵůĚĂůƐŽ ůŽŽŬĂƚĚĞǀĞůŽƉŝŶŐ  ?ŚŽǁƚŽ ?guides or templates to help LAs (1) analyse the survey data, (2) interpret the data and (3) communicate their findings to different audiences  ? in particular, the managers whose role it is to shape local policy and practice.  THE ASCS AND PSS SACE ARE SEEN AS A PERFORMANCE INDICATOR RATHER THAN A TOOL FOR LONG TERM PLANNING  Despite the recent policy shift away from national performance targets towards a system focused on local accountability, the ASCS and PSS SACE are still regarded as sources of data to populate performance indicators.  They are described variously as  “ũƵƐƚ Ă ƚŝĐŬ ďŽǆexeƌĐŝƐĞ ?,  “ĂŬĞǇĐŚĞĐŬƚŚĂƚǁĞĂƌĞŶŽƚŐĞƚƚŝŶŐƚŚŝŶŐƐǀĞƌǇďĂĚůǇǁƌŽŶŐ ? ? “primarily a quality assurance monitoring tool rather than an evaluation tool to influence policy and practice ? ?ĂŶĚ “'performance' surveys rather than potential shapers of policy and ƉƌĂĐƚŝĐĞ ?.   However, given the less than clear link between the data from the surveys and service delivery (as we have highlighted above) these data seem unsuited to performing this very narrow performance monitoring role.  The survey data require deeper interrogation and seem more suited to informing strategy and longer-term planning than monthly performance monitoring.  In the next phase of the project we will explore the various ways in which the survey data could be used within LAs and develop guidance to help LAs make use of the data for multiple purposes.   LOCAL AUTHORITY STAFF HAVE ASKED FOR A WIDE RANGE OF ǥ  The feedback in the online survey suggests that the toolkits developed during the MAX project will have to accommodate a range of needs.  Indeed, local authority staff have requested additional support and guidance in a number of different areas as detailed in Figure 1 below.   Figure 1: aspects of the survey or research process that OS respondents want the toolkit to address, by survey type  ǥWHAT TYPE OF HELP IS NEEDED?  As can be seen in Figure 1, the majority of respondents require help with interpreting the survey findings (ASCS N=70; PSS SACE N = 61), data analysis (ASCS N=60; PSS SACE N = 51), communicating the findings to different audiences (ASCS N=56; PSS SACE N = 50) and the development of survey questions (ASCS N=51; PSS SACE N = 44).  We explore these aspects in more detail below.  Support is also required for the sampling, administration and data entry aspects of the survey process, but we do not explore these aspects in detail here.  We intend to summarise this feedback and present it to the Health and Social Care Information Centre (HSCIC) who are responsible for advising on the methodological aspects of the user experience surveys for social services, before deciding on whether to take these aspects forward to the development of the toolkit.  SUPPORT TO INTERPRET THE FINDINGS  We need to be able to illustrate for strategic managers and commissioners the link between outcomes as measured by the survey and the controllable factors associated with service delivery that we could do something about.  Help with interpreting the survey findings was the most common request (ASCS N=70; PSS SACE N = 61) and, from comments provided by many online respondents, it appears that the guidance in the toolkits will need to encompass both the interpretation of ASCS and PSS SACE data (e.g. what does the data mean for policy and practice?) and the localised application of these findings (e.g. how can we act on these findings? Who/which teams within my organisation are responsible? What specific actions need to be taken?).    Some LA staff are currently struggling to translate the survey findings into ideas and actions that can guide local policy and practice to improve service delivery so, whilst technical guidance is already provided by the Department of Health and the HSCIC (e.g. how to calculate the ASCOF indicator in the ASCOF handbook of definitions), there may be a need for more practical support (e.g. what to do with the ASCOF indicator once it has been calculated).  Two issues to consider during future collaborative project activities are then (1) what guidance to provide and (2) how to present this in a manner that is both comprehensible and practical to a wide range of LA staff, and will enable them to make sense of the data in a way that is useful for steering policy and practice.  Much of this guidance will be concerned with analysis, which is discussed further in the next section of this blog, and will have to be based upon the questions local authorities want to ask.  From the responses to the online survey and our on-going conversations with LA representatives (e.g. survey leads, managers and practitioners during the telephone interviews) it appears that they want to know: (1) how to assess how well/badly the authority is doing, (2) how to improve scores for the following year, and (3) where to focus efforts to achieve such improvements.  These questions, however, will be kept under review.     SUPPORT TO ANALYSE THE SURVEY DATA  Support with analysing the ASCS and PSS SACE data was the second most popular request (ASCS N=60; PSS SACE N = 51) with many LA staff asking for guidance on both what to analyse (e.g. which questions or relationships between particular variables to explore locally) and how to conduct specific analysis (e.g. benchmarking trends in outcomes, comparing current results with those of previous years or from national data sets and other research).  For example,   A set of national key findings would be helpful as would suggested areas for exploration locally e.g. relationship between feeling clean and dignity; personal budget and satisfaction.   A notable number of respondents expressed a desire to boost their understanding and use of statistical methods and concepts, such as confidence intervals, cross tabs and correlations, so it appears that two kinds of support are actually needed here: (1) training on general statistics and (2) guidance on survey specific analysis.  Two requests to develop a software package or interactive tool to conduct correlations between different results were also received but, they are beyond the scope of the MAX project.  Nevertheless it may be valuable to explore this idea further, with the MAX advisory group and HSCIC, the latter of whom are currently developing an interactive visualisation tool.  SUPPORT TO COMMUNICATE THE FINDINGS TO DIFFERENT AUDIENCES AND PRODUCE WRITTEN REPORTS  Help in communicating the survey data to different audiences and producing written reports was the third and fifth most popular request in the online survey (ASCS N=56; PSS SACE N = 50 and ASCS N=45; PSS SACE N = 37 respectively) and it is clear from the accompanying commentary that many LA staff are keen to learn reporting strategies that will promote an active engagement with the ASCS and PSS SACE.  At present is seems that, whilst many managers are supportive of the surveys,   [r]eporting of the results is often little more than a report presented at meetings to interested parties with no actions.  Successful communication depends on a number of factors, in this particular instance: (1) extracting the outcomes and messages from the survey findings that are of most relevance / interest to the target audience; (2) illustrating or presenting the findings in an accessible and meaningful way (3) identifying the best means of effectively conveying these messages or findings (e.g. face to face presentation or discussion, a report or short summary), and (4) determining the appropriate length or duration of this communication.  Providing appropriate guidance within the toolkits will therefore require input from a wide range of stakeholders including LA colleagues (e.g. senior managers, commissioners, practitioners); service users, carers and members of the public; and providers.    INFORMATION ON THE DEVELOPMENT OF SURVEY QUESTIONS  Nearly half of the survey respondents were also interested in learning more about the development of the ASCS and PSS SACE questions (ASCS N=51; PSS SACE N = 44).  Research summaries of survey development work and the rationale underlying each question will be provided in the toolkits and, it is hoped, will successfully convey to LA staff both the purpose of each question and the scope for the resulting data to be applied to the local context.  OTHER FORMS OF SUPPORT  The other key forms of support requested by local authority staff in the online survey are summarised in Table 1 below.    Request for help, guidance or support No. of respondents (ASCS; PSS SACE) Possible ideas for toolkit Sampling 37; 29  Step by step guide to sampling; overview of different sampling strategies Data entry 25; 20  More flexible and accessible data entry tools Administration of the survey 24; 19  ŚĞĐŬůŝƐƚƐ ?ĚŽ ?ƐĂŶĚĚŽŶ ?ƚƐůŝƐƚ Table 1: Summary of the other areas where help has been requested by LA staff in the online survey As we have mentioned already, we intend to summarise this feedback and present it to the HSCIC, before deciding on whether to take these aspects forward to the development of the toolkit.   THE MAX PROJECT: NEXT STEPS  The responses to the online survey have provided us with some interesting insights into how local authorities currently view and use the ASCS and PSS SACE, and will now be combined with the findings from the on-going telephone interviews and document analysis.  These will then be summarised in a further blog (to be circulated later in the year) and be used to develop draft ideas for the toolkits.  Requests or enquiries that are beyond the current scope of the MAX project will be summarised and discussed with the appropriate parties, e.g. the MAX advisory group, the HSCIC and Department of Health for further consideration.  The MAX project team are also recruiting to the consultation panels  ? further information about which can be found at www.maxproject.org.uk  ? and aim to start developing these ideas into content for the toolkit with LAs, through workshops and via an online forum, in early 2014. If you would like to take part in these discussions or have any comments about these blogs, please do get in touch either via email or by calling Clara Heath on 01227 823963.  Disclaimer: The research on which this blog is based was funded by the Department of Health and undertaken by researchers at the Quality and Outcomes of Person-centred Care Research Unit (QORU). The views expressed here are those of the authors (the MAX project team) and are not necessarily shared by any individual, government department of agency.   

Emerging themes and provisional ideas for how the toolkits developed during the MAX project can help local authorities maximise the use of data in adult social care

http://kar.kent.ac.uk/59038/1/EMERGING%20THEMES%20AND%20PROVISIONAL%20IDEAS%20FOR%20THE%20TOOLKITS%20%28BLOG%203%29.pdf

Emerging themes and provisional ideas for how the toolkits developed during the MAX project can help local authorities maximise the use of data in adult social care

Abstract

Similar works

Full text

Available Versions

Kent Academic Repository