Greaves, Colin J.

Jones, Mat

Keable, Jaine

Kok, Michele

Smith, Jane

Solomon-Moore, Emma

Walsh, Alexis

English

OPUS

        Citation for published version:Kok, M, Jones, M, Solomon-Moore, E, Smith, J, Greaves, CJ, Keable, J & Walsh, A 2016, 'Evaluating the fidelityof implementation of a voluntary sector-led, community-based diabetes prevention and managementprogramme', South West Public Health Conference 2016, Bristol, 16/03/16 - 16/03/16 pp. 1-2.Publication date:2016Link to publicationUniversity of BathAlternative formatsIf you require this document in an alternative format, please contact:openaccess@bath.ac.ukGeneral rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.Take down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Download date: 14. May. 2024Michele Kok1, Mat Jones1, Emma Solomon-Moore2, Jane Smith3, Colin Greaves3, Jaine Keable4, Alexis Walsh41University of the West of England; 2University of Bristol; 3University of Exeter Medical School; 4Westbank Community Health & Care, DevonINTRODUCTION• Living Well, Taking Control (LWTC) is a community-based type 2diabetes (T2D) prevention and management programme:o Objective: To promote sustainable healthy lifestyle changeso Target population: People with pre-diabetes & newly-diagnosed T2Do Core component of intervention: Group-based structured educationsessions delivered weekly, for 4 weeks, by trained facilitators• This intervention was designed to meet evidence-basedrecommendations from NICE1.• The clinical and cost effectiveness of the diabetes prevention componentof LWTC is being evaluated in the ComPoD trial.• The Fidelity of Implementation (FoI) Study is part of a wider serviceevaluation of LWTC, and is critical to successful translation of evidence-based interventions into practice.• The heart of fidelity is often considered to be intervention delivery,whose core components are adherence and competence2,3:o Adherence – extent to which facilitators conform to the interventionprotocolo Competence – skilfulness in the delivery of the interventionAIM: To assess the fidelity of implementation of the LWTC programme,with a focus on facilitator adherence and competence.Table 2: Level of implementation scores for adherence criteriaRESULTSPARTICIPANT CHARACTERISTICS• Using questionnaire data, t-tests were conducted to see if there were anysignificant differences between the participant characteristics of the FoIStudy sample compared to the wider Westbank sample.• There were no significant differences between the groups except for thefollowing characteristics:o Participants in the fidelity groups were significantly heavier (p<0.05)but had a significantly lower HbA1c (p<0.05) than the overallWestbank participants.o The overall Westbank participants had a significantly higher educationlevel than participants in the fidelity groups (p<0.05).ADHERENCE• Examples of adherence items on the fidelity checklist:o Assess importance & confidence in making healthy lifestyle changeso Goal-setting or review goals set• Results suggest that the group sessions were delivered to a typical sampleof programme participants, which allows the intervention outcomes to begeneralised, to a certain degree, to the wider Westbank sample.• It might have been challenging for the facilitator to address all thediabetic participants’ questions within the allocated time of the session,while still adhering to the protocol.• Neither facilitator had previous training in mental health and wellbeingsupport, which may have affected confidence in delivering Session 4.• The moderate inter-rater agreement for the adherence criteria, may be dueto the raters having varying interpretations of some of the criteria.• All participants were required to provide consent at the start of Session 1.The ‘confidentiality agreement’ criterion was a measure of facilitators’competence in creating an open and safe environment – it may beassumed to have been implemented prior to the recorder being turned on.• Lack of local facilities, facilitator expertise, or time, are possible reasonswhy optional activities/items were not implemented more often.• It is recommended to review the programme protocol to give clearerguidance and enhance facilitator training in the area of mental health andwellbeing, in order to improve delivery of that intervention component.• A more robust method of assessing facilitator competence may need to beimplemented.• The LWTC programme facilitators displayed a satisfactory level ofadherence and a high level of competence.• The level of fidelity established for the LWTC group-based educationintervention is considered appropriate, and will provide some confidencein findings related to intervention effectiveness.• The study demonstrated the viability and value of measuring fidelity in avoluntary sector-led public health initiative.The LWTC programme and evaluation are funded by the Big Lottery. 1. NICE. Preventing type 2 diabetes: Risk identification and interventions for individualsat high risk (PH38). London: NICE; 2012. http://www.nice.org.uk/PH382. Mars, T., Ellard, D., Carnes, D., et al. (2013). Fidelity in complex behaviour changeinterventions: a standardised approach to evaluate intervention integrity. BMJ Open,3:e003555.3. Gearing, R.E., El-Bassel, N., Ghesquiere, A., et al. (2011). Major ingredients offidelity: A review and scientific guide to improving quality of intervention researchimplementation. Clinical Psychology Review, 31: 79-88.4. Peat, J. (2001). Health science research: A handbook of quantitative methods. Sydney:Allen & Unwin.Table 1: LWTC fidelity checklist item configurationEvaluating the fidelity of implementation of a voluntary sector-led, community-based diabetes prevention and management programmeMETHODSSTUDY SETTING• Four facilities in Devon: 3 in Exeter, 1 in Tiverton• Programme delivered by the voluntary-sector organisation, WestbankDATA COLLECTION• Audio recording of sessions conducted from 20 January to 5 March 2015o 5 pre-diabetes and 2 diabetes groups (total of 49 participants)o 28 sessions = 49 hours of audio recording• Course satisfaction data was used to support findings on competenceDATA ANALYSIS• Audio recordings were analysed using a fidelity checklist• The level of implementation for each item was rated: Low/not observed(1 point); Observed to a small degree (2 points); Observed to a mediumdegree (3 points); High implementation (4 points)• The level of implementation score for each component= Sum of compulsory items score / Number of compulsory items• Overall level of implementation score for each group= Average of the scores from the four sessions• The goal for an acceptable level of implementation was set at 80%• 10% recordings tested with Kappa statistics (κ)4 for inter-rater agreementIMPLEMENTATION OF OPTIONAL ITEMS• The overall percentage of optional items implemented ranged from45.45% to 63.64% across all groups.• ‘Offering refreshments’ and ‘Repeating clinical metrics’ were alwaysimplemented.• The optional walk or seated exercise in Session 3 was implemented forone group.• The optional relaxation exercise in Session 4 was never implemented.• Signposting to healthcare professionals, local services, or additionalsupport were carried out as required.• None of the three additional optional items for diabetic participants wereimplemented, i.e. expectations from healthcare professional, informationabout annual reviews, and the 15 Healthcare Essentials.OVERALL IMPRESSION OF GROUP DYNAMICSGeneral observations of group dynamics from the audio recordings:Facilitators• Professional; patient; handled questions well, with good explanations• Effectively encouraged group participation and engagement• Encouraged participants to share ideas and support each other in makingchanges• Supportive of participants who were negative, demotivated, or lessconfident in making healthy lifestyle changesParticipants• Good overall contribution to the discussions; supportive of one another• Several groups were fairly quiet at the start, but became more talkativetowards the end of Session 1• The facilitator described group D20 as “well-informed” and “well-read”,and expressed that “time always ran away” with this group.• Group D21 was comparatively small and everyone actively participatedin the discussions. One participant expressed that it was “so much easierand more comfortable to ask questions at this session, compared to theother diabetes session” conducted by another organisation.Session: Programme componentNo. of compulsory itemsNo. of optional items1: Pre-diabetes/T2D & a healthy lifestyleAdherence items: 11Competence items: 5Total: 1622: Healthy eating Adherence items: 13Competence items: 3Total: 1613: Physical activity Adherence items: 11Competence items: 3Total: 146(3 of these applied to diabetic participants only)4: Positive mental health & wellbeingAdherence items: 13Competence items: 3Total: 165CONCLUSIONREFERENCESACKNOWLEDGEMENTDISCUSSIONGroup ID[Pre-diabetes (P); Diabetes (D)]Level of implementation scores for adherence items (on a scale of 0-4) Overall level of implementation scoreSession 1 Session 2 Session 3 Session 4P31 3.55 3.31 3.55 2.54 3.21P32 3.56 2.92 3.36 3.00 3.17P33 3.73 3.00 3.64 2.62 3.21P34 3.73 3.08 3.55 2.77 3.25P35 3.64 3.00 3.73 2.85 3.27D20 3.45 2.62 3.09 2.31 2.83D21 2.82 2.77 2.82 2.46 2.71Mean3.50 2.96 3.39 2.65 3.0987.43% 73.93% 84.79% 66.25% 77.32%• The mean overall level of implementation score for adherence tointervention protocol was 77.32%.• Inter-rater agreement was moderate (κ = 0.60, p<0.001).• Generally, scores for pre-diabetes groups were higher than diabetesgroups for all sessions.• Scores for Session 4 were consistently the lowest for all groups.COMPETENCE• Examples of competence items on the fidelity checklist:o Opportunity for participants to ask questionso Opportunity & encouragement for participant-led group discussion• The overall level of implementation scores ranged from 3.71 (92.75%) to3.93 (98.25%), indicating a high level of competence across all groups.• There was good inter-rater agreement (κ = 0.71, p<0.001).• Facilitators did not achieve full implementation scores for Session 1, mostcommonly due to omission of the ‘confidentiality agreement’ criterion.• Overall, the course satisfaction data showed that participants had highsatisfaction ratings of the LWTC programme, which affirms facilitatorcompetence in intervention delivery.Table 3: Level of implementation scores for competence criteriaGroup ID[Pre-diabetes (P); Diabetes (D)]Level of implementation scores for competence items (on a scale of 0-4) Overall level of implementation scoreSession 1 Session 2 Session 3 Session 4P31 3.40 4.00 4.00 4.00 3.79P32 3.25 4.00 4.00 4.00 3.77P33 3.80 4.00 4.00 4.00 3.93P34 3.60 4.00 4.00 4.00 3.86P35 3.20 4.00 4.00 4.00 3.71D20 3.40 4.00 4.00 4.00 3.79D21 3.40 4.00 4.00 4.00 3.79Mean3.44 4.00 4.00 4.00 3.8185.89% 100% 100% 100% 95.14%

Evaluating the fidelity of implementation of a voluntary sector-led, community-based diabetes prevention and management programme

https://purehost.bath.ac.uk/ws/files/320066746/Kok_et_al_-_SWPH_Scientific_Conference_Poster_2016.pdf

Evaluating the fidelity of implementation of a voluntary sector-led, community-based diabetes prevention and management programme

Abstract

Similar works

Full text

Available Versions

OPUS