The Cheshire and Merseyside Public Health Network conducts a programme of continuing professional development through the year. The Centre for Public Health Research was commissioned to carry out an evaluation of the 2009 continuing professional development programme.Cheshire and Merseyside Public Health Networ

McKay, Jane

Thurston, Miranda

English

ChesterRep

       This work has been submitted to ChesterRep – the University of Chester’s online research repository  http://chesterrep.openrepository.com    Author(s): Jane McKay; Miranda Thurston    Title: Evaluation of the Cheshire and Merseyside Public Health Network Continuing Professional Development Programme 2009   Date: July 2010   Originally published in:    Example citation: McKay. J., & Thurston, M. (2010).  Evaluation of the Cheshire and Merseyside Public Health Network Continuing Professional Development Programme 2009. Chester: University of Chester    Version of item: Published version    Available at: http://hdl.handle.net/10034/118267         Evaluation of the Cheshire and Merseyside  Public Health Network  Continuing Professional Development Programme 2009 Summary      Jane McKay Miranda Thurston       July 2010  Summary  Background to the research The Cheshire and Merseyside Public Health Network (ChaMPs) was launched in 2003, and currently covers a population of 2.4 million people in a mix of urban and rural economies through eight primary care trusts. Five of the eight trusts are part of spearhead communities facing some of the most difficult health challenges.   ChaMPs conducts a programme of continuing professional development (CPD) throughout the year in order to provide structured, targeted and timely CPD events. As indicated in its Business Plan 2009-2010, CPD remains a cornerstone of the ChaMPs commitment to sharing knowledge and expertise. Its programme is therefore designed to enhance its four core functions:  to maximise the sharing of expertise and use of specialist knowledge while minimising duplication of effort;  to capitalise on members’ knowledge and experience to influence health improvements and tackle inequalities;  to develop innovative approaches and enhance the public health evidence base;  to enable public health specialists to meet their accreditation needs through a continuing professional development programme, and providing a forum for peer support.  The CPD programme has been externally evaluated twice, once in 2003 and again in 2005 (Samuels & Thurston, 2005). Evidence from the 2005 evaluation found that levels of satisfaction with the CPD programme were high overall, and that the programme provided for the majority of needs for public health network members. However there were some persisting concerns around increasing reflective practice, developing a culture of research and development and improving capacity building.  The Centre for Public Health Research (CPHR) at the University of Chester was commissioned by the Cheshire and Merseyside Public Health Network to carry out an evaluation of its 2009 CPD programme. The overall aim of the evaluation was to assess the overall quality of the CPD programme, in relation to the extent to which the programme has met its core functions as described above.  Methodology The evaluation used a combination of quantitative (secondary) and qualitative (primary) data.  For the quantitative stage, data were drawn from three existing sources held by the ChaMPs Public Health Network. First, a database was provided which detailed the number of network members at that point in time, and included the total number of events that each had attended in the previous 12 months. Second, delegate lists were provided for each of the 11 events held in 2009. Third, completed evaluation forms from each CPD event were examined and summarised in order to provide an overview of attendees’ views of events and identify any themes from their comments. Responses to open ended questions on evaluation forms were incorporated into the overall analysis of qualitative data.  For the qualitative stage, 25 semi-structured telephone interviews were conducted with participants selected across a range of organisations and levels of seniority from the ChaMPs network database. Seven interviews were from Merseyside PCTs, four were from Cheshire PCTs and four were from borough councils. Five interviews were conducted with delegates from other Northwest PCTs, three with voluntary organisations and two with delegates from statutory services. Overall, there were seven male and eighteen female participants. Data were analysed by coding interview transcripts and identifying key themes and features that were then organised according to the aims and objectives of the research.  Key findings Attendance Based on the numbers of attendees recorded on the evaluation summaries, the eleven events that were held in 2009 were attended by a total of 733 delegates, from a total number of 853 pre-registered delegates. In addition, the number of completed evaluations for each event was also examined. Six of the eleven events had evaluations completed by more than 50% of those attending, although there is quite a lot of variation event by event. In 2005 there were 437 delegates attending 10 events which means there has been an increase of 68% in attendance across all events. Staff from core public health organisations made up the largest proportion of attendees.     Evaluation Overall, the findings from the quantitative element of the study demonstrate that a large majority of delegates who completed evaluation forms were positive about the content of the programme, and its impact. Some events were perceived to be of greater value overall than others, particularly the March event with regard to whether or not delegates had learned from the event, or whether it was felt to be stimulating. In terms of whether the events would cause delegates to change or modify their behaviour, the October event seemed to be the least likely to do so.  Awareness For those participants who had attended CPD events, awareness of the network was high, and most prevalent amongst those interviewees working in core public health roles. Awareness of the Network was also high amongst those who had not attended events in 2009, although there were some for whom awareness was lower.  Deciding to attend The decision to attend events was influenced by a range of factors, such as: specific relevance to current role; relevance to public health role in a wider context (for example location or topic); extent to which the event increases knowledge; how well known a speaker is (their expertise and prominence in the field); general requirement of role to attend CPD events; and personal improvement and fulfilment. The mains reasons that were given for non-attendance by participants, other than a lack of awareness of the programme were: a lack of relevance of the event to the individual and limited time and conflicting priorities.  Programme content Opinions about the overall content of the CPD programme were positive. The programme overall was seen to be of a high quality in terms of the range of topics included, the relevance to national and regional policy, and the general organisation and structure of the events. Criticisms were focused on specific speakers, for example, (although this was minimal) or the use of over technical language in some presentations.  Programme format There were very positive comments about the use of small group format to encourage informal discussions amongst delegates, and to enable more in-depth exploration of some of the key issues raised during the events. There was some concern raised, however, about how effectively this could be achieved in time-limited sessions, and several participants mentioned difficulties in covering the topic as fully as might be liked in the time allowed. There was a concern that where delegates were being asked to contribute to discussion groups, the discussion leaders may be pushing for comments in order to get a clear outcome from the group, which might force some delegates to speak who were uncomfortable doing so.  The use of technology With regard to the use of technology in the CPD programme, a number of participants interviewed said that they rarely used the website, and several said that they had never accessed it. Views on more extensive use of technology to deliver the programme were mixed. Most participants thought there was potential to increase the use of technology (particularly live streaming or podcasting) across all aspects of healthcare delivery and within the public health arena. This was also discussed by some participants in relation to individual travel costs and reducing carbon footprint, so personal efficiency was a factor for some as well as their contribution to corporate efficiencies.   There were however a number of reservations expressed by participants in terms of the implementation of such methods. It was suggested that if programme events were available online, there may be reluctance from staff to make an effort to attend the event in person. If the incentive to attend in person was diminished, individuals may be less likely to dedicate enough quality time in their working day to access online material. For some, a lack of confidence in using information technology was seen to be a barrier to more extensive use, whilst others suggested that pressures of the workplace on a day-to-day basis would act as a barrier to wider use. Another concern was that one of the key aspects of such events was the ability to network with public health colleagues and other individuals from associated professions on pertinent issues, something which again would be undermined by remote access.  Networking Whilst participants did not mention networking as a key determinant in their decision to attend the events, it was one of the main benefits perceived as a result of taking part in the events. For some, however, there was a conflict in terms of finding what was seen to be ‘extra’ time in the day to combine the more formal aspects of the event (the presentations) with the opportunities to network with colleagues.   Dissemination  By far the most effective means of transferring what participants had learned from the events to the workplace was stated as being through formal dissemination and cascading information to other colleagues after the event. This made the additional materials provided by ChaMPs extremely useful in order to be able to structure such dissemination.   Conclusions The benefits of meeting up with other network members extended to their ability to expand their knowledge and develop new ways of thinking about key issues through discussion and debate. For some participants, particularly those in non-core roles, networking allowed them to meet many people across a range of disciplines that they would not normally meet. The benefits in terms of increasing subject specific knowledge were most keenly recognised amongst more junior staff, or those who were relatively new in post, who identified how attendance at events relating to key areas within their remit allowed them to ‘fast track’ their knowledge. Participants were keen to acknowledge that the events provided a platform for generating areas for debate and discussion, and although at times the opportunities within specific programme schedules might not allow sufficient time on the day to explore in detail any areas of new thinking, these could be carried forward into the workplace afterwards. Participants from non-clinical and non-core roles suggested that they were more likely to consider the opportunities for development of research than those in core clinical roles.  Those participants who expressed the strongest views on being able to engage in reflective discussion were those in clinical and core roles, and engaging with a wide range of professionals was seen as fulfilling both professionally and personally.   For one or two participants at a reasonably senior level there were reservations about the extent to which public health needs at a local level could be addressed, when wider national priorities were seen to dominate the agenda. This highlights the perceptions of a top-down approach to both knowledge dissemination and CPD. It also raised concerns amongst some participants over how effectively network members can identify and communicate local priorities through the CPD programme, which was seen to be of importance in terms of being able to make the programme more relevant to the needs of staff in a local context. Therefore for some participants, engaging in a two-way dialogue between programme leaders and delegates would provide a forum for those network members.  There did not appear to be a clear sense of the impact of the CPD programme on developing a research culture. One or two participants who were at a more senior level suggested that the programme overall should be seen as an integral element of the overall enhancement of a research culture, but for most participants, the question raised the issue of research and development as an aspect of CPD for the first time.  Recommendations Programme content  Recommendations on the content of specific events focus on improving the time allowed for delegates to take part in more informal discussion after formal presentations have been delivered, and the provision of more detailed written materials in support of the events.  It is recommended that any events held over a full day, and where possible those held over a half day, have sufficient time set aside during the programme for delegates to discuss and debate information from each presentation. This would provide benefits as follows:   greater depth of understanding and absorption of key messages to enhance professional practice;  more effective dissemination of the programme to other network colleagues increasing overall reach of programme.  It is recommended that more detailed aims and objectives are circulated in advance of each event that provide clear guidance to delegates, and detailed notes and/or learning documents are made available afterwards.   These would improve the quality of information disseminated after the event, and therefore potentially more delegates would be likely to engage with the programme at future events.   They would facilitate better understanding of the overall aims of specific events and would make the process of evaluation and feedback more effective and provide more meaningful data, which would assist in the longer term planning of the programme.  Programme administration With regard to the administration of the programme, a number of issues merit attention that focus on the current use of IT and the potential for increasing the use of technology in support of future programme delivery. It is suggested that the current website requires some modification to make it more accessible, easier to navigate and more central to the delivery of programme information for network members.  Existing links via the e-bulletin are not obvious and network members would benefit from having a clearer sense of how programme materials located on the site can be accessed.  Raising awareness of the website more generally amongst network members might improve the extent of usage.  Improvements in the method of data collection for evaluations, and processing of data are recommended.   Inconsistencies in data collection make accurate evaluation problematic. Using Excel more routinely for the collection and processing of data would remove inaccuracies and inconsistencies and enable evaluation data to be processed quickly by ChaMPs internal staff.  It is recommended that ongoing and regular management of the network database is required. (It is understood that this is in process currently).   This would enable better targeting of events according to their relevance.  Where the potential use of IT in programme delivery is to be considered, for example podcasts/streamed events, we would recommend that this is only as a supplement to the actual events, and made available after the event has taken place.  Future evaluation The findings of this research show that participants did not demonstrate a strong desire to change from the existing paper-based method of evaluation, indeed it was suggested that the uptake of evaluation would diminish if an online method were to be introduced in place of the current format. Therefore it is recommended that continuing the existing method of on-site paper-based evaluation should continue, with training for key network staff to improve Excel skills and overall data management. It is recommended that evaluation is managed by a dedicated core staff, who are trained appropriately and have a clearly defined evaluation structure and procedure to follow.  In addition it is recommended that the introduction of online evaluation be carried out alongside the existing paper-based method. This could be set up and hosted on an online data management and research site. Such a site would allow the existing evaluation form to be uploaded and accessed by all network members via a secure log-in and password system. When evaluation data is required, an automatic email prompt could be sent out. The site would operate under the usual Data Protection and Research protocols. Data that is collected could be exported to existing analysis software (for example to Excel) and analysed in the usual way, by ChaMPs staff.    

Evaluation of the Cheshire and Merseyside Public Health Network Continuing Professional Development Programme 2009

https://chesterrep.openrepository.com/bitstream/10034/118267/8/106%20ChaMPs%20CPD%20Evaluation%20-%20ReportSummary.pdf

Evaluation of the Cheshire and Merseyside Public Health Network Continuing Professional Development Programme 2009

Abstract

Similar works

Full text

Available Versions

ChesterRep