Attree

Clark

Department of Health

Ellis

Eraut

Gray

Hardwick

Hutchinson

Jan Draper

Jordan

Liz Clark

Sayer

Stark

Vize

English

Crossref

Impact of continuing professional education on practice: The rhetoric and the reality

Draper, Janet

Clark, Elizabeth

Open Research Online 

Open Research OnlineThe Open University’s repository of research publicationsand other research outputsEvaluating the impact of continuing professionaleducation on healthcare practice: the rhetoric and therealityJournal ItemHow to cite:Draper, Janet and Clark, Elizabeth (2007). Evaluating the impact of continuing professional education onhealthcare practice: the rhetoric and the reality. Nurse Education Today, 27(6) pp. 515–517.For guidance on citations see FAQs.c© [not recorded]Version: [not recorded]Link(s) to article on publisher’s website:http://dx.doi.org/doi:10.1016/j.nedt.2007.07.001Copyright and Moral Rights for the articles on this site are retained by the individual authors and/or other copyrightowners. For more information on Open Research Online’s data policy on reuse of materials please consult the policiespage.oro.open.ac.uk 1 Impact of continuing professional education on practice: the rhetoric and the reality  Despite a significant global investment in continuing professional education (CPE) in nursing and health care (Jordan, 2000) and a lack of empirical evidence of its effectiveness (Clark, 2005; Attree, 2006), it has nevertheless continued to be a focus of UK government lifelong learning strategies. For example, the Department of Health in England asserted that ‘Every aspect of healthcare delivery and strategies for health depends on the education and skills of individual staff. Investment in their learning and personal development is, in a real sense, spending on patients and is essential to the future quality of the health service’ (DoH, 2002, p.7, our emphasis).  However, in the current climate of needs-led education and outcomes-driven health services, not just in the UK but world-wide, those responsible for developing and delivering CPE simply cannot afford to rely on such rhetoric to safeguard the ongoing support for CPE across the health services, however much we would like to do so! It is imperative that we are able to articulate the added value of CPE to direct patient and client care to employers, those commissioning healthcare education as well as potential students. Our own experience in this area (Draper and Clark, 2006; Draper et al., 2007) leads us to believe that the situation internationally may not be significantly different.  It could perhaps be argued that it is already too late in the UK because of the significant cuts in CPE that have already happened in the past year: a 10% cut in training by strategic health authorities (SHAs) in the 2006/07 financial year. Moreover, on 31 May 2007 the Health Service Journal (HSJ) revealed that SHAs are also planning to take more than £117million from the training budget in the current (2007/08) financial year, with some SHAs cutting their budget in order to boost reserves (News item, page 7). In response, the Editor of the HSJ makes the case that ‘training drives up performance and productivity, which translates into higher-quality, more effective and safer care’ (Vize, 2007, p.3). But what evidence can be cited to support such an important assertion?    2 So are we in the midst of a temporary cyclical downturn in post-qualifying education similar to those previously experienced over the last few decades, or is this perhaps the beginning of a paradigm shift? By this we mean that in the light of the dearth of evidence of the benefits of previous investment in CPE, healthcare organisations may no longer be willing to commission the higher education sector to provide it. Instead, they may prefer to use their limited funding to develop and run a range of in-house training courses that are highly responsive to the fast-changing healthcare environment.  Such ‘turbulence’ in nurse education (Stark et al., 2000) raises the stakes even higher, making it imperative, we would argue, that educators can demonstrate evidence of the ‘health return’ on the financial investment in CPE (Sayer and Gray, 2006). This evidence could also be used to inform purchasing decisions and curriculum development (Jordan et al., 1999). A continuing lack of such evidence may deter future investors in the current political and economic climate (Hardwick and Jordan, 2002).   So what are the limitations related to the current evidence? Whilst a number of studies have evaluated CPE activities, these have all tended to be small scale and have focussed on process and the teaching strategies employed, rather than on their direct impact on practice. We know, for example, that research has tended to focus on general issues, such as learner satisfaction, and that patient outcomes and impact on practice are rarely assessed, if at all. Other methodological limitations include a reliance on self-perception reporting (predominantly by the student as the key stakeholder), an emphasis on evaluating small-scale individual programmes of study (and typically confined to one locality), the predominant use of retrospective methods (data are therefore subject to errors in recall and bias), and the tendency to use either positivist or naturalistic approaches. Demonstrating the outcomes of CPE on professional practice therefore remains a largely under-researched field, which also lacks validated research approaches and methods. Together these issues have resulted in a partial and very limited picture of the impact of CPE on practice.    3 Acknowledgement of the complexity of evaluating the effectiveness of education programmes is not new and has been discussed for a number of years (see, for example, Eraut, 1985 and 1994), but remarkably little progress has been made over the past two decades – possibly because of the difficulties involved. Although these challenges are recognised and have been clearly articulated we believe they are insufficient to ‘excuse complacency’ (Hutchinson, 1999, p.1269).  What seems to be needed is a robust and cost-effective approach that will enable us to differentiate between CPE that is fit for purpose and fit for practice and that provides the knowledge and skills that will improve the patient experience and patient outcomes, and that which is not. Those responsible for commissioning CPE need to be confident that they are investing in real improvements in patient care and are able to defend their investment on the basis of robust evidence. Gray (2001) reminds us of the political imperative that ‘as the pressure on resources increases, there will be a transition from opinion-based decision-making to evidence-based decision-making’ (p.11). If we are on the brink of a paradigm shift in terms of the commissioning of CPE, healthcare educators could be left looking on helplessly because we lack the necessary evidence to support the benefits of CPE.  So how, therefore, do we demonstrate that CPE is fit for professional practice? That is to say that it provides the knowledge and skills that will help shape and change things for the better, and that investment in lifelong learning and personal development really does improve the patient experience and is therefore essential to the future quality of the health services.  We feel that the urgent task before us is to develop an approach that enables a multidimensional perspective that is methodologically robust and yet that is also fit for purpose. So, what are the key methodological issues arising from such an initiative?  What are the outcomes?  Where do we start measuring/evaluating? We need first to define meaningful impact-on-practice outcome measures. But how do we do this? Data need to  4 be more than self-report, retrospective data on perceptions, moving towards student experience and clearly defined practice outcomes. Satisfaction ratings tell us very little about the student experience. This shift from concentrating on student satisfaction to focus on the student experience is an important one for educationalists to grasp (similar to recognising the limitations of patient satisfaction data when trying to gain a picture of the patient experience). Given that the majority of post-qualifying programmes are of necessity studied on a part-time basis, what do we really know about the experience of busy healthcare professionals who are studying part-time alongside competing personal and professional demands and commitments? If student experience data were to be routinely gathered by educators then at least we would be able to demonstrate whether a particular CPE programme was fit for purpose. However, in our view, shifting the focus to the student experience is still not enough for those involved in professional education. We need also to shift towards impact on practice. Who identifies these practice outcomes?   Interestingly, in response to criticism about the raiding of training budgets (referred to earlier) which undermines a service-level agreement between SHAs and higher education institutions that was announced in May 2007, a Department of Health spokesperson is quoted as saying: ‘The key issue is the outcomes of money invested in training, rather than how much money is spent for a particular purpose’ (News item, Health Service Journal, 31 May 2007, p.7). Outcomes therefore take centre stage in the debate about the value and impact of CPE.   Who are the key stakeholders?  For us, evaluation of impact on practice needs to include more than just the students. So should employers, commissioners, educators, patients/carers also be involved? And if so, how?  What about methodology? We believe a pluralistic approach is likely to serve us better rather than being committed to one methodological approach. No amount of experimental or quasi-experimental research is ever going to expose a causal relationship between CPE and practice outcomes. Indeed, those studies that have  5 attempted randomised controlled trials – like Ellis et al. (2000) in the UK – have encountered major difficulties with respect to, for example, random sampling and allocation. We have to explore this in a multidimensional way. And just as patient narratives have recently become popular methods of capturing insights into the patient experience, perhaps student narratives are worth further consideration. But when you start talking about different methodologies, then things can become potentially cumbersome and unwieldy. So, methodology has to be both fit for purpose and rigorous.   What about method? Do data collection methods have to be retrospective? And what are the preferred data? This will also depend on the discussion regarding outcomes, but is a mix of qualitative and quantitative data desirable?   What about scale? Is scale important? In the light of some of our preliminary work with key stakeholders, commissioners of education and employers of healthcare professionals seem to require a tool that could be applied to potentially large numbers of the workforce. So, any approach needs also to be scaleable, as well as rigorous.   So where does this leave us? Perhaps with more questions than answers! We recognise that many of the issues are not new and acknowledge that taking forward this work is not straightforward, but would contend that ‘no effort equals no progress!’ We need therefore to rekindle the debate and dialogue in order that some real progress is made over the next five years.  References Attree, M. (2006) Evaluating healthcare education: issues and methods, Nurse Education Today, 26 (8), pp.640–646.  Clark, E. (2005) Commentary on ‘A review of evaluative studies of computer-based learning in nursing education’ by Michael J. Lewis, Rebecca Davies, David Jenkins, Michael I. Tait [Nurse Education Today 21 (1) (2001) pp.26–37], Nurse Education Today, 25 (8), pp.598–600.  Department of Health (2002) Funding Learning and Development for the Healthcare Workforce, London: DoH.  6 Draper, J. and Clark, E. (2006) The impact of continuing professional education on professional practice: a call for urgent action, Paper presented at the First Nurse Education International Conference: Developing collaborative practice in health and social care education, 14–16 May, Vancouver, Canada.  Draper, J. Clark, E. and Attree, M. (2007) Evaluating the impact of healthcare education: approaches and challenges, Symposium at the RCN International Nursing Research Conference, 1-4 May, University of Dundee.  Ellis, L. Davies, S. and Laker, S. (2000) Attempting to set up a randomised controlled trial, Nursing Standard, 14(21), pp.32-36.  Eraut, M. (1985) Knowledge creation and knowledge use in professional contexts, Studies in Higher Education, 10 (2), pp.117–133.  Eraut, M. (1994) Developing Professional Knowledge and Competence, Falmer, London.  Gray, J.A.M. (2001) Evidence-based Healthcare: How to make health policy and management decisions (2nd edn), Churchill Livingstone, Edinburgh.  Hardwick, S. and Jordan, S. (2002) The impact of part-time post-registration degrees on practice, Journal of Advanced Nursing, 38(5), pp.524-535.  Hutchinson, L. (1999) Evaluating and researching the effectiveness of educational interventions, British Medical Journal, 318, pp.1267-1269 Jordan, S. (2000) Educational input and patient outcomes: exploring the gap, Journal of Advanced Nursing, 31 (2), pp.461–471.  Jordan, S. Coleman, M. Hardy, B. and Hughes, D. (1999) Assessing educational effectiveness: the impact of a specialist course on the delivery of care, Journal of Advanced Nursing, 30(4), pp.796-807.  Stark, S. Cooke, P. and Stronach, I. (2000) Minding the gap: some theory-practice disjunctions in nursing education research, Nurse Education Today, 20, pp.155-163.  Sayer, J. and Gray, R. (2006) Randomised controlled trials of the effectiveness of education in health and social care: can they work, Nurse Education Tomorrow Conference, Abstract proceedings pp.46-49, 5-7 September 2006, University of Durham: Durham.  Vize, R. (2007) Comment on Deficit crisis: Ground won for training must be halkd in face of cash battles, Health Service Journal, 31 May, p.3. 

http://oro.open.ac.uk/10826/1/Editorial__June_2007_.pdf

Evaluating the impact of continuing professional education on healthcare practice: the rhetoric and the reality

Evaluating the impact of continuing professional education on healthcare practice: the rhetoric and the reality

Abstract

Similar works

Full text

Available Versions

Crossref

Open Research Online