Waights V. (2009) Assessing learning for healthcare practice using an on-line interactive tool

Abstract

One of the challenges facing educators of healthcare professionals worldwide is to enable students to develop skills and to assess their competence without compromising patient safety (Koh 2002; Ravert 2002). In many instances assessments are undertaken through Objective Structured Clinical Examinations (OSCEs) using simulated patients or actors. However, these can be costly and students may have limited opportunities for practice or formative assessment. Indeed, there is much current debate about assessment for learning within Higher Education and the provision of timely, high quality feedback, which as REAP (2007) notes can take time to generate. Use of eLearning technology can lead to increased opportunities for assessment and many eAssessment assignments in Higher Education use objective question types, in particular multiple choice (Sclater et al. 2007). However, these question types alone may be limited when assessing the application of learning in complex professional situations, such as healthcare practitioners making decisions regarding patients’ health. These clinical decisions require sound theoretical knowledge and good decision-making skills, however, they are also strongly influenced by their context - as the complexity of decision tasks increases so context-specific knowledge determines the effectiveness of decision-making (Botti & Reeve 2003). To enhance skills development through practice-based learning, we have taken an experiential (Kolb 1984) approach within our curriculum and used virtual environments to introduce students to the complexities and challenges of practice, using examples from real practice to ensure validity and mimic reality (Cioffi 2001). This pilot study enhances this approach to develop and assess students’ decision-making skills using a context-driven decision maze tool that draws on their prior learning within the curriculum, is situated in practice and mimics reality. It builds on Laurillard (2002), who suggests that ‘Traditional modes of assessment of knowledge are seen as inadequate because they fail to assess students‘ capability in the authentic activities of their discipline. This approach enables students to engage with the complexity of practice, in which they need to seek information from a variety of sources to inform their decisions, and to rehearse skills before taking them into practice. In addition, through a reflective log of their decision-making, it provides an alternative to mentor-led assessment, which can bring its own problems due to the tension between nurturing and judgement (Yorke 2005). The tool delivers instant high quality feedback so that students can monitor their progress and self-assess their performance to enable improvement. The tool is currently being piloted with volunteer nursing students, who complete an on-line evaluative quantitative and qualitative questionnaire. In addition, each decision is scored and the results amalgamated to provide an objective evaluation of students’ progress. A recent review by Cook et al (2008) of research in eLearning in medical education has shown only 13% of studies undertaken in this area are evaluated, so the completed evaluation will contribute to knowledge in this field. The findings to date (n=11) suggest students’ decision-making skills improve across two attempts of the tool (73%), and that students find the tool valuable for self-assessment (87%) and feel better prepared for practice (87%). The findings also provide evidence to support Boud and Prosser’s (2002) principles for fostering high quality learning in higher education. Use of this tool allows assessment to occur in context, over time and at a distance, and provides an innovative approach to assessing complexity in professional practice. It is student-led, enabling students to engage multiple times if desired, encouraging self-assessment and self-monitoring of progress. Furthermore, it demonstrates that virtual environments are an effective learning tool, which can be tailored to meet the needs of different students in a variety of settings, making them valuable resources in education provision for healthcare professionals worldwide