Professional learning, organisational change and clinical leadership development outcomes.

INTRODUCTION: The aim of this study is to develop a conceptually sound outcome model for clinical leadership (CL) development in healthcare, linking individual professional learning and organisational change. Frontline doctors' CL is often offered as a solution to healthcare challenges worldwide. However, there is a paucity of rigorous evidence of effectiveness of CL development, or theories supporting it. Importantly, the literature currently lacks robust outcome models for CL development, impeding robust impact evaluations. METHODS: This multi-source, sequential integrated mixed-methods study draws on systematic content analysis of NHS policy documents and empirical data from a CL programme evaluation study: exploratory factor analysis (EFA) of 142 participants' survey responses and thematic qualitative analysis of 30 in-depth participant interviews across six cohorts. Through integrating findings from the three analyses we examine: (a) the expected organisational outcomes of CL, (b) individual learning outcomes of CL development, and (c) the mechanisms linking the two. RESULTS: The policy analysis identified three desired solutions to key healthcare problems which CL is expected to offer: Speeding up good practice, Inter-professional collaboration and dialogue, and Change and transformation. Triangulating the EFA results with the qualitative analysis produced five individual outcome constructs: Self-efficacy, Engaging stakeholders, Agency, Boundary-crossing expertise, and Willingness to take risks and to learn from risks and failures. Further qualitative analysis uncovered key mechanisms linking the individual outcomes with the desired organisational changes. DISCUSSION: Despite significant investments into CL development in the UK and worldwide, the absence of conceptually robust and operationally specific outcome models linking individual and organisational impact impedes rigorous evaluations of programme effectiveness. Our study developed a novel individual and organisational outcome model including a theory of change for clinical leadership. Our findings further contribute to professional learning theory in medical settings by conceptualising and operationalising the mechanisms operating between individual and organisational learning outcomes.The authors received a grant from the Cambridge University Health Partners (CUHP) who run the Chief Residents Leadership and Management programme which covered the empirical part of the work reported here

Measuring students’ self-regulated learning in professional education: bridging the gap between event and aptitude measurements

Self-regulated learning has benefits for students’ academic performance in school, but also for expertise development during their professional career. This study examined the validity of an instrument to measure student teachers’ regulation of their learning to teach across multiple and different kinds of learning events in the context of a postgraduate professional teacher education programme. Based on an analysis of the literature, we developed a log with structured questions that could be used as a multiple-event instrument to determine the quality of student teachers’ regulation of learning by combining data from multiple learning experiences. The findings showed that this structured version of the instrument measured student teachers’ regulation of their learning in a valid and reliable way. Furthermore, with the aid of the Structured Learning Report individual differences in student teachers’ regulation of learning could be discerned. Together the findings indicate that a multiple-event instrument can be used to measure regulation of learning in multiple contexts for various learning experiences at the same time, without the necessity of relying on students’ ability to rate themselves across all these different experiences. In this way, this instrument can make an important contribution to bridging the gap between two dominant approaches to measure SRL, the traditional aptitude and event measurement approach

Analyzing student-teacher interactions in challenge-based learning

Challenge-based learning (CBL) exposes students to the complexities of openended and real-life challenges and encourages them to be in the lead of their learning. The role of teachers remains important but shifts from being the expert to the role of a coach who gradually scaffolds students into becoming independent learners. Accordingly, the interplay between teachers' and students' regulation of teaching and learning can result in friction and influence students' learning experience. This study explores incidents of constructive or destructive friction between student and teacher regulation during a 9-week CBL course for first-year engineering students. Thematic analysis is employed to identify critical incidents of friction during students' learning via analyzing students' weekly learning portfolios. Results suggest that students' experience in CBL is not linear, and there is a constant interplay between students' ability to regulate their learning and teachers' scaffolding. Initial exposure to CBL was characterized by friction in student and teacher interactions. Several students increased their self-regulated learning skills by resolving the initial friction by adopting a more proactive approach to their learning by actively asking questions and feedback from their teachers. The findings of this study are particularly relevant for CBL, where much attention is paid to students' autonomy, self-directedness, and collaboration. Building on the insights of this research, we make recommendations for further research and educational practice

Conceptualizing Socially Shared Regulation In Challenge-Based Learning

Students in Challenge-based learning (CBL) courses work in multidisciplinary groups to develop a solution to an open-ended and ill-defined challenge.Thus, in CBL, students need to regulate their learning individually and collectively to learn. Socially shared regulation of learning (SSRL) refers to the development of collective and co-constructed task perceptions or shared goals by multiple students working as a group. Existing knowledge about conceptualizing and researching SSRL in CBL is currently lacking. In this paper, we provide evidence from a qualitative study we conducted in a CBL course, using analysis of individual learning portfolios and in-depth interviews about students’ perceptions of SRRL. We discuss, firstly, which individual characteristics students perceive as important for SSRL. Secondly, we discuss the identified processes of SSRL identified in our data. Finally, we discuss how groups with high and low SSRL differ. For example, groups with high SSRL spend more time in task planning and role division. They also discussed shared goals early in the process and frequently monitored and evaluated their collective work and progress. On the other hand, groups with low SSRL need guidance individually and as a group to plan and evaluate their activities in different project stages. In addition, they had fewer conversations as a group about their shared goals, and they had more difficulties getting along at a social level. Finally, theoretical implications, practical recommendations, and future directions for research are discussed

Exploring students' perceptions on the use of significant event analysis, as part of a portfolio assessment process in general practice, as a tool for learning how to use reflection in learning

BACKGROUND: Portfolio learning enables students to collect evidence of their learning. Component tasks making up a portfolio can be devised that relate directly to intended learning outcomes. Reflective tasks can stimulate students to recognise their own learning needs. Assessment of portfolios using a rating scale relating to intended learning outcomes offers high content validity. This study evaluated a reflective portfolio used during a final-year attachment in general practice (family medicine). Students were asked to evaluate the portfolio (which used significant event analysis as a basis for reflection) as a learning tool. The validity and reliability of the portfolio as an assessment tool were also measured. METHODS: 81 final-year medical students completed reflective significant event analyses as part of a portfolio created during a three-week attachment (clerkship) in general practice (family medicine). As well as two reflective significant event analyses each portfolio contained an audit and a health needs assessment. Portfolios were marked three times; by the student's GP teacher, the course organiser and by another teacher in the university department of general practice. Inter-rater reliability between pairs of markers was calculated. A questionnaire enabled the students' experience of portfolio learning to be determined. RESULTS: Benefits to learning from reflective learning were limited. Students said that they thought more about the patients they wrote up in significant event analyses but information as to the nature and effect of this was not forthcoming. Moderate inter-rater reliability (Spearman's Rho .65) was found between pairs of departmental raters dealing with larger numbers (20 – 60) of portfolios. Inter-rater reliability of marking involving GP tutors who only marked 1 – 3 portfolios was very low. Students rated highly their mentoring relationship with their GP teacher but found the portfolio tasks time-consuming. CONCLUSION: The inter-rater reliability observed in this study should be viewed alongside the high validity afforded by the authenticity of the learning tasks (compared with a sample of a student's learning taken by an exam question). Validity is enhanced by the rating scale which directly connects the grade given with intended learning outcomes. The moderate inter-rater reliability may be increased if a portfolio is completed over a longer period of time and contains more component pieces of work. The questionnaire used in this study only accessed limited information about the effect of reflection on students' learning. Qualitative methods of evaluation would determine the students experience in greater depth. It would be useful to evaluate the effects of reflective learning after students have had more time to get used to this unfamiliar method of learning and to overcome any problems in understanding the task

Evidence-Based Practice in Teacher Education: The Mediating Role of Self-Efficacy Beliefs and Practical Knowledge

European educational reforms call for the implementation of evidence-based teaching (EBT) in universities. Based on the evidence-based research paradigm in medical education, this study investigates the relationship between teacher educators' research experience, practical knowledge, self-efficacy beliefs, and frequency of EBT implementation. We report on survey data from N = 243 teacher educators from Germany, Austria, Switzerland, and the United Kingdom. A set of mediation analyses were run to identify the mediating role of self-efficacy beliefs and practical knowledge in the interplay among teacher educators' research experience and frequency of research evidence implementation. The results indicate that self-efficacy beliefs are a strong predictor of how frequently teacher educators implement EBT. Implications about the role of self-efficacy beliefs in teacher educators' professional learning and development along with future steps that are necessary to increase the implementation of EBT practices in teacher education will be discussed

How do students regulate their learning in challenge-based learning?:An analysis of students’ learning portfolios

Self-regulated learning (SRL) is one the key pedagogical principles of Challenge-based Learning (CBL) in engineering curricula. Students in CBL have the primary responsibility for planning, implementing, and evaluating their effort and progress. This study explores the use of learning portfolios as a pedagogical tool aimed to document students’ SRL in a CBL course for 1st year engineering students. The research question was: How is SRL documented in a personal learning portfolio during a CBL course? Students were expected to work for 9 weeks with a group of peers on an open-ended challenge. Students were asked to complete a learning portfolio at 3 moments. In week 1, they were asked to set, individually, 5 disciplinary and 5 professional goals they wanted to achieve and in week 5 and 9 they were encouraged to reflect on the progress and attainment of those goals. Twelve students’ learning portfolios were included for analysis in this study. Content analysis of the learning portfolios revealed that students in week 1, described goal setting and in week 5 described SRL processes such as monitoring and self- evaluation while in the final submission in week 9, students reflected on the attainment of their individual goals and the overall success of their project, revealing a need for balancing their own disciplinary and professional goals and the overall goals of group they were members of. The study suggests that learning portfolios provide a useful instrument to encourage SRL in CBL. Limitations and implications for education and research are discussed