The one minute mentor : a pilot study assessing medical students’ and residents’ professional behaviours through recordings of clinical preceptors’ immediate feedback

Introduction: The assessment of professional development and behaviour is an important issue in the training of medical students and physicians. Several methods have been developed for doing so. What is still needed is a method that combines assessment of actual behaviour in the workplace with timely feedback to learners. Goal: We describe the development, piloting and evaluation of a method for assessing professional behaviour using digital audio recordings of clinical supervisors’ brief feedback. We evaluate the inter-rater reliability, acceptability and feasibility of this approach. Methods: Six medical students in Year 5 and three GP registrars (residents) took part in this pilot project. Each had a personal digital assistant (PDA) and approached their clinical supervisors to give approximately one minute of verbal feedback on professionalism-related behaviours they had observed in the registrar’s clinical encounters. The comments, both in transcribed text format and audio, were scored by five evaluators for competence (the learner’s performance) and confidence (how confident the evaluator was that the comment clearly described an observed behaviour or attribute that was relevant). Students and evaluators were surveyed for feedback on the process. Results: Study evaluators rated 29 comments from supervisors in text and audio format. There was good inter-rater reliability (Cronbach α around 0.8) on competence scores. There was good agreement (paired t-test) between scores across supervisors for assessments of comments in both written and audio formats. Students found the method helpful in providing feedback on professionalism. Evaluators liked having a relatively objective approach for judging behaviours and attributes but found scoring audio comments to be time-consuming. Discussion: This method of assessing learners’ professional behaviour shows potential for providing both formative and summative assessment in a way that is feasible and acceptable to students and evaluators. Initial data shows good reliability but to be valid, training of clinical supervisors is necessary to help them provide useful comments based on defined behaviours and attributes of students. In addition, the validity of the scoring method remains to be confirmed

Is Content Really King? An Objective Analysis of the Public's Response to Medical Videos on YouTube

Medical educators and patients are turning to YouTube to teach and learn about medical conditions. These videos are from authors whose credibility cannot be verified & are not peer reviewed. As a result, studies that have analyzed the educational content of YouTube have reported dismal results. These studies have been unable to exclude videos created by questionable sources and for non-educational purposes. We hypothesize that medical education YouTube videos, authored by credible sources, are of high educational value and appropriately suited to educate the public. Credible videos about cardiovascular diseases were identified using the Mayo Clinic's Center for Social Media Health network. Content in each video was assessed by the presence/absence of 7 factors. Each video was also evaluated for understandability using the Suitability Assessment of Materials (SAM). User engagement measurements were obtained for each video. A total of 607 videos (35 hours) were analyzed. Half of all videos contained 3 educational factors: treatment, screening, or prevention. There was no difference between the number of educational factors present & any user engagement measurement (p NS). SAM scores were higher in videos whose content discussed more educational factors (p<0.0001). However, none of the user engagement measurements correlated with higher SAM scores. Videos with greater educational content are more suitable for patient education but unable to engage users more than lower quality videos. It is unclear if the notion “content is king� applies to medical videos authored by credible organizations for the purposes of patient education on YouTube

Employment Services Utilization and Outcomes among Substance Abusing Offenders Participating in California’s Proposition 36 Drug Treatment Initiative

California drug treatment programs may use funds to address barriers to work faced by Proposition 36 offenders, most of whom are not working at treatment entry, but employment services utilization and related behavioral outcomes have never been studied. This study examined primary data collected on 1,453 offenders by 30 programs during 2004 to explore the characteristics, employment services utilization, and outcomes of those who did and did not receive employment services while in drug treatment. One-year outcomes were mostly similar across groups, however, increases in the proportion of offenders employed, receiving income from employment and family or friends, and being paid for work were significantly greater among the received-employment-services group, and a greater proportion of this group also completed drug treatment. Employment services utilization was less likely for persons recruited from outpatient settings and more likely with greater severity of family/social problems and desire for services. Odds of employment one-year post-treatment entry were higher for those of Hispanic race/ethnicity (vs. White) and for those with treatment completion/longer retention but lower for those who were older, lived in specific counties, had greater employment problem severity at intake, and received other income-related services. Strategies for improving employment services utilization and outcomes among Proposition 36 offenders are discussed