Evaluation of an online Diabetes Needs Assessment Tool (DNAT) for health professionals: a randomised controlled trial

Background: Continuous medical education is traditionally reliant to a large extent on self-directed learning based on individuals' perceived learning priorities. Evidence suggests that this ability to self-assess is limited, and more so in the least competent. Therefore, it may be of benefit to utilise some form of external assessment for this purpose. Many diabetes educational programmes have been introduced, but few have been assessed for their benefit in a systematic manner. As diabetes is an increasingly prevalent disease, methods for the dissemination and understanding of clinical guidelines need to be explored for their effectiveness. This paper describes the study design of a randomised controlled trial to evaluate the effectiveness of using an interactive online Diabetes Needs Assessment Tool (DNAT), that builds a learning curriculum based on identified knowledge gaps, compared with conventional self-directed learning. The study assesses the effect of these interventions on health professionals' knowledge of diabetes management, evaluates the acceptability of this process of learning and self-reported changes in clinical practice as a result of this novel educational process. Methods: Following a baseline assessment, participants will be randomised to undergo a 4-month learning period where they will either be given access to the diabetes learning modules alone (control group) or a Diabetes Needs Assessment Tool (DNAT) plus the diabetes learning modules (intervention group). On completion of the DNAT, a personalised learning report will be created for each participant identifying needs alongside individualised recommendations of the most appropriate learning modules to meet those requirements. All participants will complete a Diabetes Knowledge Test before and immediately after the allocated learning and the primary outcome will be the state of knowledge at 4 months. Learners will also be surveyed immediately after the learning period to assess the acceptability of the learning formats and the perceived usefulness and usability of the materials. After a further month, all learners will receive a series of questions to evaluate self-reported changes in clinical practice as a result of this educational experience and asked to include specific examples of any changes in their diabetes care practice

Combined student ratings and self-assessment provide useful feedback for clinical teachers

Many evaluation instruments have been developed to provide feedback to physicians on their clinical teaching but written feedback alone is not always effective. We explored whether feedback effectiveness improved when teachers’ self-assessment was added to written feedback based on student ratings. 37 physicians (10 residents, 27 attending physicians) from different specialties (Internal Medicine, Surgery, Obstetrics/Gynecology, Pediatrics, Neurology, Dermatology, Ophthalmology, ENT, and Psychiatry) were invited to fill out a self-assessment questionnaire on their teaching skills. Students completed an almost identical questionnaire to evaluate the same teachers based on their experiences during clerkships. After receiving written feedback incorporating their self-assessment and the student ratings, the teachers indicated their perceptions of the self-assessment exercise and the written feedback in a questionnaire (five-point Likert scale items) and next, in more detail, in semi-structured interviews with a purposive sample of 12 of the participating teachers. 25 physicians participated (67%). The results showed that self-assessment and student feedback were both perceived as useful (3.7, SD 1.0) but the latter was considered more effective. The physicians we interviewed considered the combination of self-assessment with student ratings more effective than either self-assessment or written feedback alone. Notably, discrepancies between student ratings and self-assessment were deemed a strong incentive for change. We conclude that self-assessment can be a useful tool to stimulate improvement of clinical teaching when it is combined with written feedback based on student ratings. Future research among larger groups is needed to confirm our findings and examine whether these combined tools actually lead to improved teaching

Evaluation of an online interactive Diabetes Needs Assessment Tool (DNAT) versus online self-directed learning: a randomised controlled trial

<p>Abstract</p> <p>Background</p> <p>Methods for the dissemination, understanding and implementation of clinical guidelines need to be examined for their effectiveness to help doctors integrate guidelines into practice. The objective of this randomised controlled trial was to evaluate the effectiveness of an interactive online Diabetes Needs Assessment Tool (DNAT) (which constructs an e-learning curriculum based on individually identified knowledge gaps), compared with self-directed e-learning of diabetes guidelines.</p> <p>Methods</p> <p>Health professionals were randomised to a 4-month learning period and either given access to diabetes learning modules alone (control group) or DNAT plus learning modules (intervention group). Participants completed knowledge tests before and after learning (primary outcome), and surveys to assess the acceptability of the learning and changes to clinical practice (secondary outcomes).</p> <p>Results</p> <p>Sixty four percent (677/1054) of participants completed both knowledge tests. The proportion of nurses (5.4%) was too small for meaningful analysis so they were excluded. For the 650 doctors completing both tests, mean (SD) knowledge scores increased from 47.4% (12.6) to 66.8% (11.5) [intervention group (n = 321, 64%)] and 47.3% (12.9) to 67.8% (10.8) [control group (n = 329, 66%)], (ANCOVA p = 0.186). Both groups were satisfied with the usability and usefulness of the learning materials. Seventy seven percent (218/284) of the intervention group reported combining the DNAT with the recommended reading materials was "<it>very useful"/"useful"</it>. The majority in both groups (184/287, 64.1% intervention group and 206/299, 68.9% control group) [95% CI for the difference (-2.8 to 12.4)] reported integrating the learning into their clinical practice.</p> <p>Conclusions</p> <p>Both groups experienced a similar and significant improvement in knowledge. The learning materials were acceptable and participants incorporated the acquired knowledge into practice.</p> <p>Trial registration</p> <p>ISRCTN: <a href="http://www.controlled-trials.com/ISRCTN67215088">ISRCTN67215088</a></p

Validation of a self-efficacy instrument and its relationship to performance of crisis resource management skills

Self-efficacy is thought to be important for resuscitation proficiency in that it influences the development of and access to the associated medical knowledge, procedural skills and crisis resource management (CRM) skills. Since performance assessment of CRM skills is challenging, self-efficacy is often used as a measure of competence in this area. While self-efficacy may influence performance, the true relationship between self-efficacy and performance in this setting has not been delineated. We developed an instrument to measure pediatric residents’ self-efficacy in CRM skills and assessed its content validity, internal structure, and relationship to other variables. After administering the instrument to 125 pediatric residents, critical care fellows and faculty, we performed an exploratory factor analysis within a confirmatory factor analysis as well as a known group comparison. The analyses specified four factors that we defined as: situation awareness, team management, environment management, and decision making. Pediatric residents reported lower self-efficacy than fellows and faculty in each factor. We also examined the correlation between self-efficacy and performance scores for a subset of 30 residents who led video recorded simulated resuscitations and had their performances rated by three observers. We found a significant, positive correlation between residents’ self-efficacy in situation awareness and environment management and their overall performance of CRM skills. Our findings suggest that in a specific context, self-efficacy as a form of self-assessment may be informative with regards to performance