Student self-assessment of operative dentistry experiences : a time dependent exercise in best practice outcomes.

Purpose: The purpose of this project was to evaluate student self-assessment of operative dentistry experiences. Methods: First, a retrospective analysis of student self-assessment and faculty assessment grade sheets were evaluated for mean differences and correlations across time. Both preclinical (D2: n=120) and clinical (D3: n=120; D4: n=120) grades sheets were evaluated. Secondly, twenty-five (n=25) students from each of the D1, D2, D3 and D4 classes were asked to evaluate dentoform work. Twenty-five (n=25) operative calibrated faculty graded the same two dentoforms. Results: D2 student’s self-assessment scores were significantly higher than the faculty assessment scores (t-test; p0.05) and there was a positive correlation of scores (r= 0.408). On the prospective analysis, D1 dental students, D2 dental students and D3 dental students graded significantly higher (ANOVA;

The calibration of student judgement through self-assessment: disruptive effects of assessment patterns

© 2014, © 2014 HERDSA. Can extended opportunities for self-assessment over time help students develop the capacity to make better judgements about their work? Using evidence gathered through students' voluntary self-assessment of their performance with respect to assessment tasks in two different disciplines at two Australian universities, the paper focuses on the effects of sequences of units of study and the use of different types of assessment task (written, oral, analysis, and project) in the development of student judgement. Convergence between student criteria-based gradings of their own performance in units of study and those allocated by tutors was analysed to explore the calibration of students' judgement over time. First, it seeks to replicate analyses from an earlier smaller-scale study to confirm that students' judgements can be calibrated through continuing opportunities for self-assessment and feedback. Second, it extends the analysis to coherently designed sequences of units of study and explores the effects of different types of assessment. It finds that disruptive patterns of assessment within a sequence of subjects can reduce convergence between student and tutor judgements

Overestimation of Skills Affects Drivers’ Adaptation to Task Demands

Inadequate self-assessment, and specifically, overestimation of skill, results in insufficient adaptation to task demands, which can manifest itself on different levels of the driving task. A total of 130 drivers (83 novice and 47 experienced drivers) participated in an on-road driving assessment. Their performance in this assessment (i.e., fail or pass) was compared to the participants’ reported confidence in their driving skills (i.e., high or low confidence), resulting in three calibration groups: a) well-calibrated drivers (reported confidence matched performance on assessment), b) overconfident drivers (high confidence but failed assessment) and c) insecure drivers (low confidence but passed assessment). Furthermore, participants completed a questionnaire which focused on choices made on the strategic and manoeuvring level of the driving task. No significant difference was found between the calibration groups for the strategic level. Overconfident drivers reported significantly more violating behaviour than the well-calibrated and the insecure drivers. At the manoeuvring level, overconfident drivers showed significantly less instances of adaptation to traffic complexity. In conclusion, the current study suggests that overconfidence is related to inadequate adaptation to task demands

Insightful Practice : a robust measure of medical students' professional response to feedback on their performance

Background: Healthcare professionals need to show accountability, responsibility and appropriate response to audit feedback. Assessment of Insightful Practice (engagement, insight and appropriate action for improvement) has been shown to offer a robust system, in general practice, to identify concerns in doctors' response to independent feedback. This study researched the system's utility in medical undergraduates. Methods: Setting and participants: 28 fourth year medical students reflected on their performance feedback. Reflection was supported by a staff coach. Students' portfolios were divided into two groups (n∈=∈14). Group 1 students were assessed by three staff assessors (calibrated using group training) and Group 2 students' portfolios were assessed by three staff assessors (un-calibrated by one-to-one training). Assessments were by blinded web-based exercise and assessors were senior Medical School staff. Design: Case series with mixed qualitative and quantitative methods. A feedback dataset was specified as (1) student-specific End-of-Block Clinical Feedback, (2) other available Medical School assessment data and, (3) an assessment of students' identification of prescribing errors. Analysis and statistical tests: Generalisability G-theory and associated Decision D- studies were used to assess the reliability of the system and a subsequent recommendation on students' suitability to progress training. One-to-one interviews explored participants' experiences. Main outcome measures: The primary outcome measure was inter-rater reliability of assessment of students' Insightful Practice. Secondary outcome measures were the reaction of participants and their self-reported behavioural change. Results: The method offered a feasible and highly reliable global assessment for calibrated assessors, G (inter-rater reliability)∈>∈0.8 (two assessors), but not un-calibrated assessors G∈<∈0.31. Calibrated assessment proved an acceptable basis to enhance feedback and identify concern in professionalism. Students reported increased awareness in teamwork and in the importance of heeding advice. Coaches reported improvement in their feedback skills and commitment to improving the quality of student feedback. Conclusions: Insightful practice offers a reliable and feasible method to evaluate medical undergraduates' professional response to their training feedback. The piloted system offers a method to assist the early identification of students at risk and monitor, where required, the remediation of students to get their level(s) of professional response to feedback back 'on track'.Publisher PDFPeer reviewe

Insightful Practice:a robust measure of medical students’ professional response to feedback on their performance

Background:&nbsp; Healthcare professionals need to show accountability, responsibility and appropriate response to audit feedback. Assessment of Insightful Practice (engagement, insight and appropriate action for improvement) has been shown to offer a robust system, in general practice, to identify concerns in doctors' response to independent feedback. This study researched the system's utility in medical undergraduates.&nbsp; Methods:&nbsp; Setting and participants: 28 fourth year medical students reflected on their performance feedback. Reflection was supported by a staff coach. Students' portfolios were divided into two groups (n&isin;=&isin;14). Group 1 students were assessed by three staff assessors (calibrated using group training) and Group 2 students' portfolios were assessed by three staff assessors (un-calibrated by one-to-one training). Assessments were by blinded web-based exercise and assessors were senior Medical School staff.&nbsp; Design: Case series with mixed qualitative and quantitative methods. A feedback dataset was specified as (1) student-specific End-of-Block Clinical Feedback, (2) other available Medical School assessment data and, (3) an assessment of students' identification of prescribing errors.&nbsp; Analysis and statistical tests: Generalisability G-theory and associated Decision D- studies were used to assess the reliability of the system and a subsequent recommendation on students' suitability to progress training. One-to-one interviews explored participants' experiences.&nbsp; Main outcome measures: The primary outcome measure was inter-rater reliability of assessment of students' Insightful Practice. Secondary outcome measures were the reaction of participants and their self-reported behavioural change.&nbsp; Results:&nbsp; The method offered a feasible and highly reliable global assessment for calibrated assessors, G (inter-rater reliability)&isin;&gt;&isin;0.8 (two assessors), but not un-calibrated assessors G&isin;&lt;&isin;0.31. Calibrated assessment proved an acceptable basis to enhance feedback and identify concern in professionalism. Students reported increased awareness in teamwork and in the importance of heeding advice. Coaches reported improvement in their feedback skills and commitment to improving the quality of student feedback. Conclusions: Insightful practice offers a reliable and feasible method to evaluate medical undergraduates' professional response to their training feedback. The piloted system offers a method to assist the early identification of students at risk and monitor, where required, the remediation of students to get their level(s) of professional response to feedback back 'on track'. &copy; 2015 Murphy et al

Statistical Aspects of the Use of Biomarkers in Nutritional Epidemiology Research

Few strong and consistent associations have arisen from observational studies of dietary consumption in relation to chronic disease risk. Measurement error in self-reported dietary assessment may be obscuring many such associations. Attempts to correct for measurement error have mostly used a second self-report assessment in a subset of a study cohort to calibrate the self-report assessment used throughout the cohort, under the dubious assumption of uncorrelated measurement errors between the two assessments. The use, instead, of objective biomarkers of nutrient consumption to produce calibrated consumption estimates provides a promising approach to enhance study reliability. As summarized here, we have recently applied this nutrient biomarker approach to examine energy, protein, and percent of energy from protein, in relation to disease incidence in Women’s Health Initiative cohorts, and find strong associations that are not evident without biomarker calibration. A major bottleneck for the broader use of a biomarker-calibration approach is the rather few nutrients for which a suitable biomarker has been developed. Some methodologic approaches to the development of additional pertinent biomarkers, including the possible use of a respiratory quotient from indirect calorimetry for macronutrient biomarker development, and the potential of human feeding studies for the evaluation of a range of urine- and blood-based potential biomarkers, will briefly be described

A looming revolution: Implications of self-generation for the risk exposure of retailers. ESRI WP597, September 2018

Managing the risk associated with uncertain load has always been a challenge for retailers in electricity markets. Yet the load variability has been largely predictable in the past, especially when aggregating a large number of consumers. In contrast, the increasing penetration of unpredictable, small-scale electricity generation by consumers, i.e. self-generation, constitutes a new and yet greater volume risk. Using value-at-risk metrics and Monte Carlo simulations based on German historical loads and prices, the contribution of decentralized solar PV self-generation to retailers’ load and revenue risks is assessed. This analysis has implications for the consumers’ welfare and the overall efficiency of electricity markets

Developmental profile and diagnoses in children presenting with motor stereotypies

Introduction: Motor stereotypies represent a typical example of the difficulty in distinguishing non-clinical behaviors (physiological and transient) from symptoms or among different disorders (“primary stereotypies”, associated with Autistic Spectrum Disorder, Intellectual Disabilities, genetic syndromes, sensory impairment). Aim of this study was to get an accurate analysis on the relationship between stereotypies and neurodevelopmental disorders. Methods: We studied 23 children (3 girls) aged 36 to 95 months, who requested a consultation due to the persistence or the increase severity of motor stereotypies. None of patients had a previous diagnosis of ASD. The assessment included the Motor Severity Stereotypy Scale (MSSS), the Repetitive Behavior Scale-Revised (RBS-R), the Raven’s Colored Progressive Matrices (CPM), the Child Behavior Checklist for ages 1 ½ -5 or 4-18 (CBCL), the Social Responsiveness Scale (SRS) and the Autism Diagnostic Observation Schedule- Second edition (ADOS 2). Results: All patients were showing motor stereotypies for periods of time varying from 6 to 77 months. The MSSS showed each child had a limited number of stereotypies; their frequency and intensity were mild; the interference of stereotypies was variable; the impairment in the daily life was mild. The RBS-R scores resulted positive for the subscale of “Stereotypic behaviors” in all children; moreover, several children presented other repetitive behaviors, mainly “Ritualistic behavior” and “Sameness behavior”. All patients showed a normal cognitive level. The CBCL evidenced behavioral problems in 22% of the children: Internalizing problems, Attention and Withdrawn were the main complaints. On the SRS, all but one of the tested patients obtained clinical scores in the clinical range at least in one area. On the ADOS 2, four patients obtained scores indicating a moderate level of ASD symptoms, four had a mild level and fifteen showed no or minimal signs of ASD. Discussion: Motor stereotypies in children with normal cognitive level represent a challenging diagnostic issue for which a finely tailored assessment is mandatory in order to define a precise developmental profile. Notably, a careful and cautious use of standardized tests is warranted to avoid misdiagnosis. Furthermore, it is hard to consider motor stereotypies, even the primary ones, exclusively as a movement disorder