Discrepancy between instructor and student evaluations of instruction: Effect on instructor

To find out whether a discrepancy between the instructor's and the student's evaluations of teaching influence teaching, 13 introductory and educational psychology instructors and their students were given a Student Opinion Questionnaire (SOQ) twice: on the fourth week of a fall term and eight weeks after feedback sessions with the instructors. The instructors received feedback on the direction and amount of initial discrepancy. The results showed that the unfavorable discrepant instructors (instructor rating better than students) changed more on skill, feedback, rapport, general teaching ability, and the overall value of the course than the favorably discrepant instructors (student ratings better than instructor). The unfavorably discrepant instructors improved their teaching significantly more than the favorably discrepant instructors.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/43863/1/11251_2004_Article_BF00120231.pd

Faculty evaluation: Reliability of peer assessments of research, teaching, and service

In this paper, assessments of faculty performance for the determination of salary increases are analyzed to estimate interrater reliability. Using the independent ratings by six elected members of the faculty, correlations between the ratings are calculated and estimates of the reliability of the composite (group) ratings are generated. Average intercorrelations are found to range from 0.603 for teaching, to 0.850 for research. The average intercorrelation for the overall faculty ratings is 0.794. Using these correlations, the reliability of the six-person group (the composite reliability) is estimated to be over 0.900 for each of the three areas and 0.959 for the overall faculty rating. Furthermore, little correlation is found between the ratings of performance levels of individual faculty members in the three areas of research, teaching, and service. The high intercorrelations and, consequently, the high composite reliabilities suggest that a reduction in the number of raters would have relatively small effects on reliability. The findings are discussed in terms of their relationship to issues of validity as well as to other questions of faculty assessment.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/43600/1/11162_2004_Article_BF00991934.pd

Factors confounding the assessment of reflection: a critical review

BACKGROUND: Reflection on experience is an increasingly critical part of professional development and lifelong learning. There is, however, continuing uncertainty about how best to put principle into practice, particularly as regards assessment. This article explores those uncertainties in order to find practical ways of assessing reflection. DISCUSSION: We critically review four problems: 1. Inconsistent definitions of reflection; 2. Lack of standards to determine (in)adequate reflection; 3. Factors that complicate assessment; 4. Internal and external contextual factors affecting the assessment of reflection. SUMMARY: To address the problem of inconsistency, we identified processes that were common to a number of widely quoted theories and synthesised a model, which yielded six indicators that could be used in assessment instruments. We arrived at the conclusion that, until further progress has been made in defining standards, assessment must depend on developing and communicating local consensus between stakeholders (students, practitioners, teachers, supervisors, curriculum developers) about what is expected in exercises and formal tests. Major factors that complicate assessment are the subjective nature of reflection's content and the dependency on descriptions by persons being assessed about their reflection process, without any objective means of verification. To counter these validity threats, we suggest that assessment should focus on generic process skills rather than the subjective content of reflection and where possible to consider objective information about the triggering situation to verify described reflections. Finally, internal and external contextual factors such as motivation, instruction, character of assessment (formative or summative) and the ability of individual learning environments to stimulate reflection should be considered

Exploring the (missed) connections between digital scholarship and faculty development: a conceptual analysis

Abstract The aim of this paper is to explore the relationship between two research topics: digital scholarship and faculty development. The former topic drives attention on academics' new practices in digital, open and networked contexts; the second is focused on the requirements and strategies to promote academics' professional learning and career advancement. The research question addressing this study is: are faculty development strategies hindered by the lack of a cohesive view in the research on digital scholarship? The main assumption guiding this research question is that clear conceptual frameworks and models of professional practice lead to effective faculty development strategies. Through a wide overview of the evolution of both digital scholarship and faculty development, followed by a conceptual analysis of the intersections between fields, the paper attempts to show the extent on which the situation in one area (digital scholarship) might encompass criticalities for the other (faculty development) in terms of research and practices. Furthermore, three scenarios based on the several perspectives of digital scholarship are built in order to explore the research question in depth. We conclude that at the current state of art the relationship between these two topics is weak. Moreover, the dialogue between digital scholarship and faculty development could put the basis to forge effective professional learning contexts and instruments, with the ultimate goal of supporting academics to become digital scholars towards a more open and democratic vision of scholarship

Sicily statement on classification and development of evidence-based practice learning assessment tools

<p>Abstract</p> <p>Background</p> <p>Teaching the steps of evidence-based practice (EBP) has become standard curriculum for health professions at both student and professional levels. Determining the best methods for evaluating EBP learning is hampered by a dearth of valid and practical assessment tools and by the absence of guidelines for classifying the purpose of those that exist. Conceived and developed by delegates of the Fifth International Conference of Evidence-Based Health Care Teachers and Developers, the aim of this statement is to provide guidance for purposeful classification and development of tools to assess EBP learning.</p> <p>Discussion</p> <p>This paper identifies key principles for designing EBP learning assessment tools, recommends a common taxonomy for new and existing tools, and presents the Classification Rubric for EBP Assessment Tools in Education (CREATE) framework for classifying such tools. Recommendations are provided for developers of EBP learning assessments and priorities are suggested for the types of assessments that are needed. Examples place existing EBP assessments into the CREATE framework to demonstrate how a common taxonomy might facilitate purposeful development and use of EBP learning assessment tools.</p> <p>Summary</p> <p><it>The widespread adoption of EBP into professional education requires valid and reliable measures of learning. Limited tools exist with established psychometrics. This international consensus statement strives to provide direction for developers of new EBP learning assessment tools and a framework for classifying the purposes of such tools</it>.</p

Brown and black in white: The social adjustment and academic performance of Chicano and black students in a predominately white university

This article explores the academic and social experiences of Chicago and black students at UCLA. The analysis proceeds by examining differences in social backgrounds, high school and college experiences, and explores the relationship between these factors and college adjustment and achievement (GPA). Drawing upon recent theory on class reproduction and schooling we show particular concern with the role of social class in explaining differential outcomes. The findings indicate that blacks are more likely than Chicanos to feel alienated and perform poorly, and that social class makes no difference in these outcomes for blacks. However, middle class Chicanos perform better and are better adjusted than working class Chicanos. We discuss our findings in the light of theories of class reproduction, cultural capital, and racial signaling, suggesting that theories of reproduction must acknowledge the role of race in unequal school outcomes.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/43870/1/11256_2005_Article_BF01141631.pd