Instructor performance evaluation: a nationwide survey of performance appraisal in two-year colleges

This study identified the current practices of two-year colleges in the area of instructor performance appraisal; classified the models of evaluation currently in use in the various types and sizes of two-year colleges; established the efficacy of each model as it relates to the twenty-one personnel evaluation standards recently identified by the Joint Committee (1988); and developed a profile of the typical supervisor conducting instructor performance appraisal;The results of this study support previous recommendations of related research which strongly suggests the need for improvement in two-year college instructor evaluation. The extent to which the practices of instructor performance appraisal are being carried out indicate a need for training;The major approaches being used in instructor evaluation are characterized by a close relationship between the instructor and the supervisor with emphasis on collegial rather than authoritarian orientation. However, evaluation is synonymous with observation and the major emphasis is on summative evaluation; standardized criteria; and comparative judgements. This process usually takes its principal data from the classroom and is designed to improve the instructor\u27s performance;This study concludes that differences in the essential elements in use and current practices of instructor performance appraisal were substantial. Additionally, only a few two-year colleges had instructor performance appraisal processes that appeared to represent a well-developed system which were balanced to facilitate the attainment of both improvement and accountability. Additionally, the process of instructor performance appraisal appears to be an under conceptualized and underdeveloped activity in the majority of two-year colleges. However, there is consensus about what design process results in successful teacher evaluation. This study concludes that efficacy ratings (the power or capacity to produce the desired effect) of the Personnel Standards were higher when clinical supervision was identified as the model of instructor performance appraisal;Common procedures for capturing data about instructor performance included the use of: a single instrument to evaluate most of the instructors; student feedback as a source of evaluation data; and peer feedback. The span of control, which was in some cases in excess of 100 instructors, is far too large to conduct quality, comprehensive instructor evaluations

Measuring the meta and cognitive abilities of air defence operators

Introduction: This study aimed to understand more fully some factors that influence decision confidence and accuracy related to air defence. To investigate the metacognitive abilities of air defence operators a Within-Subjects Confidence-Accuracy (W-S C-A) measure was used. Specifically, therefore, this study investigated the impact of Decision Criticality (DC) and Task Stress (TS) on decision making, measures of confidence, accuracy, and the W-S C-A relation. Personality constructs, workload and situation awareness were also included. Method: Participants were allocated to either a high, moderate or low task stress condition. Each participant then took part in a computer generated air defence scenario where they were required to make various decisions and provide a confidence rating for each of those decisions. Confidence, accuracy and W-S C-A were calculated. Results & Discussion: DC impacted both on decision confidence and accuracy, with low DC increasing confidence in decisions and high DC increasing accuracy in decisions

Linking Teacher Evaluation, Professional Growth, and Motivation: A Multiple-site Case Study

The primary purpose of this study was to investigate the nature of the activities, processes, and structures used to link a teacher evaluation program to professional growth and motivation. A total of 52 teachers in the Johnson City School System, who were scheduled for evaluation during the 1994-95 school year, were selected by both random and purposeful sampling techniques to participate. Data were collected through both quantitative and qualitative methods. Principals of the nine schools involved also participated in the naturalistic inquiry component of the study. Data were analyzed both deductively and inductively. The analysis revealed attitudes, behaviors, and perceptions of those involved in the implementation of a growth-oriented approach to teacher evaluation. What were the reasons for the success or failure of the program? Through data analysis the investigator identified 12 critical elements within four major categories that influence the linking of teacher evaluation, professional growth, and motivation. The four major categories are: characteristics of the culture, characteristics of the administrator, characteristics of the teacher, and characteristics of the process. The 12 critical elements were identified as follows: a trusting environment; collaborative relationships; high expectations of growth; administrators who are facilitators or coaches and resource providers; teachers who are mature, responsible, and self-directed; and a continuous process that is individualized, formative, and structured. Based on the findings, the following recommendations were suggested: (1) assess the culture of the school before implementing the Professional Teacher Evaluation Model, (2) provide training for teachers and administrators, (3) allow teachers to choose professional growth options, (4) identify teachers\u27 level of readiness for self-directive learning, (5) emphasize importance of the principal\u27s role, (6) ensure presence of the 12 critical elements identified, and (7) implement the Professional Teacher Evaluation Model as the professional growth option of the proposed Tennessee State Model for Local Evaluation

Modelling and supporting flight crew decision-making during aircraft engine malfunctions: developing design recommendations from cognitive work analysis

In this article, we analyse flight crew response to an in-flight powerplant system malfunction (PSM) using control task analysis. We demonstrate the application of the decision ladder template and the skills, rules, and knowledge (SRK) framework to this new area of inquiry. Despite the high reliability of turbofan engines, accidents and incidents involving PSM still occur. During these unusual events, flight crew have not always responded appropriately, leading to a reduction in safety margins or disruption of operations. This article proposes recommendations for technological and information system that can support flight crew in responding safely and appropriately to a PSM. These recommendations focus on new ways in which information from engine health monitoring system and other sources of data can be utilised and displayed. Firstly, we conducted knowledge elicitation using Critical Decision Method (CDM) interviews with airline pilots who have experienced real or simulated PSM events. We then developed generic decision ladders using the interview data, operations manual, training manual, and other guideline documents. The generic decision ladders characterise the different stages of responding to PSM identified as part of the research. These stages include: regaining and maintaining control of aircraft, identifying PSM and selecting appropriate checklists to secure the engine, and modifying the flight plan. Using the decision ladders and insights from the CDM interviews, we were able to identify cognitive processes and states that are more prone to errors and therefore more likely to generate an inappropriate response. Using the SRK framework, we propose design recommendations for technological and information systems to minimise the likelihood of such inappropriate response. We conclude that this combination of methods provides a structured and reliable approach to identifying system improvements in complex and dynamic work situations. Our specific contributions are the application of these techniques in the unrepresented area of flight operations, and the development of evidence-based design recommendations to improve flight crew response to in-flight powerplant system malfunctions

Cognitive work analysis: Extensions and applications in command and control

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.'Complex sociotechnical systems' are systems made up of numerous interacting parts, both human and non-human, operating in dynamic, ambiguous and safety critical domains. The system design, and representation, has significant implications for the usability of the system, its performance, errors and reliability. Due to its formative constraint-based approach, Cognitive Work Analysis (CWA) has been frequently proposed as a framework for modelling these complex systems, further it is contended that the approach can be extended to design for interaction without significant deviation from the accepted framework. This thesis presents a number of applications in complex military domains to explore and develop the benefits of CWA. Unlike much of the previous literature, particular attention is placed on exploring the CWA framework in its entirety. This holistic approach focuses on the system environment, the activity that takes place within it, the strategies used to conduct this activity, the way in which the constituent parts of the system (both human and non-human) interact and the behaviour required. Each stage of this analysis identifies the constraints governing the system; it is contended that through this holistic understanding of constraints, recommendations can be made for the design of system interaction; increasing the ability of users to cope with unanticipated, unexpected situations. This thesis discusses the applicability of the approach in system analysis, development and evaluation. It provides process to what was previously a loosely defined framework.This work is part-funded by the Human Sciences Domain of the UK Ministry of Defence Scientific Research Programme

A Cognitive Analysis of Truck Drivers’ Right-hand Turns

This paper presents an investigation of truck drivers' performance during right-hand turns performed in in- tersections with traffic lights in order to elicit the truck drivers' domain, decision-making processes, and the strategies used while executing the turns. To gain knowledge about this, a truck driving instructor was inter- viewed, and four truck drivers were observed and interviewed. This resulted in a cognitive work analysis with the phases work domain analysis, control task analysis and strategies analysis. Through this study it is indi- cated that many different types of information are sought using several brief glances in mirrors, and the results shows how the front and near-zone mirrors were never used by the participants during the observed right-hand turns. Controlling the vehicle was found to be mainly automatic, whereas their focus is on the orientation about surroundings. Lastly, the drivers' behavior is discussed in relation to change blindness, con- firmation bias and schemes

A Cognitive Analysis of Truck Drivers’ Right-hand Turns

This paper presents an investigation of truck drivers' performance during right-hand turns performed in intersections with traffic lights in order to elicit the truck drivers' domain, decision-making processes, and the strategies used while executing the turns. To gain knowledge about this, a truck driving instructor was interviewed, and four truck drivers were observed and interviewed. This resulted in a cognitive work analysis with the phases work domain analysis, control task analysis and strategies analysis. Through this study it is indi- cated that many different types of information are sought using several brief glances in mirrors, and the results shows how the front and near-zone mirrors were never used by the participants during the observed right-hand turns. Controlling the vehicle was found to be mainly automatic, whereas their focus is on the orientation about surroundings. Lastly, the drivers' behavior is discussed in relation to change blindness, confirmation bias and schemes