Modelos de Valor Agregado da Eficácia Docente

Recent policy interest in tying student learning to teacher evaluation has led to growing use of value-added methods for assessing student learning gains linked to individual teachers. VAM analyses rely on complex assumptions about the roles of schools, multiple teachers, student aptitudes and efforts, homes and families in producing measured student learning gains. This article reports on analyses that examine the stability of high school teacher effectiveness rankings across differing conditions. We find that judgments of teacher effectiveness for a given teacher can vary substantially across statistical models, classes taught, and years. Furthermore, student characteristics can impact teacher rankings, sometimes dramatically, even when such characteristics have been previously controlled statistically in the value-added model. A teacher who teaches less advantaged students in a given course or year typically receives lower effectiveness ratings than the same teacher teaching more advantaged students in a different course or year. Models that fail to take student demographics into account further disadvantage teachers serving large numbers of low-income, limited English proficient, or lower-tracked students. We examine a number of potential reasons for these findings, and we conclude that caution should be exercised in using student achievement gains and value-added methods to assess teachers effectiveness, especially when the stakes are high.El reciente inters para relacionar los resultados de los aprendizajes de los estudiantes con las evaluaciones docentes ha provocado un uso creciente de Mtodos de Valor Aadido (MVA) para evaluar incrementos en los aprendizajes de los alumnos. Los anlisis MVA se basan en hiptesis complejas acerca del papel que desempean las escuelas, los profesores, las aptitudes de los estudiantes, sus esfuerzos, sus hogares y familias en la produccin de incrementos medibles de aprendizaje estudiantil . Este artculo discute los anlisis que examinan la estabilidad de las clasificaciones de eficacia de profesores de escuelas secundarias a travs de diferentes condiciones. Encontramos que valoraciones de la efectividad docente de un profesor/a pueden variar substancialmente a travs de los distintos modelos estadsticos, las clases enseadas y los aos de experiencia. Adems, las caractersticas de los estudiantes pueden impactar drsticamente las clasificaciones de los profesores, an cuando tales caractersticas hayan sido previamente controladas estadsticamente en un MVA. Los profesores que ensean a los estudiantes menos aventajados en un curso o ao determinado reciben clasificaciones de eficacia ms bajas que cuando un mismo profesor est enseando a los estudiantes ms aventajados en otro curso o ao. Los modelos que no toman en cuenta las caractersticas demogrficas de los estudiantes perjudican adicionalmente a los profesores que sirven a un gran nmero de estudiantes de bajos ingresos, con conocimientos limitados del idioma ingls o que estn en cursos de menor rendimiento acadmico. Examinamos un gran nmero de posibles razones para explicar estos resultados, entre ellas la naturaleza especfica de las medidas de las evaluaciones y los mtodos estadsticos utilizados y concluimos que se debe tener precaucin al hacer inferencias sobre la efectividad de profesores individuales basadas en modelos de Valor Aadido especialmente para determinar resultados que puedan tener consecuencias severas (como promocin de un ciclo para otro).O recente interesse poltico de relacionar resultados da aprendizagem dos alunos com as avaliaes de professores ocasionou uma crescente utilizao de Mtodos de Valor Agregado (MVA) para avaliar ganhos na aprendizagem dos alunos. Anlises MVA so baseadas em hipteses complexas sobre o papel das escolas, dos variados professores, das competncias e empenho dos alunos e do ambiente familiar na produo de aumentos mensurveis da aprendizagem dos alunos. Este artigo discute trabalhos que examinam a estabilidade das classificaes dos professores de ensino mdio em condies diferentes. Descobrimos que as avaliaes de eficcia do ensino de um professor podem variar consideravelmente a partir dos diferentes modelos estatsticos, das aulas ministradas e dos anos de experincia. Alm disso, caractersticas dos estudantes podem modificar drasticamente a classificao dos professores, mesmo que tais caractersticas tenham sido controladas estatisticamente nos MVA. Professores que ensinam alunos mais desfavorecidos em um determinado curso ou srie recebem avaliaes de desempenho inferiores do que o mesmo professor ensinando alunos mais favorecidos em outro curso ou srie. Modelos que no levam em conta as caractersticas demogrficas dos estudantes podem prejudicar professores que atendem um grande nmero de alunos de baixa renda, com habilidades limitadas em ingls ou que esto em cursos considerados fracos. Examinamos um grande nmero de possveis razes para estes resultados, incluindo a natureza especfica das medidas de avaliao e os mtodos estatsticos utilizados. Conclumos que necessrio muito cuidado ao se fazer inferncias sobre a eficcia dos professores com base em Modelos de Valor Agregado (MVA), especialmente para determinar resultados que podem ter consequncias graves (tais como a promoo de um ciclo para outro)

2. Teacher-Performance Assessments: A New Kind of Teacher Examination

During the last week of July 1987,20 fourth- and fifth-grade teachers spent four days at a simulated assessment center in an elementary school. Each teacher completed 10 performance exercises on the teaching of equivalent fractions. The following week, 20 high school teachers of United States history spent four days completing a like number of exercises on the American revolution and the formation of the new government. These field tests were the culmination of over a year\u27s work by the Teacher Assessment Project (TAP), sponsored by the Carnegie Corporation of New York, under the direction of Professor Lee S. Shulman at Stanford University. The TAP prototype exercises represent a fundamentally new kind of teacher examination, based on structured observations of teachers\u27 performance in situations designed to elicit the same kinds of knowledge and skills as they use in teaching, lesson planning, textbook selection, or related activities. Used in conjunction with more conventional examination formats and additional kinds of evidence (e.g., academic training or documentation of on-the-job performance), exercises based on some of these prototypes are expected to play an important role in the certification process being developed by the recently created National Board for Professional Teaching Standards. Some of the TAP prototypes directly simulate activities that are part of teaching or preparing to teach. Others, like discussing the performance of another teacher viewed on videotape, are more remote from the day-to-day work of teaching. All of the TAP exercises are designed to elicit forms of knowledge and analysis that may be critical for expert teaching. For purposes of the TAP\u27s research, prototypes were developed around two specific topics: the teaching of fractions, in particular the equivalence of fractions, at the upper elementary level, and the teaching of the American Revolution and the formation of the new government in a high school course on American history. Different exercises in each of these two content areas require from 45 minutes to 3 hours to complete and call on teachers to plan a lesson, critique a videotape of another teacher presenting a lesson, discuss the use of specific instructional materials, analyze and critique a textbook, or teach a lesson of their own choosing to a group of six students. Teacher examinees respond to particular student questions and comment on student homework problems (in mathematics) or brief essays (in history). Another exercise in elementary mathematics requires teachers to discuss the relationships among a set of possible topics from a unit on fractions, to select an appropriate sequence in which to teach those topics, and to explain their selection. They also demonstrate or describe methods of using specific household articles for teaching the equivalence of fractions, discuss the advantages and disadvantages of teaching students to use different methods for solving fraction problems, and describe their classroom routines for checking mathematics homework. High school history teachers engage in a group-planning exercise, in which three or four teachers work together to plan a unit on a specified topic. In another exercise, each teacher designs packets of instructional materials for a particular form of cooperative group learning activity. The 1987 field tests of these exercises yielded a rich and extensive data base. In two weeks, they generated roughly 200 videotapes, 400 audiotapes, several thousand pages of observer notes, and hundreds of pages of notes and other writing by the 40 participating teachers themselves

An application of latent class models to assessment data

Responses of 17-year-olds to selected 1977-78 National Assessment of Educational Progress (NAEP) mathematics exercises were analyzed, using latent class models. A single model was fitted to data from five independent samples of examinees, each of which responded to a different set of six algebra or prealgebra exercises. Four categories of items were found, defining five levels of content mastery, ranging from examinees unable to solve any of the exercises (43%) through those able to solve all the exercises (19%). The methods demonstrated are broadly applicable to assessment data, including matrix-sampled data, and provide an aggregate description of examinee abilities independent of the specific characteristics of individual exercises administered

Evaluating Teacher Evaluation

There is a growing consensus that evidence of teachers' contributions to student learning should be a component of teacher evaluation systems, along with evidence about the quality of teachers' practice. Value-added models (VAMs), designed to evaluate student test score gains from one year to the next are often promoted as tools to accomplish this goal. However, current research suggests that VAM ratings are not sufficiently reliable or valid to support high-stakes, individual-level decisions about teachers. Other tools for teacher evaluation have shown greater success in measuring and improving teaching, especially those that examine teachers' practices in relation to professional standards