How good are value-added measures of teacher performance? A review and empirical investigation using data from Turkey

Education systems all over the world aim to provide good quality education for their citizens. This would require a good supply of quality teachers. The role of teachers is now more complex than ever before. Consequently, evaluating the quality of a teacher has also become more complex. While we may feel that we know intuitively what an effective teacher looks like, there is little consensus on how best to measure or capture the essence of a good teacher. Classroom observations protocols, interviews and surveys with teachers and pupils are commonly used to assess teachers. Increasingly, governments and schools are using standardised pupil test scores in teacher performance appraisal as a way of estimating how much difference teachers can make to student attainment by comparing the progress students make. This is seen as perhaps more objective or fair because students' test scores are considered objective measures. Such evaluation of teachers, also known as value-added models or VAMs, are increasingly used to measure teacher effectiveness for high-stake decisions, such as teachers' salaries and promotions. Teachers are rewarded or penalise based on these value-added measures. VAMs have attracted considerable attention in recent years. Many researchers have raised concerns about their validity and reliability. There are also concerns about VAM's ability to predict the effectiveness of teachers consistently. Value-added measures of teachers are known to vary from year to year and from subject to subject. Different value-added models can also produce different estimates of teacher performance depending on the student achievement test scores used. This study adds to the current debates by examining the stability of VAMs to see whether teacher effectiveness can be predicted consistently using different parameters, such as observable student, teacher/classroom and school characteristics, the number of student test scores obtained over time, and the data analysis methods used. Value-added measures can only be useful in estimating teacher effectiveness if they produce consistent results for the same teachers across time for different students. This new research begins with a systematic review of the existing literature examining the stability of different VAMs as a measure of teacher performance. Of 1,439 results, 50 studies met the inclusion criteria to be included in the synthesis. Each of these studies was given a padlock rating in terms of the trustworthiness of its findings based on four criteria (such as research design and threats to validity) using a bespoke assessment tool. Studies were rated from 0 (very weak) to 4 padlocks (the most secure that can be expected). Since the main research question (stability of estimates) is descriptive, correlational/comparative studies are appropriate in design. Most studies retrieved were correlational/comparative in design. Some of them rated the highest, as they were large-scale, allowed random teacher-student allocations, and had low attrition. The majority of the studies in the review were rated 3 padlocks as they employed administrative/panel data where students are not randomly assigned to teachers in value-added estimates and/or were smaller or had higher attrition. The strongest studies revealed that using one prior attainment score is sufficient to predict teacher performance. Using additional prior test scores does not increase the stability of value-added teacher effectiveness estimates consistently. Including student, school and teacher/classroom-level variables adds little to the predictive power of teacher performance assessment models. This suggests that these variables are not good predictors of teacher effectiveness. The systematic review found no evidence that any particular data analysis method is better in its ability to estimate teachers' effectiveness reliably. Most studies in the review were conducted in the US using national administrative data. To see if the findings also apply in other contexts, longitudinal data of five teaching subjects (maths, Turkish science, history, and English) from one province in Turkey was then used to test the stability of value-added estimates. The data included 35,435 Grade 8 students (age 13-14, equivalent to Year 9 in the UK), matched to 1,027 teachers. To test how much progress in student academic achievement is available to be attributed to a teacher from one year to the next, a series of regression analyses were run. Models included contextual predictors at student-, school-, and teacher/classroom-level. Consistent with the findings of the systematic review, the results show that the best predictor of students' later test scores is their prior attainment. Using additional years' test scores instead of a single prior-year attainment score contributed little to improving value-added teacher effectiveness estimates. Including other factors, such as student, teacher, and school characteristics in the model also explains very little in the variations in students' test scores once the prior attainment is taken into account (although the data on teacher characteristics was limited in the dataset). Correlation analyses suggested that there was no meaningful relationship between teacher effectiveness scores and the teacher/classroom-level variables. Interestingly, teacher experience, regardless of whether it refers to their total experience or only that in their current schools, is negatively related to teacher effectiveness scores. In other words, more experienced teachers tend to have lower effectiveness scores on the value-added estimate. There was no evidence that teachers are more effective in smaller classes. Only a modest correlation was found between class size and teacher effectiveness. Intriguingly, students in large classes tend to have more “effective” teachers in value-added terms (except in history), although the difference is minimal. The analysis also found that teachers’ previous effectiveness scores had little or no relationship with their current effectiveness scores, regardless of teaching subjects. Consistent with the literature in the review, this study also found that teacher effectiveness scores based on value-added estimates vary substantially across years. This means that the same teacher can be considered “effective” in one year and “ineffective” in other. This casts doubt on the reliability and meaningfulness of value-added measures. As with previous studies in the systematic review, there is no evidence from the Turkish data that any single value-added approach is superior to any other approach regarding the ability to consistently estimate teachers’ effectiveness. There is no advantage in using more sophisticated statistical models. The findings of this study suggest that regardless of the number of test scores, or variables used or data analysis methods, there is no consistent or reliable way of measuring teacher effectiveness. This highlights the danger of using value-added models in measuring teacher effectiveness. Studies suggested that some of the inconsistencies could be the result of measurement error and the timing of the test. There is, therefore, the risk of misclassifying teachers as “effective” or “ineffective”. Some teachers may be deemed 'effective' on one test but not another simply based on when the tests are scheduled. These findings have important implications for policy and practice. Value-added models should not be used to make high stake personnel decisions. They may have some value for research purposes or to provide formative feedback to headteachers about a class or a teacher as part of a larger set of evidence. One major limitation of VAMs is that they measure teacher performance using tests designed to measure student performance. The assumption is that student performance is directly related to teacher quality. While there has been a lot of research on developing teacher quality, measuring teacher quality is itself problematic. The issue of measuring teachers performance has been one of the leading issues in education policies. A critical question that needs to be asked is not how effective teacher are, but what is the purpose of evaluating teacher performance? If such an exercise aims to differentiate “effective” from “non-effective” teachers since there is no reliable method or no methods that have been robustly tested and shown to work in identifying effective teachers, why are we still doing it? To improve teachers’ effectiveness and keep them updated with robustly tested and proven teaching approaches, it might be better to provide teachers with training, professional development to develop pedagogic skills, social and personal relationship skills, behavioural management, and subject knowledge. Assuming that classroom teachers have gone through teacher training and are certified, then they should be qualified to teach. If they are not deemed “effective”, it is perhaps the failure of the selection and training process more so than the quality of the individual teacher. Another major limitation of VAMs is that they are comparative and zero-sum. For a teacher to be deemed effective, another must be deemed ineffective. Thus, if all teachers were actually effective (or ineffective), a VAM would still assess up to half of them to be ineffective (or effective). They are not fit for purpose

Impact of Contextual Predictors on Value-Added Teacher Effectiveness Estimates

It is widely believed that the teacher is one of the most important factors influencing a student’s success at school. In many countries, teachers’ salaries and promotion prospects are determined by their students’ performance. Value-added models (VAMs) are increasingly used to measure teacher effectiveness to reward or penalize teachers. The aim of this paper is to examine the relationship between teacher effectiveness and student academic performance, controlling for other contextual factors, such as student and school characteristics. The data are based on 7543 Grade 8 students matched with 230 teachers from one province in Turkey. To test how much progress in student academic achievement can be attributed to a teacher, a series of regression analyses were run including contextual predictors at the student, school and teacher/classroom level. The results show that approximately half of the differences in students’ math test scores can be explained by their prior attainment alone (47%). Other factors, such as teacher and school characteristics explain very little the variance in students’ test scores once the prior attainment is taken into account. This suggests that teachers add little to students’ later performance. The implication, therefore, is that any intervention to improve students’ achievement should be introduced much earlier in their school life. However, this does not mean that teachers are not important. Teachers are key to schools and student learning, even if they are not differentially effective from each other in the local (or any) school system. Therefore, systems that attempt to differentiate “effective” from “ineffective” teachers may not be fair to some teachers

Best proximity point theorems for cyclic p-contractions with some consequences and applications

In this paper, we introduce the concept of cyclic p-contraction pair for single-valued mappings. Then we present some best proximity point results for such mappings defined on proximally complete pair of subsets of a metric space. Also, we provide some illustrative examples that compared our results with some earliest. Finally, by taking into account a fixed point consequence of our main result we give an existence and uniqueness result for a common solution of a system of second order boundary value problems

The Role of Teacher Identity on Using Formative Assessment to Support the Learning of Students with ASC in Mainstream Classes

At the centre of this study is the question of the role of teacher identity in how formative assessment (FA) methods are used to support the learning of students with Autism Spectrum Conditions (ASC) in mainstream classes. To address this question, a qualitative case study was conducted with 6 teachers, 6 teaching assistants and 4 students with ASC in two primary mainstream schools in England. In this case study, interviews, observations and documents were used as the data collection method, and consequently, the thematic inductive analysis method was employed in the analysis of the data. Findings suggest that teacher identity is influenced by three main factors, and these factors directly and indirectly affect teachers' formative assessment practices while teaching students with ASC. The first category is personal factors; teachers are influenced by their biography, emotions, self-esteem and role conflict in forming their teacher identities. The second category is contextual factors that relate to policy, school context and school leadership and are part of school culture. These factors significantly affect and shape teachers' pedagogy and assessment practices. The last category is professional factors, including teaching experience, collaborations, pedagogical content knowledge, curriculum and assessment. From these findings, it is understood that teachers and teaching assistants recognise the importance of inclusive education and the use of formative assessment, and despite the many difficulties they encounter, teachers and TAs support the education of students with ASC in mainstream settings. Teachers consider formative assessment to be a flexible and effective way to reach the student with ASC and follow the learning journey. Overall, this study demonstrates why teachers use formative assessment, the link between their identity and formative assessment, and how students with ASC respond to these methods. For this reason, the research findings are important in suggesting that teachers and teacher assistants should develop awareness of the power of their identity, inform parents about their supporting roles in the assessment process, inform the school leaders of how to prepare a more confident learning and teaching environment for teachers and students, and lastly, give an idea to policy makers about preparing an assessment policy that should include detailed information about formative assessment in a mainstream environment

The European states system and Ottoman-Russian relations, 1815- 1856

This thesis looks at the way that the European states system operated and affected the Ottoman-Russian relations between 1815 and 1856. The 1815 Settlements established a new system of international relations in Europe. Russia played the greatest role in foiling the Napoleonic bid for hegemony. The most distinguishing characteristic of this new system was that its structure made cooperation effective within the framework of the institution of Concert of Europe. In this respect the four victorious great powers, i.e. Russia, England, Prussia and Austria, did not exclude the defeated power, France, and they took on the governance of international affairs. The new system was built on the political and territorial balance. To this end, the restriction of France and moderation of Russia was necessary. Both countries had some revisionist objectives. Consequently, Near East became the centre stage of the international politics after 1815. Ottoman Empire did not take part in the 1815 Settlements. Therefore, Ottoman-Russian relations were to continue on a bilateral base. Russia’s strategic goal to secure her south-east frontiers clashed with her responsibility for maintaining the provisions of 1815 Settlements in Central Europe since any change in Near East would affect the territorial and political balance in Central Europe, too. Under these circumstances, Russia faced a dilemma in her relations with the Ottoman Empire. Russia was very advantageous owing to her enormous power and her treaty rights regarding the Ottoman Empire which had acquired since 1774. Ottoman-Russian relations developed around three main events during 1815-1856: the revolt of Greeks (1821-29), the rebellion of Viceroy of Egypt (1833 and 1839) and the Holy Place Issue (1852-1854). In all those events Russia was successfully restrained against the Ottoman Empire by the structure of the new system. The thesis draws a number of conclusions. The underlying economic structure of the new state system almost remained the same during the 1815-1856 period. The thesis concludes that the course of Ottoman-Russian relations was increasingly determined by the elements of relationship structure. In particular, the foreign policy objectives of France played the significant role in shaping the Ottoman-Russian relations during 1815-1856

The Role of Teacher Identity on Using Formative Assessment to Support the Learning of Students with ASC in Mainstream Classes

At the centre of this study is the question of the role of teacher identity in how formative assessment (FA) methods are used to support the learning of students with Autism Spectrum Conditions (ASC) in mainstream classes. To address this question, a qualitative case study was conducted with 6 teachers, 6 teaching assistants and 4 students with ASC in two primary mainstream schools in England. In this case study, interviews, observations and documents were used as the data collection method, and consequently, the thematic inductive analysis method was employed in the analysis of the data. Findings suggest that teacher identity is influenced by three main factors, and these factors directly and indirectly affect teachers' formative assessment practices while teaching students with ASC. The first category is personal factors; teachers are influenced by their biography, emotions, self-esteem and role conflict in forming their teacher identities. The second category is contextual factors that relate to policy, school context and school leadership and are part of school culture. These factors significantly affect and shape teachers' pedagogy and assessment practices. The last category is professional factors, including teaching experience, collaborations, pedagogical content knowledge, curriculum and assessment. From these findings, it is understood that teachers and teaching assistants recognise the importance of inclusive education and the use of formative assessment, and despite the many difficulties they encounter, teachers and TAs support the education of students with ASC in mainstream settings. Teachers consider formative assessment to be a flexible and effective way to reach the student with ASC and follow the learning journey. Overall, this study demonstrates why teachers use formative assessment, the link between their identity and formative assessment, and how students with ASC respond to these methods. For this reason, the research findings are important in suggesting that teachers and teacher assistants should develop awareness of the power of their identity, inform parents about their supporting roles in the assessment process, inform the school leaders of how to prepare a more confident learning and teaching environment for teachers and students, and lastly, give an idea to policy makers about preparing an assessment policy that should include detailed information about formative assessment in a mainstream environment

S-Store: Streaming Meets Transaction Processing

Stream processing addresses the needs of real-time applications. Transaction processing addresses the coordination and safety of short atomic computations. Heretofore, these two modes of operation existed in separate, stove-piped systems. In this work, we attempt to fuse the two computational paradigms in a single system called S-Store. In this way, S-Store can simultaneously accommodate OLTP and streaming applications. We present a simple transaction model for streams that integrates seamlessly with a traditional OLTP system. We chose to build S-Store as an extension of H-Store, an open-source, in-memory, distributed OLTP database system. By implementing S-Store in this way, we can make use of the transaction processing facilities that H-Store already supports, and we can concentrate on the additional implementation features that are needed to support streaming. Similar implementations could be done using other main-memory OLTP platforms. We show that we can actually achieve higher throughput for streaming workloads in S-Store than an equivalent deployment in H-Store alone. We also show how this can be achieved within H-Store with the addition of a modest amount of new functionality. Furthermore, we compare S-Store to two state-of-the-art streaming systems, Spark Streaming and Storm, and show how S-Store matches and sometimes exceeds their performance while providing stronger transactional guarantees