Response to Intervention at the Secondary Level: Identifying Students At Risk for High School Dropout

Using the basic premise of RTI, which is to identify those students at risk, in order to provide timely intervention, this study contributes to the development of a universal screening measure to identify high school at risk of drop out. This study was designed to apply the Early Warning System (EWS) tool developed by the National High School Center as a possible Tier 1 universal screening measure to identify students at risk as part of an RTI framework for the secondary school level. Freshmen data from one high school from the 2004-2005 school year was entered into the EWS tool and compared against the graduation outcomes for the eventual 2007-2008 senior class. Predictor and outcome variables were applied in a logistic regression statistical model and statistical significance was found for the “Quarter 2 absence” and “Core courses failed” predictor variables. Further research is needed in this area to deepen the understanding of building, implementing and maintaining RTI frameworks at the secondary level. Further research is also needed to define predictor variables unique to Idaho for students at risk for drop out

Alternative Service Delivery Models for Students with Learning Disabilities

More than 40 years of intervention research describes effective interventions for increasing the academic performance of students with learning disabilities. However, the performance and outcomes for students with learning disabilities remain discouraging, especially in light of the increasing cost to provide special education services. Between 60 – 70 percent of students with learning disabilities are unable to meet grade level performance standards in the basic academic areas of reading, writing and mathematics. The disconnect between research and practice suggests a need for rethinking special education service delivery. This paper reports the results of a program evaluation of a not-for-profit center specializing in working with students with learning disabilities, comparing the pre to post test gains and program costs to those typically seen in public education special education systems. A paired samples t-test examined pre to post test differences in reading, math and writing performance for a total of 104 students with learning disabilities between the ages of 6–17. Results were statistically and practically significant across all areas, and these results were obtained at costs much lower than those incurred under the current public education model. Implications for future research are discussed

Examining Interrater Agreement Analyses of a Pilot Special Education Observation Tool

This paper reports the results of interrater agreement analyses on a pilot special education teacher evaluation instrument, the Recognizing Effective Special Education Teachers (RESET) Observation Tool (OT). Using evidence-based instructional practices as the basis for the evaluation, the RESET OT is designed for the spectrum of different instructional needs found within special education classrooms. The RESET OT informs what Danielson (2011) maintains are the two features of a teacher evaluation system 1) ensuring teacher quality and 2) promoting professional development. In June 2012, six special education teachers participated in a data coding session using the pilot RESET OT to evaluate video observations of special education instructional practice from the 2011-2012 school year. The teacher coders received an introductory training session to the RESET OT, and participated in two whole-group coding sessions before completing individual coding assignments. The results of the interrater agreement analysis report weak to no agreement within specific instructional practices, indicating the need for 1) additional research and development on the RESET OT 2) repeating the data coding session using a group of teacher coders who have received in-depth training on the RESET OT and evidence-based instructional practices, and 3) further investigation into the specific components of evidence-based instructional practice and how these might be applied across settings

Using Generalizability Theory to Measure Sources of Variance on a Special Education Teacher Observation Tool

This study used generalizability theory to identify sources of variance on a pilot observation tool designed to evaluate special education teacher effectiveness, and was guided by the question: How many occasions and raters are needed for acceptable levels of reliability when using the pilot RESET observation tool to evaluate special education teachers? At the time of this study, the pilot Recognizing Effective Special Education Teachers (RESET) observation tool included three evidence-based instructional practices (direct, explicit instruction, whole-group instruction, and discrete trial teaching) as the basis for special education teacher evaluation. Eight teachers (raters) were invited to attend two sessions (October 2012 and April 2013) to evaluate special education classroom instruction collected from the 2011-2012 and 2012-2013 school years, via the Teachscape 360-degree video system. The raters were trained on the pilot RESET observation tool, and participated in whole-group coding sessions to establish interrater agreement (minimum of 80%) before evaluating assigned videos. Data collected from raters were analyzed in a two-facet “partially” nested design where occasions (o) (observations/lessons) were nested within teachers (t), o:t, and crossed with raters (r), {o:t} x r. Using the results from the generalizability study analyses, decision studies were then completed to determine optimal facet conditions for the highest levels of reliability (the relative G coefficient and standard error of measurement scores were used to inform the decision study analyses). Results from this study are in alignment with similar studies that found multiple observations and multiple raters are critical for ensuring acceptable levels of reliability. Recommendations for future studies include investigating the use of different raters (e.g., principals, university faculty, etc.), and using larger facet sample sizes to increase the overall measurement precision of the RESET tool. Considerations for the feasibility of practice must also be observed in future reliability and validity studies on the RESET tool

Special Educator Evaluation: Cautions, Concerns and Considerations

There is currently little consensus on how special education teachers should be evaluated in a way that is effective, fair and responsive to their unique teaching responsibilities. In this paper, we explain several of the current approaches to teacher evaluation under consideration, and then provide an overview of the challenges associated with the use of these models for special education teachers. We describe a model currently under development that is designed to better meet the unique characteristics of special education teacher evaluation. Our alternative approach proposes to evaluate special education teacher effectiveness through two primary components: observations of the special educator’s use of research-based instructional practices, and the resulting student outcomes reported through effect sizes on measures aligned with relevant student goals

Validating an Observation Protocol to Measure Special Education Teacher Effectiveness

This study used Kane’s (2013) Interpretation/Use Argument (IUA) to measure validity on the Recognizing Effective Special Education Teachers (RESET) observation tool. The RESET observation tool is designed to evaluate special education teacher effectiveness using evidence-based instructional practices as the basis for evaluation. In alignment with other studies (Bell et al., 2012), we applied and interpreted Kane’s (2006) four inferences for trait observation: scoring, generalization, extrapolation, and decision rules. Results from this study show that acceptable levels of validity are promising for the RESET observation tool. Because the RESET observation tool is premised on the idea that by increasing the use of evidence-based practices, student achievement will also increase, further investigations into the relationship between fidelity of implementation of instruction and student achievement will be critical for moving project work forward

Why Did They Come Here? - The Influences and Expectations of First-Year Students’ College Experience

Students’ expectations and experiences with university life are influenced by a number of variables. Many universities develop programs or promote aspects of existing programs to market the university’s image. This research was motivated by our desire to determine the relationships between first-year students’ college expectations and experiences, their awareness of the university’s programming and projected image, the influence of the programming and image on their decision to attend the institution, and the students’ personal characteristics. Our survey of 351 first-year students revealed positive perceptions of their university expectations and experiences, a mixture of influences on their decision to attend the university, and correlations between program awareness and the influence on students’ university attendance. We also uncovered a number of relationships between expectations, experience, and perceptions of influences with the personal characteristics of the students. Implications, directions for future research, and limitations are discussed

The Predictive Validity of the Early Warning System Tool

The Early Warning System (EWS) is a tool developed by the National High School Center to collect data on indicators including attendance, GPA, course failures and credits earned. These indicators have been found to be highly predictive of a student’s likelihood of dropping out of high school in large, urban areas. The EWS tool was studied in two suburban schools. With the exception of attendance data, findings suggest that the indicators and suggested threshold for risk determination are predictive in suburban contexts

Introduction to \u3cem\u3eAEI’s\u3c/em\u3e Special Issue on Special Education Teacher Evaluations

The purpose of this introductory article is to provide an overview to the special issue on special education teacher evaluation. This special issue features five articles that collectively present a comprehensive review of the current state of understanding about the issues concerning devising and implementing special education teacher evaluation systems. Within this special issue are articles that provide a discussion of some of the primary challenges associated with evaluating special education teachers; an overview of the issues associated with using observation tools to measure special education teachers; and three studies that each feature a unique approach to special education teacher evaluation via opportunities to learn (My Instructional Learning Opportunities Guidance System; Classroom Observations of Student-Teacher Interactions–Mathematics) and evidence-based instructional practices (Recognizing Effective Special Education Teachers). Because of the current gap in special education teacher evaluation research, we hope that this special issue will contribute to the field, and advance existing research

Measuring Rater Reliability on a Special Education Observation Tool

This study used generalizability theory to measure reliability on the Recognizing Effective Special Education Teachers (RESET) observation tool designed to evaluate special education teacher effectiveness. At the time of this study, the RESET tool included three evidence-based instructional practices (direct, explicit instruction; whole-group instruction; and discrete trial teaching) as the basis for special education teacher evaluation. Five raters participated in two sessions to evaluate special education classroom instruction collected from two school years, via the Teachscape 360-degree video system. Data collected from raters were analyzed in a two-facet “partially” nested design where occasions (o) were nested within teachers (t), o:t, and crossed with raters (r), {o:t} x r. Results from this study are in alignment with similar studies that found multiple observations and multiple raters are critical for ensuring acceptable levels of measurement score reliability. Considerations for the feasibility of practice should be observed in future reliability and validity studies on the RESET tool, and further work is needed to address the lack of research on rater reliability issues within special education teacher evaluation