The research literature is replete with information about the teacher shortage.
The connection between teacher shortages and teacher classroom effectiveness with
student achievement substantiates the need for interventions. Research has identified the
potential of developmental mentoring and supervision programs for increasing teacher
effectiveness, teacher retention, and student achievement.
The purpose of this study was to develop and to analyze the psychometric
properties of the Conceptual Level Teacher Behavior Observation Tool (CLTBOT). The
purpose of this study was important because the development of the CLTBOT filled a
void in the literature for an observation tool that would evaluate teacher behaviors in the
conceptual domain. The potential use for these data is tied to mentoring or supervisory
practices designed specifically for the teacher’s current need for structure, as well as for
showing evidence of growth resulting from program activities.
This study was organized into three steps. Step one focused on the development
of the CLTBOT. Step two, of this study, explored the validity of the first draft of the
CLTBOT in a pilot study. The pilot study indicated a moderate association between an adapted version of Hunt’s Paragraph Completion Method (PCM), the established
measure for conceptual development, and the CLTBOT, the focal instrument of this
study. The pilot proved an essential step in the process of developing and analyzing the
CLTBOT as revisions were made following the results. Step three was the research
study designed to answer the research questions. Research question one required an
item by item analysis of the CLTBOT. Cohen’s kappa coefficients of between .699 and
.867 demonstrated that the two raters’ scores were consistent. Research question two
was answered with evaluations of the CLTBOT by two experts who awarded high
ratings for the items based on relevance and clarity. A Cramer’s V coefficient of .56
revealed a strong relationship between the CLTBOT and the PCM, establishing evidence
for concurrent validity and answering research question three. The results provided
preliminary validity and reliability evidence for the use of the Conceptual Level Teacher
Behavior Observation Tool (CLTBOT)