Using Teacher Logs to Measure the Enacted Curriculum: A Study of Literacy Teaching in Third‐Grade Classrooms

In this article we examine methodological and conceptual issues that emerge when researchers measure the enacted curriculum in schools. After outlining key theoretical considerations that guide measurement of this construct and alternative strategies for collecting and analyzing data on it, we illustrate one approach to gathering and analyzing data on the enacted curriculum. Using log data on the reading/language arts instruction of more than 150 third‐grade teachers in 53 high‐poverty elementary schools participating in the Study of Instructional Improvement, we estimated several hierarchical linear models and found that the curricular content of literacy instruction (a) varied widely from day to day, (b) did not vary much among students in the same classroom, but (c) did vary greatly across classrooms, largely as the result of teachers’ participation in 1 of the 3 instructional improvement interventions (Accelerated Schools, America’s Choice, and Success for All) under study. The implications of these findings for future research on the enacted curriculum are discussed

Using instructional logs to identify quality in educational settings

When attempting to identify educational settings that are most effective in improving student achievement, classroom process (that is, the way in which a teacher interacts with his or her students) is a key feature of interest. Unfortunately, high-quality assessment of the student-teacher interaction occurs all too infrequently, despite the critical role that understanding and measuring such processes can play in school improvement. This article discusses the strengths and weaknesses of two common approaches to studying these processes—direct classroom observation and annual surveys of teachers—and then describes the ways in which instructional logs can be used to overcome some of the limitations of these two approaches when gathering data on curriculum content and coverage. Classroom observations are expensive, require extensive training of raters to ensure consistency in the observations, and because of their expense generally cannot be conducted frequently enough to enable the researcher to generalize observational findings to the entire school year or illuminate the patterns of instructional change that occur across the school year. Annual surveys are less expensive but often suffer from self-report bias and the bias that occurs when teachers are asked to retrospectively report on their activities over the course of a single year. Instructional logs offer a valid, reliable, and relatively cost-effective alternative for collecting detailed information about classroom practice and can overcome some of the limitations of both observations and annual surveys.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/62153/1/294_ftp.pd

What Large-Scale, Survey Research Tells Us About Teacher Effects on Student Achievement: Insights From the Prospectus Study of Elementary Schools

This report is about conceptual and methodological issues that arise when educational researchers use data from large-scale, survey research studies to investigate teacher effects on student achievement. In the report, we illustrate these issues by reporting on a series of analyses we conducted using data from Prospects: The Congressionally Mandated Study of Educational Opportunity. This large-scale, survey research effort gathered a rich store of data on instructional processes and student achievement in a large sample of U.S. elementary schools during the early 1990s as part of the federal government\u27s evaluation of the Title I program. We use data from Prospects to estimate the overall size of teacher effects on student achievement and to test some specific hypotheses about why such effects occur. On the basis of these analyses, we draw some substantive conclusions about the magnitude and sources of teacher effects on student achievement and suggest some ways that survey-based research on teaching can be improved

Predicting the Quality of Revisions in Argumentative Writing

The ability to revise in response to feedback is critical to students' writing success. In the case of argument writing in specific, identifying whether an argument revision (AR) is successful or not is a complex problem because AR quality is dependent on the overall content of an argument. For example, adding the same evidence sentence could strengthen or weaken existing claims in different argument contexts (ACs). To address this issue we developed Chain-of-Thought prompts to facilitate ChatGPT-generated ACs for AR quality predictions. The experiments on two corpora, our annotated elementary essays and existing college essays benchmark, demonstrate the superiority of the proposed ACs over baselines.Comment: In The 18th BEA Workshop, held in conjunction with The Association for Computational Linguistics (ACL), July 202

Utilizing Natural Language Processing for Automated Assessment of Classroom Discussion

Rigorous and interactive class discussions that support students to engage in high-level thinking and reasoning are essential to learning and are a central component of most teaching interventions. However, formally assessing discussion quality 'at scale' is expensive and infeasible for most researchers. In this work, we experimented with various modern natural language processing (NLP) techniques to automatically generate rubric scores for individual dimensions of classroom text discussion quality. Specifically, we worked on a dataset of 90 classroom discussion transcripts consisting of over 18000 turns annotated with fine-grained Analyzing Teaching Moves (ATM) codes and focused on four Instructional Quality Assessment (IQA) rubrics. Despite the limited amount of data, our work shows encouraging results in some of the rubrics while suggesting that there is room for improvement in the others. We also found that certain NLP approaches work better for certain rubrics.Comment: to be published in AIED 202

Using Teacher Logs to Measure the Enacted Curriculum: A Study of Literacy Teaching in Third‐Grade Classrooms

In this article we examine methodological and conceptual issues that emerge when researchers measure the enacted curriculum in schools. After outlining key theoretical considerations that guide measurement of this construct and alternative strategies for collecting and analyzing data on it, we illustrate one approach to gathering and analyzing data on the enacted curriculum. Using log data on the reading/language arts instruction of more than 150 third‐grade teachers in 53 high‐poverty elementary schools participating in the Study of Instructional Improvement, we estimated several hierarchical linear models and found that the curricular content of literacy instruction (a) varied widely from day to day, (b) did not vary much among students in the same classroom, but (c) did vary greatly across classrooms, largely as the result of teachers’ participation in 1 of the 3 instructional improvement interventions (Accelerated Schools, America’s Choice, and Success for All) under study. The implications of these findings for future research on the enacted curriculum are discussed

eRevise: Using Natural Language Processing to Provide Formative Feedback on Text Evidence Usage in Student Writing

Writing a good essay typically involves students revising an initial paper draft after receiving feedback. We present eRevise, a web-based writing and revising environment that uses natural language processing features generated for rubric-based essay scoring to trigger formative feedback messages regarding students' use of evidence in response-to-text writing. By helping students understand the criteria for using text evidence during writing, eRevise empowers students to better revise their paper drafts. In a pilot deployment of eRevise in 7 classrooms spanning grades 5 and 6, the quality of text evidence usage in writing improved after students received formative feedback then engaged in paper revision.Comment: Published in IAAI 1

Kinematics and Chemistry of Stars Along the Sagittarius Trailing Tidal Tail and Constraints on the Milky Way Mass Distribution

We present three-dimensional kinematics of Sagittarius (Sgr) trailing tidal debris in six fields located 70-130 degrees along the stream from the Sgr dwarf galaxy core. The data are from our proper-motion (PM) survey of Kapteyn's Selected Areas, in which we have measured accurate PMs to faint magnitudes in 40x40 arcmin fields evenly spaced across the sky. The radial velocity (RV) signature of Sgr has been identified among our follow-up spectroscopic data in four of the six fields and combined with mean PMs of spectroscopically-confirmed members to derive space motions of Sgr debris based on 15-64 confirmed stream members per field. These kinematics are compared to predictions of the Law & Majewski (2010) model of Sgr disruption; we find reasonable agreement with model predictions in RVs and PMs along Galactic latitude. However, an upward adjustment of the Local Standard of Rest velocity Theta_LSR from its standard 220 km/s to at least $232\pm14$ km/s (and possibly as high as $264\pm23$ km/s) is necessary to bring 3-D model debris kinematics and our measurements into agreement. Satisfactory model fits that simultaneously reproduce known position, distance, and radial velocity trends of the Sgr tidal streams, while significantly increasing Theta_LSR}, could only be achieved by increasing the Galactic bulge and disk mass while leaving the dark matter halo fixed to the best-fit values from Law & Majewski (2010). We derive low-resolution spectroscopic abundances along this stretch of the Sgr stream and find a constant [Fe/H] = -1.15 (with ~0.5 dex scatter in each field -- typical for dwarf galaxy populations) among the four fields with reliable measurements. A constant metallicity suggests that debris along the ~60-degree span of this study was all stripped from Sgr on the same orbital passage.Comment: 26 pages, 24 figures (some with degraded resolution), accepted to ApJ; full-resolution version available at http://www.rpi.edu/~carlij/sgr_paper/ms.apjformat.ps.g

The Science Performance of JWST as Characterized in Commissioning

This paper characterizes the actual science performance of the James Webb Space Telescope (JWST), as determined from the six month commissioning period. We summarize the performance of the spacecraft, telescope, science instruments, and ground system, with an emphasis on differences from pre-launch expectations. Commissioning has made clear that JWST is fully capable of achieving the discoveries for which it was built. Moreover, almost across the board, the science performance of JWST is better than expected; in most cases, JWST will go deeper faster than expected. The telescope and instrument suite have demonstrated the sensitivity, stability, image quality, and spectral range that are necessary to transform our understanding of the cosmos through observations spanning from near-earth asteroids to the most distant galaxies.Comment: 5th version as accepted to PASP; 31 pages, 18 figures; https://iopscience.iop.org/article/10.1088/1538-3873/acb29