Includes bibliographical references (pages 44-46)Estimating the efficacy of different instructional modalities, techniques and\ud
interventions is challenging because teaching style covaries with instructor, and the typical student only takes a course once. We introduce the individualized treatment effect (ITE) from analyses of personalized medicine as a means to quantify individual student performance under different instructional modalities or intervention strategies, despite the fact that each student may experience only one ???treatment???. The ITE is presented within an ensemble machine learning approach to evaluate student performance, identify factors indicative of student success, and estimate persistence. A key element is the use of a priori student information from institutional records. The methods are motivated and illustrated in two learning analytics problems: 1) comparing an online and standard face-to-face offerings of an upper division applied statistics course that is a curriculum bottleneck at San Diego State University; 2) evaluating a new supplementary instruction component to a large enrollment introductory statistics course recognized as presenting an undesirably high repeatable grade rate. The ITE in particular allows us also to characterize students that benefit from pedagogical innovations (e.g., online or traditional course offerings) and intervention strategies (e.g., supplemental instruction). We discuss the general implications of this analytics framework for assessing pedagogical innovations and interventions strategies, identifying and characterizing at-risk students, and optimizing the individualized student learning environment
