Algorithm- Versus Human-Generated Academic Plans: Determining Optimality from Community College Articulation Agreements

Abstract

We developed a low-fidelity prototype of a report that contains an algorithmically-generated optimal academic plan. Optimal is defined as the minimal set of community college courses that satisfy the transfer requirements for multiple universities a student is preparing to apply to. We recruited 24 California community college transfer students to participate in a research session, consisting of an experiment, survey, and interview. We experimentally compared the prototype to ASSIST, California's official statewide database of articulation agreement reports. Compared to students who used the prototype, students assigned to use ASSIST reports to manually create an optimal academic plan underperformed in optimality mistakes, time required, and usability scores. Moving to our non-experimental results, a sizable minority of students had a negative assessment of counselors' ability and willingness to manually create optimal academic plans using ASSIST. Our last results revolved around students' recommendations for supplemental software features to improve the optimization prototype