OBJECTIVE: Surgical education is changing in an era of new regulations and evolving training cultures. We sought to understand the factors that affect operative experiences during otolaryngology residency. METHODS: From December 2019 to December 2020, five otolaryngology training programs used the SIMPL OR smartphone application to evaluate residents after each operation. Residents and attendings rated the trainees autonomy on a 4-level Zwisch scale, performance on a 5-level scale, and case complexity on a 3-level scale. We examined associations between ratings of autonomy and performance with variables including postgraduate year (PGY), case complexity, gender, week of the academic year (AY), and whether multiple procedures were logged. RESULTS: 78 attendings and 92 residents logged 2984 evaluations. PGY level and week of the AY were positively associated with attending ratings of autonomy and performance (PGY3 vs. PGY2: B = 0.63, p < .001 for autonomy and B = 1.05, p < .001 for performance; week of the AY: B = 0.013, p = .002 for autonomy; B = 0.025, p < .001 for performance). Multiple procedures logged and increasing case complexity were negatively associated with attending ratings (multiple procedures: B = -0.19, p = .04 for autonomy and B = -0.48, p < .001 for performance; hardest vs. easiest 1/3 of cases: B = -1.01, p < .001 for autonomy and B = -0.59, p < .001 for performance). Attending and trainee genders were not associated with attending ratings of autonomy or performance. CONCLUSION: Resident autonomy and performance were positively associated with PGY level and week of the academic year, and negatively associated with case complexity and multiple procedures. These findings highlight the need to align training level with case complexity to promote quality operative experiences. LEVEL OF EVIDENCE: 2
