Using a leading-order semiclassical approximation, we calculate the third- A nd fourth-order virial coefficients of nonrelativistic spin-1/2 fermions in a harmonic trapping potential in arbitrary spatial dimensions, and as functions of temperature, trapping frequency, and coupling strength. Our simple, analytic results for the interaction-induced changes Δb3 and Δb4 agree qualitatively, and in some regimes quantitatively, with previous numerical calculations for the unitary limit of three-dimensional Fermi gases