We report on the broadband transient optical response from anisotropic
nanobrick amorphous silicon particles, exhibiting Mie-type resonances. A
quantitative model is developed to identify and disentangle the three physical
processes that govern the ultrafast changes of the nanobrick optical
properties, namely two-photon absorption, free-carrier relaxation, and lattice
heating. We reveal a set of operating windows where ultrafast all-optical
modulation of transmission is achieved with full return to zero in 20 ps. This
is made possible due to the interplay between the competing nonlinear processes
and despite the slow (nanosecond) internal lattice dynamics. The observed
ultrafast switching behavior can be independently engineered for both or-
thogonal polarizations using the large anisotropy of nanobricks thus allowing
ultrafast anisotropy control. Our results categorically ascertain the potential
of all-dielectric resonant nanophotonics as a platform for ultrafast optical
devices, and reveal the pos- sibility for ultrafast polarization-multiplexed
displays and polarization rotators