We study variational quantum algorithms from the perspective of free
fermions. By deriving the explicit structure of the associated Lie algebras, we
show that the Quantum Approximate Optimization Algorithm (QAOA) on a
one-dimensional lattice -- with and without decoupled angles -- is able to
prepare all fermionic Gaussian states respecting the symmetries of the circuit.
Leveraging these results, we numerically study the interplay between these
symmetries and the locality of the target state, and find that an absence of
symmetries makes nonlocal states easier to prepare. An efficient classical
simulation of Gaussian states, with system sizes up to 80 and deep circuits,
is employed to study the behavior of the circuit when it is overparameterized.
In this regime of optimization, we find that the number of iterations to
converge to the solution scales linearly with system size. Moreover, we observe
that the number of iterations to converge to the solution decreases
exponentially with the depth of the circuit, until it saturates at a depth
which is quadratic in system size. Finally, we conclude that the improvement in
the optimization can be explained in terms of of better local linear
approximations provided by the gradients
