We study the deformations of twisted harmonic maps f with respect to the
representation ρ. After constructing a continuous "universal" twisted
harmonic map, we give a construction of every first order deformation of f in
terms of Hodge theory; we apply this result to the moduli space of reductive
representations of a K\"ahler group, to show that the critical points of the
energy functional E coincide with the monodromy representations of polarized
complex variations of Hodge structure. We then proceed to second order
deformations, where obstructions arise; we investigate the existence of such
deformations, and give a method for constructing them, as well. Applying this
to the energy functional as above, we prove (for every finitely presented
group) that the energy functional is a potential for the K\"ahler form of the
"Betti" moduli space; assuming furthermore that the group is K\"ahler, we study
the eigenvalues of the Hessian of E at critical points.Comment: 32 pages. Several typos have been corrected and some references have
been added. To appear on Math.