In many families of distributions, maximum likelihood estimation is
intractable because the normalization constant for the density which enters
into the likelihood function is not easily available. The score matching
estimator of Hyv\"arinen (2005) provides an alternative where this
normalization constant is not required. The corresponding estimating equations
become linear for an exponential family. The score matching estimator is shown
to be consistent and asymptotically normally distributed for such models,
although not necessarily efficient. Gaussian linear concentration models are
examples of such families. For linear concentration models that are also linear
in the covariance we show that the score matching estimator is identical to the
maximum likelihood estimator, hence in such cases it is also efficient.
Gaussian graphical models and graphical models with symmetries form
particularly interesting subclasses of linear concentration models and we
investigate the potential use of the score matching estimator for this case