Computing the posterior distribution of a probabilistic program is a hard
task for which no one-fit-for-all solution exists. We propose Gaussian
Semantics, which approximates the exact probabilistic semantics of a bounded
program by means of Gaussian mixtures. It is parametrized by a map that
associates each program location with the moment order to be matched in the
approximation. We provide two main contributions. The first is a universal
approximation theorem stating that, under mild conditions, Gaussian Semantics
can approximate the exact semantics arbitrarily closely. The second is an
approximation that matches up to second-order moments analytically in face of
the generally difficult problem of matching moments of Gaussian mixtures with
arbitrary moment order. We test our second-order Gaussian approximation (SOGA)
on a number of case studies from the literature. We show that it can provide
accurate estimates in models not supported by other approximation methods or
when exact symbolic techniques fail because of complex expressions or
non-simplified integrals. On two notable classes of problems, namely
collaborative filtering and programs involving mixtures of continuous and
discrete distributions, we show that SOGA significantly outperforms alternative
techniques in terms of accuracy and computational time