This paper considers Bayesian inference for the partially linear model. Our
approach exploits a parametrization of the regression function that is tailored
toward estimating a low-dimensional parameter of interest. The key property of
the parametrization is that it generates a Neyman orthogonal moment condition
meaning that the low-dimensional parameter is less sensitive to the estimation
of nuisance parameters. Our large sample analysis supports this claim. In
particular, we derive sufficient conditions under which the posterior for the
low-dimensional parameter contracts around the truth at the parametric rate and
is asymptotically normal with a variance that coincides with the semiparametric
efficiency bound. These conditions allow for a larger class of nuisance
parameters relative to the original parametrization of the regression model.
Overall, we conclude that a parametrization that embeds Neyman orthogonality
can be a useful device for debiasing posterior distributions in semiparametric
models