We consider three Bayesian penalized regression models and show that the
respective deterministic scan Gibbs samplers are geometrically ergodic
regardless of the dimension of the regression problem. We prove geometric
ergodicity of the Gibbs samplers for the Bayesian fused lasso, the Bayesian
group lasso, and the Bayesian sparse group lasso. Geometric ergodicity along
with a moment condition results in the existence of a Markov chain central
limit theorem for Monte Carlo averages and ensures reliable output analysis.
Our results of geometric ergodicity allow us to also provide default starting
values for the Gibbs samplers