We address a classical problem in statistics: adding two-way interaction
terms to a regression model. As the covariate dimension increases
quadratically, we develop an estimator that adapts well to this increase, while
providing accurate estimates and appropriate inference. Existing strategies
overcome the dimensionality problem by only allowing interactions between
relevant main effects. Building on this philosophy, we implement a softer link
between the two types of effects using a local shrinkage model. We empirically
show that borrowing strength between the amount of shrinkage for main effects
and their interactions can strongly improve estimation of the regression
coefficients. Moreover, we evaluate the potential of the model for inference,
which is notoriously hard for selection strategies. Large-scale cohort data are
used to provide realistic illustrations and evaluations. Comparisons with other
methods are provided. The evaluation of variable importance is not trivial in
regression models with many interaction terms. Therefore, we derive a new
analytical formula for the Shapley value, which enables rapid assessment of
individual-specific variable importance scores and their uncertainties.
Finally, while not targeting for prediction, we do show that our models can be
very competitive to a more advanced machine learner, like random forest, even
for fairly large sample sizes. The implementation of our method in RStan is
fairly straightforward, allowing for adjustments to specific needs.Comment: 28 pages, 18 figure