Analysis of variance (ANOVA) is an extremely important method in exploratory
and confirmatory data analysis. Unfortunately, in complex problems (e.g.,
split-plot designs), it is not always easy to set up an appropriate ANOVA. We
propose a hierarchical analysis that automatically gives the correct ANOVA
comparisons even in complex scenarios. The inferences for all means and
variances are performed under a model with a separate batch of effects for each
row of the ANOVA table. We connect to classical ANOVA by working with
finite-sample variance components: fixed and random effects models are
characterized by inferences about existing levels of a factor and new levels,
respectively. We also introduce a new graphical display showing inferences
about the standard deviations of each batch of effects. We illustrate with two
examples from our applied data analysis, first illustrating the usefulness of
our hierarchical computations and displays, and second showing how the ideas of
ANOVA are helpful in understanding a previously fit hierarchical model.Comment: This paper discussed in: [math.ST/0508526], [math.ST/0508527],
[math.ST/0508528], [math.ST/0508529]. Rejoinder in [math.ST/0508530