The ability to generalize compositionally is key to understanding the
potentially infinite number of sentences that can be constructed in a human
language from only a finite number of words. Investigating whether NLP models
possess this ability has been a topic of interest: SCAN (Lake and Baroni, 2018)
is one task specifically proposed to test for this property. Previous work has
achieved impressive empirical results using a group-equivariant neural network
that naturally encodes a useful inductive bias for SCAN (Gordon et al., 2020).
Inspired by this, we introduce a novel group-equivariant architecture that
incorporates a group-invariant hard alignment mechanism. We find that our
network's structure allows it to develop stronger equivariance properties than
existing group-equivariant approaches. We additionally find that it outperforms
previous group-equivariant networks empirically on the SCAN task. Our results
suggest that integrating group-equivariance into a variety of neural
architectures is a potentially fruitful avenue of research, and demonstrate the
value of careful analysis of the theoretical properties of such architectures.Comment: Accepted at COLING 202