This paper proposes a framework to formally link a fragment of an algebraic
language to a Graph Neural Network (GNN). It relies on Context Free Grammars
(CFG) to organise algebraic operations into generative rules that can be
translated into a GNN layer model. Since the rules and variables of a CFG
directly derived from a language contain redundancies, a grammar reduction
scheme is presented making tractable the translation into a GNN layer. Applying
this strategy, a grammar compliant with the third-order Weisfeiler-Lehman
(3-WL) test is defined from MATLANG. From this 3-WL CFG, we derive a provably
3-WL GNN model called G2N2. Moreover, this grammatical approach allows us
to provide algebraic formulas to count the cycles of length up to six and
chordal cycles at the edge level, which enlightens the counting power of 3-WL.
Several experiments illustrate that G2N2 efficiently outperforms other
3-WL GNNs on many downstream tasks.Comment: 27 pages, 7 figure