On the Systematicity of Internal Representations in Connectionist Networks

Abstract

It is frequently claimed that connectionist architectures, because they cannot support structured items, cannot provide an alternative cognitive architecture. For an architecture to be cognitive, there must be systematicity (as well as compositionality and productivity), and systematicity, it is claimed, depends crucially on the ability of the system to represent and use structured items. This paper introduces the idea of grouped hyperplanes, where such groups designate abstractions located in an n-dimensional space criss-crossed by simple decision lines. This lead to the idea of emergent primitives supporting complex internal representations, both of which --- in addition to exemplifying characteristics of connectionist systematicity --- are grounded in computable mathematical regions in a hyperspace. 1 Systematicity and compositionality Symbolists and connectionists can agree that any non-trivial cognitive system must be able to represent complex structured items (CSIs). A CSI is ty..