High level cognitive information processing in neural networks

Two related research efforts were addressed: (1) high-level connectionist cognitive modeling; and (2) local neural circuit modeling. The goals of the first effort were to develop connectionist models of high-level cognitive processes such as problem solving or natural language understanding, and to understand the computational requirements of such models. The goals of the second effort were to develop biologically-realistic model of local neural circuits, and to understand the computational behavior of such models. In keeping with the nature of NASA's Innovative Research Program, all the work conducted under the grant was highly innovative. For instance, the following ideas, all summarized, are contributions to the study of connectionist/neural networks: (1) the temporal-winner-take-all, relative-position encoding, and pattern-similarity association techniques; (2) the importation of logical combinators into connection; (3) the use of analogy-based reasoning as a bridge across the gap between the traditional symbolic paradigm and the connectionist paradigm; and (4) the application of connectionism to the domain of belief representation/reasoning. The work on local neural circuit modeling also departs significantly from the work of related researchers. In particular, its concentration on low-level neural phenomena that could support high-level cognitive processing is unusual within the area of biological local circuit modeling, and also serves to expand the horizons of the artificial neural net field

Beyond the icon: Core cognition and the bounds of perception

This paper refines a controversial proposal: that core systems belong to a perceptual kind, marked out by the format of its representational outputs. Following Susan Carey, this proposal has been understood in terms of core representations having an iconic format, like certain paradigmatically perceptual outputs. I argue that they don’t, but suggest that the proposal may be better formulated in terms of a broader analogue format type. Formulated in this way, the proposal accommodates the existence of genuine icons in perception, and avoids otherwise troubling objections

Holistic processing of hierarchical structures in connectionist networks

Despite the success of connectionist systems to model some aspects of cognition, critics argue that the lack of symbol processing makes them inadequate for modelling high-level cognitive tasks which require the representation and processing of hierarchical structures. In this thesis we investigate four mechanisms for encoding hierarchical structures in distributed representations that are suitable for processing in connectionist systems: Tensor Product Representation, Recursive Auto-Associative Memory (RAAM), Holographic Reduced Representation (HRR), and Binary Spatter Code (BSC). In these four schemes representations of hierarchical structures are either learned in a connectionist network or constructed by means of various mathematical operations from binary or real-value vectors.It is argued that the resulting representations carry structural information without being themselves syntactically structured. The structural information about a represented object is encoded in the position of its representation in a high-dimensional representational space. We use Principal Component Analysis and constructivist networks to show that well-separated clusters consisting of representations for structurally similar hierarchical objects are formed in the representational spaces of RAAMs and HRRs. The spatial structure of HRRs and RAAM representations supports the holistic yet structure-sensitive processing of them. Holistic operations on RAAM representations can be learned by backpropagation networks. However, holistic operators over HRRs, Tensor Products, and BSCs have to be constructed by hand, which is not a desirable situation. We propose two new algorithms for learning holistic transformations of HRRs from examples. These algorithms are able to generalise the acquired knowledge to hierarchical objects of higher complexity than the training examples. Such generalisations exhibit systematicity of a degree which, to our best knowledge, has not yet been achieved by any other comparable learning method.Finally, we outline how a number of holistic transformations can be learned in parallel and applied to representations of structurally different objects. The ability to distinguish and perform a number of different structure-sensitive operations is one step towards a connectionist architecture that is capable of modelling complex high-level cognitive tasks such as natural language processing and logical inference

A Defense of Pure Connectionism

Connectionism is an approach to neural-networks-based cognitive modeling that encompasses the recent deep learning movement in artificial intelligence. It came of age in the 1980s, with its roots in cybernetics and earlier attempts to model the brain as a system of simple parallel processors. Connectionist models center on statistical inference within neural networks with empirically learnable parameters, which can be represented as graphical models. More recent approaches focus on learning and inference within hierarchical generative models. Contra influential and ongoing critiques, I argue in this dissertation that the connectionist approach to cognitive science possesses in principle (and, as is becoming increasingly clear, in practice) the resources to model even the most rich and distinctly human cognitive capacities, such as abstract, conceptual thought and natural language comprehension and production. Consonant with much previous philosophical work on connectionism, I argue that a core principle—that proximal representations in a vector space have similar semantic values—is the key to a successful connectionist account of the systematicity and productivity of thought, language, and other core cognitive phenomena. My work here differs from preceding work in philosophy in several respects: (1) I compare a wide variety of connectionist responses to the systematicity challenge and isolate two main strands that are both historically important and reflected in ongoing work today: (a) vector symbolic architectures and (b) (compositional) vector space semantic models; (2) I consider very recent applications of these approaches, including their deployment on large-scale machine learning tasks such as machine translation; (3) I argue, again on the basis mostly of recent developments, for a continuity in representation and processing across natural language, image processing and other domains; (4) I explicitly link broad, abstract features of connectionist representation to recent proposals in cognitive science similar in spirit, such as hierarchical Bayesian and free energy minimization approaches, and offer a single rebuttal of criticisms of these related paradigms; (5) I critique recent alternative proposals that argue for a hybrid Classical (i.e. serial symbolic)/statistical model of mind; (6) I argue that defending the most plausible form of a connectionist cognitive architecture requires rethinking certain distinctions that have figured prominently in the history of the philosophy of mind and language, such as that between word- and phrase-level semantic content, and between inference and association

Classical and connectionist models of cognition

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Linguistics and Philosophy, 1990.Includes bibliographical references (leaves 206-208).by Eric Paul Lormand.Ph.D

Crossing the symbolic threshold: a critical review of Terrence Deacon's The Symbolic Species

Terrence Deacon's views about the origin of language are based on a particular notion of a symbol. While the notion is derived from Peirce's semiotics, it diverges from that source and needs to be investigated on its own terms in order to evaluate the idea that the human species has crossed the symbolic threshold. Deacon's view is defended from the view that symbols in the animal world are widespread and from the extreme connectionist view that they are not even to be found in humans. Deacon's treatment of symbols involves a form of holism, as a symbol needs to be part of a system of symbols. He also appears to take a realist view of symbols. That combination of holism and realism makes the threshold a sharp threshold, which makes it hard to explain how the threshold was crossed. This difficulty is overcome if we take a mild realist position towards symbols, in the style of Dennett. Mild realism allows intermediate stages in the crossing but does not undermine Deacon's claim that the threshold is difficult to cross or the claim that it needs to be crossed quickly

Of One Mind: Proposal for a Non-Cartesian Cognitive Architecture

Intellectually, we may reject Cartesian Dualism, but dualism often dominates our everyday thinking: we talk of “mental” illness as though it were non-physical; we tend to blame people for the symptoms of brain malfunctions in a way that differs from how we treat other illnesses. An examination of current theories of mind will reveal that some form of dualism is not always limited to the non-scientific realm. While very few, if any, cognitive scientists support mind-body dualism, those who support the view of the mind as a symbol-manipulator are often constrained to postulate more than one cognitive system in response to the failure of the symbol-system model to account for all aspects of human cognition. In this dissertation, I argue for an empiricist, rather than a realist, theory of perception, for an internalist semantics, and for a model of cognitive architecture which combines a connectionist approach with highly-specialized, symbolic, computational component which includes functions that provide input to a a causally-inert conscious mind. I reject the symbol-system hypothesis and propose a cognitive architecture which, I contend, is biologically-plausible and more consistent with the results of recent neuroscientific studies. This hybrid model can accommodate the processes commonly discussed by dual-process theorists and can also accommodate the processes which have proved to be so problematic for models based on the symbol-system hypothesis