Biologically Plausible Connectionist Prediction of Natural Language Thematic Relations

In Natural Language Processing (NLP) symbolic systems, several linguistic phenomena, for instance, the thematic role relationships between sentence constituents, such as AGENT, PATIENT, and LOCATION, can be accounted for by the employment of a rule-based grammar. Another approach to NLP concerns the use of the connectionist model, which has the benefits of learning, generalization and fault tolerance, among others. A third option merges the two previous approaches into a hybrid one: a symbolic thematic theory is used to supply the connectionist network with initial knowledge. Inspired on neuroscience, it is proposed a symbolic-connectionist hybrid system called BIO theta PRED (BIOlogically plausible thematic (theta) symbolic-connectionist PREDictor), designed to reveal the thematic grid assigned to a sentence. Its connectionist architecture comprises, as input, a featural representation of the words (based on the verb/noun WordNet classification and on the classical semantic microfeature representation), and, as output, the thematic grid assigned to the sentence. BIO theta PRED is designed to ""predict"" thematic (semantic) roles assigned to words in a sentence context, employing biologically inspired training algorithm and architecture, and adopting a psycholinguistic view of thematic theory.Fapesp - Fundacao de Amparo a Pesquisa do Estado de Sao Paulo, Brazil[2008/08245-4

A Defense of Pure Connectionism

Connectionism is an approach to neural-networks-based cognitive modeling that encompasses the recent deep learning movement in artificial intelligence. It came of age in the 1980s, with its roots in cybernetics and earlier attempts to model the brain as a system of simple parallel processors. Connectionist models center on statistical inference within neural networks with empirically learnable parameters, which can be represented as graphical models. More recent approaches focus on learning and inference within hierarchical generative models. Contra influential and ongoing critiques, I argue in this dissertation that the connectionist approach to cognitive science possesses in principle (and, as is becoming increasingly clear, in practice) the resources to model even the most rich and distinctly human cognitive capacities, such as abstract, conceptual thought and natural language comprehension and production. Consonant with much previous philosophical work on connectionism, I argue that a core principle—that proximal representations in a vector space have similar semantic values—is the key to a successful connectionist account of the systematicity and productivity of thought, language, and other core cognitive phenomena. My work here differs from preceding work in philosophy in several respects: (1) I compare a wide variety of connectionist responses to the systematicity challenge and isolate two main strands that are both historically important and reflected in ongoing work today: (a) vector symbolic architectures and (b) (compositional) vector space semantic models; (2) I consider very recent applications of these approaches, including their deployment on large-scale machine learning tasks such as machine translation; (3) I argue, again on the basis mostly of recent developments, for a continuity in representation and processing across natural language, image processing and other domains; (4) I explicitly link broad, abstract features of connectionist representation to recent proposals in cognitive science similar in spirit, such as hierarchical Bayesian and free energy minimization approaches, and offer a single rebuttal of criticisms of these related paradigms; (5) I critique recent alternative proposals that argue for a hybrid Classical (i.e. serial symbolic)/statistical model of mind; (6) I argue that defending the most plausible form of a connectionist cognitive architecture requires rethinking certain distinctions that have figured prominently in the history of the philosophy of mind and language, such as that between word- and phrase-level semantic content, and between inference and association

Neurocognitive Informatics Manifesto.

Informatics studies all aspects of the structure of natural and artificial information systems. Theoretical and abstract approaches to information have made great advances, but human information processing is still unmatched in many areas, including information management, representation and understanding. Neurocognitive informatics is a new, emerging field that should help to improve the matching of artificial and natural systems, and inspire better computational algorithms to solve problems that are still beyond the reach of machines. In this position paper examples of neurocognitive inspirations and promising directions in this area are given

A Competitve Attachment Model for Resolving Syntactic Ambiguities in Natural Language Parsing

Linguistic ambiguity is the greatest obstacle to achieving practical computational systems for natural language understanding. By contrast, people experience surprisingly little difficulty in interpreting ambiguous linguistic input. This dissertation explores distributed computational techniques for mimicking the human ability to resolve syntactic ambiguities efficiently and effectively. The competitive attachment theory of parsing formulates the processing of an ambiguity as a competition for activation within a hybrid connectionist network. Determining the grammaticality of an input relies on a new approach to distributed communication that integrates numeric and symbolic constraints on passing features through the parsing network. The method establishes syntactic relations both incrementally and efficiently, and underlies the ability of the model to establish long-distance syntactic relations using only local communication within a network. The competitive distribution of numeric evidence focuses the activation of the network onto a particular structural interpretation of the input, resolving ambiguities. In contrast to previous approaches to ambiguity resolution, the model makes no use of explicit preference heuristics or revision strategies. Crucially, the structural decisions of the model conform with human preferences, without those preferences having been incorporated explicitly into the parser. Furthermore, the competitive dynamics of the parsing network account for additional on-line processing data that other models of syntactic preferences have left unaddressed. (Also cross-referenced as UMIACS-TR-95-55

Of One Mind: Proposal for a Non-Cartesian Cognitive Architecture

Intellectually, we may reject Cartesian Dualism, but dualism often dominates our everyday thinking: we talk of “mental” illness as though it were non-physical; we tend to blame people for the symptoms of brain malfunctions in a way that differs from how we treat other illnesses. An examination of current theories of mind will reveal that some form of dualism is not always limited to the non-scientific realm. While very few, if any, cognitive scientists support mind-body dualism, those who support the view of the mind as a symbol-manipulator are often constrained to postulate more than one cognitive system in response to the failure of the symbol-system model to account for all aspects of human cognition. In this dissertation, I argue for an empiricist, rather than a realist, theory of perception, for an internalist semantics, and for a model of cognitive architecture which combines a connectionist approach with highly-specialized, symbolic, computational component which includes functions that provide input to a a causally-inert conscious mind. I reject the symbol-system hypothesis and propose a cognitive architecture which, I contend, is biologically-plausible and more consistent with the results of recent neuroscientific studies. This hybrid model can accommodate the processes commonly discussed by dual-process theorists and can also accommodate the processes which have proved to be so problematic for models based on the symbol-system hypothesis