A Defense of Pure Connectionism

Connectionism is an approach to neural-networks-based cognitive modeling that encompasses the recent deep learning movement in artificial intelligence. It came of age in the 1980s, with its roots in cybernetics and earlier attempts to model the brain as a system of simple parallel processors. Connectionist models center on statistical inference within neural networks with empirically learnable parameters, which can be represented as graphical models. More recent approaches focus on learning and inference within hierarchical generative models. Contra influential and ongoing critiques, I argue in this dissertation that the connectionist approach to cognitive science possesses in principle (and, as is becoming increasingly clear, in practice) the resources to model even the most rich and distinctly human cognitive capacities, such as abstract, conceptual thought and natural language comprehension and production. Consonant with much previous philosophical work on connectionism, I argue that a core principle—that proximal representations in a vector space have similar semantic values—is the key to a successful connectionist account of the systematicity and productivity of thought, language, and other core cognitive phenomena. My work here differs from preceding work in philosophy in several respects: (1) I compare a wide variety of connectionist responses to the systematicity challenge and isolate two main strands that are both historically important and reflected in ongoing work today: (a) vector symbolic architectures and (b) (compositional) vector space semantic models; (2) I consider very recent applications of these approaches, including their deployment on large-scale machine learning tasks such as machine translation; (3) I argue, again on the basis mostly of recent developments, for a continuity in representation and processing across natural language, image processing and other domains; (4) I explicitly link broad, abstract features of connectionist representation to recent proposals in cognitive science similar in spirit, such as hierarchical Bayesian and free energy minimization approaches, and offer a single rebuttal of criticisms of these related paradigms; (5) I critique recent alternative proposals that argue for a hybrid Classical (i.e. serial symbolic)/statistical model of mind; (6) I argue that defending the most plausible form of a connectionist cognitive architecture requires rethinking certain distinctions that have figured prominently in the history of the philosophy of mind and language, such as that between word- and phrase-level semantic content, and between inference and association

Knowledge-based decision model construction for hierarchical diagnosis and repair

Knowledge-Based Model Construction (KBMC) has generated a lot of attention due to its importance as a technique for generating probabilistic or decision-theoretic models whose range of applicability in AI has been vastly increased. However, no one has tried to analyze the essential issues in KBMC, to determine if there exists a general efficient KBMC method for any problem domain, or to y identify the fruitful future research on KBMC. This research presents a unified framework for comparative analysis of KBMC systems identifying the essential issues in KBMC, showing that there is no such general efficient KBMC method, and listing the fruitful future research on KBMC. This thesis then presents a new KBMC mechanism for hierarchical diagnosis and repair. Diagnosis is formulated as a stochastic process and modeled using influence diagrams. In the best case using an abstraction hierarchy in problem-solving can yield an exponential speedup in search efficiency. However, this speedup assumes backtracking never occurs across abstraction levels. When this assumption fails, search may have to consider different abstract solutions before finding one that can be refined to a base solution, and, therefore, search efficiency is not necessarily improved. In this thesis, we present a decision model construction method for hierarchical diagnosis and repair. We show analytically and experimentally that our method always yields a significant speedup in search efficiency, and that hierarchies with smaller branching factors yield more significant efficiency gains. This thesis employs two causal pathways (functional and bridge fault) of domain knowledge in device trouble shooting, preventing either whole class of faults we will never be able to diagnose. Each causal pathway models the knowledge of adjacency and behavior within the corresponding interaction layer. Careful search of causal pathways allows us to restrict the search space of fault hypotheses at each time. We model this search among causal pathways decision-theoretically. Decision-theoretic control usually results in significant improvements over unaided human expert judgments. Furthermore, these improvements in performance are robust to substantial errors in the assessed costs and probabilities

Using First-Order Probability Logic for the Construction of Bayesian Networks

We present a mechanism for constructing graphical models, specifically Bayesian networks, from a knowledge base of general probabilistic information. The unique feature of our approach is that it uses a powerful first-order probabilistic logic for expressing the general knowledge base. This logic allows for the representation of a wide range of logical and probabilistic information. The model construction procedure we propose uses notions from direct inference to identify pieces of local statistical information from the knowledge base that are most appropriate to the particular event we want to reason about. These pieces are composed to generate a joint probability distribution specified as a Bayesian network. Although there are fundamental difficulties in dealing with fully general knowledge, our procedure is practical for quite rich knowledge bases and it supports the construction of a far wider range of networks than allowed for by current template technology. 1 Introduction The de..