From Simple Associations to Systemic Reasoning: A Connectionist Representation of Rules, Variables and Dynamic Bindings

Human agents draw a variety of inferences effortlessly, spontaneously, and with remarkable efficiency - as though these inferences are a reflex response of their cognitive apparatus. The work presented in this paper is a step toward a computational account of this remarkable reasoning ability. We describe how a connectionist system made up of simple and slow neuron-like elements can encode millions of facts and rules involving n-ary predicates and variables, and yet perform a variety of inferences within hundreds of milliseconds. We observe that an efficient reasoning system must represent and propagate, dynamically, a large number of variable bindings. The proposed system does so by propagating rhythmic patterns of activity wherein dynamic bindings are represented as the in-phase, i.e., synchronous, firing of appropriate nodes. The mechanisms for representing and propagating dynamic bindings are biologically plausible. Neurophysiological evidence suggests that similar mechanisms may in fact be used by the brain to represent and process sensorimotor information

A Connectionist System for Rule Based Reasoning With Multi-Place Predicates and Variables

McCarthy has observed that the representational power of most connectionist systems is restricted to unary predicates applied to a fixed object. More recently, Fodor and Pylyshyn have made a sweeping claim that connectionist systems cannot incorporate systematicity and compositionality. These comments suggest that representing structured knowledge in a connectionist network and using this knowledge in a systematic way is considered difficult if not impossible. The work reported in this paper demonstrates that a connectionist system can not only represent structured knowledge and display systematic behavior, but it can also do so with extreme efficiency. The paper describes a connectionist system that can represent knowledge expressed as rules and facts involving multi-place predicates (i.e., n-ary relations), and draw limited, but sound, inferences based on this knowledge. The system is extremely efficient - in fact, optimal, as it draws conclusions in time proportional to the length of the proof. It is observed that representing and reasoning with structured knowledge requires a solution to the variable binding problem. A solution to this problem using a multi-phase clock is proposed. The solution allows the system to maintain and propagate an arbitrary number of variable bindings during the reasoning process. The work also identifies constraints on the structure of inferential dependencies and the nature of quantification in individual rules that are required for efficient reasoning. These constraints may eventually help in modelling the remarkable human ability of performing certain inferences with extreme efficiency

Rule-Based Reasoning in Connectionist Networks

This thesis addresses the problem of efficiently representing large knowledge bases and performing a class of inferences extremely fast. The speed of reasoning depends on a number of factors including the expressiveness of the system, the nature of the computational architecture and the representation methodology. A number of knowledge representation and reasoning schemes have given very high emphasis to just one of such issues while neglecting others. This dissertation work is based on the belief that it is beneficial to take an approach where all such issues are simultaneously addressed. With respect to the issue of computational architecture, it is argued that a connectionist architecture has some significant advantages. Having made that choice, we explore how to represent and reason with rules involving multi-place predicates and variables in a connectionist architecture. The main hurdle that needs to be crossed in order to be able achieve this is the dynamic binding problem. In essence, the problem is that of representing the dynamic grouping of nodes located in different parts of the network. We use what we refer to as the synchronous activation approach to solve the binding problem. Simply stated, the idea is just that the dynamic grouping of a set of nodes is represented by the fact that all those nodes fire synchronously. This happens to be a solution that is technically attractive as well as biologically plausible. Incorporating the synchronous activation approach to solve the binding problem, rulebased forward and backward reasoning systems have been designed to perform deductive inferences. These systems represent knowledge very efficiently: the number of nodes and links required is only linear in the size of the knowledge base. They also perform inferences extremely fast: an inference takes time that is just linear in the length of the shortest proof. We also examine various ways of extending the expressiveness and reasoning abilities of these systems. An alternative representation scheme more amenable to learning is also presented along with a proposal for doing abductive reasoning in connectionist networks