Investigating diagrammatic reasoning with deep neural networks

Diagrams in mechanised reasoning systems are typically en- coded into symbolic representations that can be easily processed with rule-based expert systems. This relies on human experts to define the framework of diagram-to-symbol mapping and the set of rules to reason with the symbols. We present a new method of using Deep artificial Neu- ral Networks (DNN) to learn continuous, vector-form representations of diagrams without any human input, and entirely from datasets of dia- grammatic reasoning problems. Based on this DNN, we developed a novel reasoning system, Euler-Net, to solve syllogisms with Euler diagrams. Euler-Net takes two Euler diagrams representing the premises in a syl- logism as input, and outputs either a categorical (subset, intersection or disjoint) or diagrammatic conclusion (generating an Euler diagram rep- resenting the conclusion) to the syllogism. Euler-Net can achieve 99.5% accuracy for generating syllogism conclusion. We analyse the learned representations of the diagrams, and show that meaningful information can be extracted from such neural representations. We propose that our framework can be applied to other types of diagrams, especially the ones we don’t know how to formalise symbolically. Furthermore, we propose to investigate the relation between our artificial DNN and human neural circuitry when performing diagrammatic reasoning

Neural Diagrammatic Reasoning

Diagrams have been shown to be effective tools for humans to represent and reason about complex concepts. They have been widely used to represent concepts in science teaching, to communicate workflow in industries and to measure human fluid intelligence. Mechanised reasoning systems typically encode diagrams into symbolic representations that can be easily processed with rule-based expert systems. This relies on human experts to define the framework of diagram-to-symbol mapping and the set of rules to reason with the symbols. This means the reasoning systems cannot be easily adapted to other diagrams without a new set of human-defined representation mapping and reasoning rules. Moreover such systems are not able to cope with diagram inputs as raw and possibly noisy images. The need for human input and the lack of robustness to noise significantly limit the applications of mechanised diagrammatic reasoning systems. A key research question then arises: can we develop human-like reasoning systems that learn to reason robustly without predefined reasoning rules? To answer this question, I propose Neural Diagrammatic Reasoning, a new family of diagrammatic reasoning systems which does not have the drawbacks of mechanised reasoning systems. The new systems are based on deep neural networks, a recently popular machine learning method that achieved human-level performance on a range of perception tasks such as object detection, speech recognition and natural language processing. The proposed systems are able to learn both diagram to symbol mapping and implicit reasoning rules only from data, with no prior human input about symbols and rules in the reasoning tasks. Specifically I developed EulerNet, a novel neural network model that solves Euler diagram syllogism tasks with 99.5% accuracy. Experiments show that EulerNet learns useful representations of the diagrams and tasks, and is robust to noise and deformation in the input data. I also developed MXGNet, a novel multiplex graph neural architecture that solves Raven Progressive Matrices (RPM) tasks. MXGNet achieves state-of-the-art accuracies on two popular RPM datasets. In addition, I developed Discrete-AIR, an unsupervised learning architecture that learns semi-symbolic representations of diagrams without any labels. Lastly I designed a novel inductive bias module that can be readily used in today’s deep neural networks to improve their generalisation capability on relational reasoning tasks.EPSRC Studentship and Cambridge Trust Scholarshi

Characterizing the Shape of Activation Space in Deep Neural Networks

The representations learned by deep neural networks are difficult to interpret in part due to their large parameter space and the complexities introduced by their multi-layer structure. We introduce a method for computing persistent homology over the graphical activation structure of neural networks, which provides access to the task-relevant substructures activated throughout the network for a given input. This topological perspective provides unique insights into the distributed representations encoded by neural networks in terms of the shape of their activation structures. We demonstrate the value of this approach by showing an alternative explanation for the existence of adversarial examples. By studying the topology of network activations across multiple architectures and datasets, we find that adversarial perturbations do not add activations that target the semantic structure of the adversarial class as previously hypothesized. Rather, adversarial examples are explainable as alterations to the dominant activation structures induced by the original image, suggesting the class representations learned by deep networks are problematically sparse on the input space

Abstract Diagrammatic Reasoning with Multiplex Graph Networks

Abstract reasoning, particularly in the visual domain, is a complex human ability, but it remains a challenging problem for artificial neural learning systems. In this work we propose MXGNet, a multilayer graph neural network for multi-panel diagrammatic reasoning tasks. MXGNet combines three powerful concepts, namely, object-level representation, graph neural networks and multiplex graphs, for solving visual reasoning tasks. MXGNet first extracts object-level representations for each element in all panels of the diagrams, and then forms a multi-layer multiplex graph capturing multiple relations between objects across different diagram panels. MXGNet summarises the multiple graphs extracted from the diagrams of the task, and uses this summarisation to pick the most probable answer from the given candidates. We have tested MXGNet on two types of diagrammatic reasoning tasks, namely Diagram Syllogisms and Raven Progressive Matrices (RPM). For an Euler Diagram Syllogism task MXGNet achieves state-of-the-art accuracy of 99.8%. For PGM and RAVEN, two comprehensive datasets for RPM reasoning, MXGNet outperforms the state-of-the-art models by a considerable margin

Artificial general intelligence: Proceedings of the Second Conference on Artificial General Intelligence, AGI 2009, Arlington, Virginia, USA, March 6-9, 2009

Artificial General Intelligence (AGI) research focuses on the original and ultimate goal of AI – to create broad human-like and transhuman intelligence, by exploring all available paths, including theoretical and experimental computer science, cognitive science, neuroscience, and innovative interdisciplinary methodologies. Due to the difficulty of this task, for the last few decades the majority of AI researchers have focused on what has been called narrow AI – the production of AI systems displaying intelligence regarding specific, highly constrained tasks. In recent years, however, more and more researchers have recognized the necessity – and feasibility – of returning to the original goals of the field. Increasingly, there is a call for a transition back to confronting the more difficult issues of human level intelligence and more broadly artificial general intelligence

Novel analysis and modelling methodologies applied to pultrusion and other processes

Often a manufacturing process may be a bottleneck or critical to a business. This thesis focuses on the analysis and modelling of such processest, to both better understand them, and to support the enhancement of quality or output capability of the process. The main thrusts of this thesis therefore are: To model inter-process physics, inter-relationships, and complex processes in a manner that enables re-exploitation, re-interpretation and reuse of this knowledge and generic elements e.g. using Object Oriented (00) & Qualitative Modelling (QM) techniques. This involves the development of superior process models to capture process complexity and reuse any generic elements; To demonstrate advanced modelling and simulation techniques (e.g. Artificial Neural Networks(ANN), Rule-Based-Systems (RBS), and statistical modelling) on a number of complex manufacturing case studies; To gain a better understanding of the physics and process inter-relationships exhibited in a number of complex manufacturing processes (e.g. pultrusion, bioprocess, and logistics) using analysis and modelling. To these ends, both a novel Object Oriented Qualitative (Problem) Analysis (OOQA) methodology, and a novel Artificial Neural Network Process Modelling (ANNPM) methodology were developed and applied to a number of complex manufacturing case studies- thermoset and thermoplastic pultrusion, bioprocess reactor, and a logistics supply chain. It has been shown that these methodologies and the models developed support capture of complex process inter-relationships, enable reuse of generic elements, support effective variable selection for ANN models, and perform well as a predictor of process properties. In particular the ANN pultrusion models, using laboratory data from IKV, Aachen and Pera, Melton Mowbray, predicted product properties very well