184 research outputs found
Abstract Diagrammatic Reasoning with Multiplex Graph Networks
Abstract reasoning, particularly in the visual domain, is a complex human ability, but it remains a challenging problem for artificial neural learning systems. In this work we propose MXGNet, a multilayer graph neural network for multi-panel diagrammatic reasoning tasks. MXGNet combines three powerful concepts, namely, object-level representation, graph neural networks and multiplex graphs, for solving visual reasoning tasks. MXGNet first extracts object-level representations for each element in all panels of the diagrams, and then forms a multi-layer multiplex graph capturing multiple relations between objects across different diagram panels. MXGNet summarises the multiple graphs extracted from the diagrams of the task, and uses this summarisation to pick the most probable answer from the given candidates. We have tested MXGNet on two types of diagrammatic reasoning tasks, namely Diagram Syllogisms and Raven Progressive Matrices (RPM). For an Euler Diagram Syllogism task MXGNet achieves state-of-the-art accuracy of 99.8%. For PGM and RAVEN, two comprehensive datasets for RPM reasoning, MXGNet outperforms the state-of-the-art models by a considerable margin
Recommended from our members
Neural Diagrammatic Reasoning
Diagrams have been shown to be effective tools for humans to represent and reason about
complex concepts. They have been widely used to represent concepts in science teaching, to
communicate workflow in industries and to measure human fluid intelligence. Mechanised
reasoning systems typically encode diagrams into symbolic representations that can be
easily processed with rule-based expert systems. This relies on human experts to define the
framework of diagram-to-symbol mapping and the set of rules to reason with the symbols.
This means the reasoning systems cannot be easily adapted to other diagrams without
a new set of human-defined representation mapping and reasoning rules. Moreover such
systems are not able to cope with diagram inputs as raw and possibly noisy images. The
need for human input and the lack of robustness to noise significantly limit the applications
of mechanised diagrammatic reasoning systems.
A key research question then arises: can we develop human-like reasoning systems that
learn to reason robustly without predefined reasoning rules? To answer this question, I
propose Neural Diagrammatic Reasoning, a new family of diagrammatic reasoning
systems which does not have the drawbacks of mechanised reasoning systems. The new
systems are based on deep neural networks, a recently popular machine learning method
that achieved human-level performance on a range of perception tasks such as object
detection, speech recognition and natural language processing. The proposed systems are
able to learn both diagram to symbol mapping and implicit reasoning rules only from data,
with no prior human input about symbols and rules in the reasoning tasks. Specifically I
developed EulerNet, a novel neural network model that solves Euler diagram syllogism
tasks with 99.5% accuracy. Experiments show that EulerNet learns useful representations
of the diagrams and tasks, and is robust to noise and deformation in the input data. I
also developed MXGNet, a novel multiplex graph neural architecture that solves Raven
Progressive Matrices (RPM) tasks. MXGNet achieves state-of-the-art accuracies on two
popular RPM datasets. In addition, I developed Discrete-AIR, an unsupervised learning
architecture that learns semi-symbolic representations of diagrams without any labels.
Lastly I designed a novel inductive bias module that can be readily used in today’s deep
neural networks to improve their generalisation capability on relational reasoning tasks.EPSRC Studentship and Cambridge Trust Scholarshi
Extrapolatable Relational Reasoning With Comparators in Low-Dimensional Manifolds
While modern deep neural architectures generalise well when test data is sampled from the same distribution as training data, they fail badly for cases when the test data distribution differs from the training distribution even along a few dimensions. This lack of out-of-distribution generalisation is increasingly manifested when the tasks become more abstract and complex, such as in relational reasoning. In this paper we propose a neuroscience-inspired inductive-biased module that can be readily amalgamated with current neural network architectures to improve out-of-distribution (o.o.d) generalisation performance on relational reasoning tasks. This module learns to project high-dimensional object representations to low-dimensional manifolds for more efficient and generalisable relational comparisons. We show that neural nets with this inductive bias achieve considerably better o.o.d generalisation performance for a range of relational reasoning tasks. We finally analyse the proposed inductive bias module to understand the importance of lower dimension projection, and propose an augmentation to the algorithmic alignment theory to better measure algorithmic alignment with generalisation
Topological Deep Learning: Going Beyond Graph Data
Topological deep learning is a rapidly growing field that pertains to the
development of deep learning models for data supported on topological domains
such as simplicial complexes, cell complexes, and hypergraphs, which generalize
many domains encountered in scientific computations. In this paper, we present
a unifying deep learning framework built upon a richer data structure that
includes widely adopted topological domains.
Specifically, we first introduce combinatorial complexes, a novel type of
topological domain. Combinatorial complexes can be seen as generalizations of
graphs that maintain certain desirable properties. Similar to hypergraphs,
combinatorial complexes impose no constraints on the set of relations. In
addition, combinatorial complexes permit the construction of hierarchical
higher-order relations, analogous to those found in simplicial and cell
complexes. Thus, combinatorial complexes generalize and combine useful traits
of both hypergraphs and cell complexes, which have emerged as two promising
abstractions that facilitate the generalization of graph neural networks to
topological spaces.
Second, building upon combinatorial complexes and their rich combinatorial
and algebraic structure, we develop a general class of message-passing
combinatorial complex neural networks (CCNNs), focusing primarily on
attention-based CCNNs. We characterize permutation and orientation
equivariances of CCNNs, and discuss pooling and unpooling operations within
CCNNs in detail.
Third, we evaluate the performance of CCNNs on tasks related to mesh shape
analysis and graph learning. Our experiments demonstrate that CCNNs have
competitive performance as compared to state-of-the-art deep learning models
specifically tailored to the same tasks. Our findings demonstrate the
advantages of incorporating higher-order relations into deep learning models in
different applications
Analysis and Visualisation of Edge Entanglement in Multiplex Networks
Cette thèse présente une nouvelle méthodologie pour analyser des réseaux. Nous développons l'intrication d'un réseau multiplex, qui se matérialise sous forme d'une mesure d'intensité et d'homogénéité, et d'une abstraction, le réseau d'interaction des catalyseurs, auxquels sont associés des indices d'intrication. Nous présentons ensuite la mise en place d'outils spécifiques pour l'analyse visuelle des réseaux complexes qui tirent profit de cette méthodologie. Ces outils présente une vue double de deux réseaux,qui inclue une un algorithme de dessin, une interaction associant brossage d'une sélection et de multiples liens pré-attentifs. Nous terminons ce document par la présentation détaillée d'applications dans de multiples domaines.When it comes to comprehension of complex phenomena, humans need to understand what interactions lie within them.These interactions are often captured with complex networks. However, the interaction pluralism is often shallowed by traditional network models. We propose a new way to look at these phenomena through the lens of multiplex networks, in which catalysts are drivers of the interaction through substrates. To study the entanglement of a multiplex network is to study how edges intertwine, in other words, how catalysts interact. Our entanglement analysis results in a full set of new objects which completes traditional network approaches: the entanglement homogeneity and intensity of the multiplex network, and the catalyst interaction network, with for each catalyst, an entanglement index. These objects are very suitable for embedment in a visual analytics framework, to enable comprehension of a complex structure. We thus propose of visual setting with coordinated multiple views. We take advantage of mental mapping and visual linking to present simultaneous information of a multiplex network at three different levels of abstraction. We complete brushing and linking with a leapfrog interaction that mimics the back-and-forth process involved in users' comprehension. The method is validated and enriched through multiple applications including assessing group cohesion in document collections, and identification of particular associations in social networks.BORDEAUX1-Bib.electronique (335229901) / SudocSudocFranceF
- …