8 research outputs found
Graphs in machine learning: an introduction
Graphs are commonly used to characterise interactions between objects of
interest. Because they are based on a straightforward formalism, they are used
in many scientific fields from computer science to historical sciences. In this
paper, we give an introduction to some methods relying on graphs for learning.
This includes both unsupervised and supervised methods. Unsupervised learning
algorithms usually aim at visualising graphs in latent spaces and/or clustering
the nodes. Both focus on extracting knowledge from graph topologies. While most
existing techniques are only applicable to static graphs, where edges do not
evolve through time, recent developments have shown that they could be extended
to deal with evolving networks. In a supervised context, one generally aims at
inferring labels or numerical values attached to nodes using both the graph
and, when they are available, node characteristics. Balancing the two sources
of information can be challenging, especially as they can disagree locally or
globally. In both contexts, supervised and un-supervised, data can be
relational (augmented with one or several global graphs) as described above, or
graph valued. In this latter case, each object of interest is given as a full
graph (possibly completed by other characteristics). In this context, natural
tasks include graph clustering (as in producing clusters of graphs rather than
clusters of nodes in a single graph), graph classification, etc. 1 Real
networks One of the first practical studies on graphs can be dated back to the
original work of Moreno [51] in the 30s. Since then, there has been a growing
interest in graph analysis associated with strong developments in the modelling
and the processing of these data. Graphs are now used in many scientific
fields. In Biology [54, 2, 7], for instance, metabolic networks can describe
pathways of biochemical reactions [41], while in social sciences networks are
used to represent relation ties between actors [66, 56, 36, 34]. Other examples
include powergrids [71] and the web [75]. Recently, networks have also been
considered in other areas such as geography [22] and history [59, 39]. In
machine learning, networks are seen as powerful tools to model problems in
order to extract information from data and for prediction purposes. This is the
object of this paper. For more complete surveys, we refer to [28, 62, 49, 45].
In this section, we introduce notations and highlight properties shared by most
real networks. In Section 2, we then consider methods aiming at extracting
information from a unique network. We will particularly focus on clustering
methods where the goal is to find clusters of vertices. Finally, in Section 3,
techniques that take a series of networks into account, where each network i
Neural Networks for Complex Data
Artificial neural networks are simple and efficient machine learning tools.
Defined originally in the traditional setting of simple vector data, neural
network models have evolved to address more and more difficulties of complex
real world problems, ranging from time evolving data to sophisticated data
structures such as graphs and functions. This paper summarizes advances on
those themes from the last decade, with a focus on results obtained by members
of the SAMM team of Universit\'e Paris
Reservoir Computing for Learning in Structured Domains
The study of learning models for direct processing complex data structures has gained an
increasing interest within the Machine Learning (ML) community during the last decades.
In this concern, efficiency, effectiveness and adaptivity of the ML models on large classes
of data structures represent challenging and open research issues.
The paradigm under consideration is Reservoir Computing (RC), a novel and extremely
efficient methodology for modeling Recurrent Neural Networks (RNN) for adaptive
sequence processing. RC comprises a number of different neural models, among which the
Echo State Network (ESN) probably represents the most popular, used and studied one.
Another research area of interest is represented by Recursive Neural Networks (RecNNs),
constituting a class of neural network models recently proposed for dealing with
hierarchical data structures directly.
In this thesis the RC paradigm is investigated and suitably generalized in order to
approach the problems arising from learning in structured domains. The research studies
described in this thesis cover classes of data structures characterized by increasing
complexity, from sequences, to trees and graphs structures. Accordingly, the research focus
goes progressively from the analysis of standard ESNs for sequence processing, to the
development of new models for trees and graphs structured domains. The analysis of ESNs
for sequence processing addresses the interesting problem of identifying and
characterizing the relevant factors which influence the reservoir dynamics and the ESN performance.
Promising applications of ESNs in the emerging field of Ambient Assisted Living are also
presented and discussed. Moving towards highly structured data representations, the
ESN model is extended to deal with complex structures directly, resulting in the proposed
TreeESN, which is suitable for domains comprising hierarchical structures, and Graph-ESN,
which generalizes the approach to a large class of cyclic/acyclic directed/undirected
labeled graphs. TreeESNs and GraphESNs represent both novel RC models for structured
data and extremely efficient approaches for modeling RecNNs, eventually contributing
to the definition of an RC framework for learning in structured domains. The problem
of adaptively exploiting the state space in GraphESNs is also investigated, with specific
regard to tasks in which input graphs are required to be mapped into flat vectorial outputs,
resulting in the GraphESN-wnn and GraphESN-NG models. As a further point, the
generalization performance of the proposed models is evaluated considering both artificial
and complex real-world tasks from different application domains, including Chemistry,
Toxicology and Document Processing
Graph self-organizing maps for cyclic and unbounded graphs
Self-organizing maps capable of processing graph structured information are a relatively new concept. This paper describes a novel concept on the processing of graph structured information using the self-organizing map framework which allows the processing of much more general types of graphs, e.g. cyclic graphs, directed graphs. Previous approaches to this problem were limited to the processing of bounded graphs, their computational complexity can grow rapidly with the level of connectivity of the graphs concerned, and are restricted to the processing of positional graphs. The novel concept proposed in this paper, namely, by using the clusters formed in the state space of the self-organizing map to represent the "strengths" of the activation of the neighboring vertices, rather than as in previous approaches, using the state space of the surrounding vertices to represent such "strengths" of activations. Such an approach resulted in reduced computational demand, and in allowing the processing of non-positional graphs
Graph self-organizing maps for cyclic and unbounded graphs
Self-Organizing Maps capable of processing graph structured information are a relatively new concept. This paper describes a novel concept on the processing of graph structured information using the self organizing map framework which allows the processing of much more general types of graphs, e.g. cyclic graphs, directed graphs. Previous approaches to this problem were limited to the processing of bounded graphs, their computational complexity can grow rapidly with the level of connectivity of the graphs concerned, and are restricted to the processing of positional graphs. The novel concept proposed in this paper, namely, by using the clusters formed in the state space of the self organizing map to represent the ``strengths\u27\u27 of the activation of the neighboring vertices, rather than as in previous approaches, using the state space of the surrounding vertices to represent such ``strengths\u27\u27 of activations. Such an approach resulted in reduced computational demand, and in allowing the processing of non-positional graphs