2,102 research outputs found
Learning to Discover Sparse Graphical Models
We consider structure discovery of undirected graphical models from
observational data. Inferring likely structures from few examples is a complex
task often requiring the formulation of priors and sophisticated inference
procedures. Popular methods rely on estimating a penalized maximum likelihood
of the precision matrix. However, in these approaches structure recovery is an
indirect consequence of the data-fit term, the penalty can be difficult to
adapt for domain-specific knowledge, and the inference is computationally
demanding. By contrast, it may be easier to generate training samples of data
that arise from graphs with the desired structure properties. We propose here
to leverage this latter source of information as training data to learn a
function, parametrized by a neural network that maps empirical covariance
matrices to estimated graph structures. Learning this function brings two
benefits: it implicitly models the desired structure or sparsity properties to
form suitable priors, and it can be tailored to the specific problem of edge
structure discovery, rather than maximizing data likelihood. Applying this
framework, we find our learnable graph-discovery method trained on synthetic
data generalizes well: identifying relevant edges in both synthetic and real
data, completely unknown at training time. We find that on genetics, brain
imaging, and simulation data we obtain performance generally superior to
analytical methods
Geometric deep learning: going beyond Euclidean data
Many scientific fields study data with an underlying structure that is a
non-Euclidean space. Some examples include social networks in computational
social sciences, sensor networks in communications, functional networks in
brain imaging, regulatory networks in genetics, and meshed surfaces in computer
graphics. In many applications, such geometric data are large and complex (in
the case of social networks, on the scale of billions), and are natural targets
for machine learning techniques. In particular, we would like to use deep
neural networks, which have recently proven to be powerful tools for a broad
range of problems from computer vision, natural language processing, and audio
analysis. However, these tools have been most successful on data with an
underlying Euclidean or grid-like structure, and in cases where the invariances
of these structures are built into networks used to model them. Geometric deep
learning is an umbrella term for emerging techniques attempting to generalize
(structured) deep neural models to non-Euclidean domains such as graphs and
manifolds. The purpose of this paper is to overview different examples of
geometric deep learning problems and present available solutions, key
difficulties, applications, and future research directions in this nascent
field
Biologically inspired feature extraction for rotation and scale tolerant pattern analysis
Biologically motivated information processing has been an important area of scientific research for decades. The central topic addressed in this dissertation is utilization of lateral inhibition and more generally, linear networks with recurrent connectivity along with complex-log conformal mapping in machine based implementations of information encoding, feature extraction and pattern recognition. The reasoning behind and method for spatially uniform implementation of inhibitory/excitatory network model in the framework of non-uniform log-polar transform is presented. For the space invariant connectivity model characterized by Topelitz-Block-Toeplitz matrix, the overall network response is obtained without matrix inverse operations providing the connection matrix generating function is bound by unity. It was shown that for the network with the inter-neuron connection function expandable in a Fourier series in polar angle, the overall network response is steerable. The decorrelating/whitening characteristics of networks with lateral inhibition are used in order to develop space invariant pre-whitening kernels specialized for specific category of input signals. These filters have extremely small memory footprint and are successfully utilized in order to improve performance of adaptive neural whitening algorithms. Finally, the method for feature extraction based on localized Independent Component Analysis (ICA) transform in log-polar domain and aided by previously developed pre-whitening filters is implemented. Since output codes produced by ICA are very sparse, a small number of non-zero coefficients was sufficient to encode input data and obtain reliable pattern recognition performance
Network Filtering of Spatial-temporal GNN for Multivariate Time-series Prediction
We propose an architecture for multivariate time-series prediction that integrates a spatial-temporal graph neural network with a filtering module which filters the inverse correlation matrix into a sparse network structure. In contrast with existing sparsification methods adopted in graph neural networks, our model explicitly leverages time-series filtering to overcome the low signal-to-noise ratio typical of complex systems data. We present a set of experiments, where we predict future sales volume from a synthetic time-series sales volume dataset. The proposed spatial-temporal graph neural network displays superior performances to baseline approaches with no graphical information, fully connected, disconnected graphs, and unfiltered graphs, as well as the state-of-the-art spatial-temporal GNN. Comparison of the results with Diffusion Convolutional Recurrent Neural Network (DCRNN) suggests that, by combining a (inferior) GNN with graph sparsification and filtering, one can achieve comparable or better efficacy than the state-of-the-art in multivariate time-series regression
Reconstruction of three-dimensional porous media using generative adversarial neural networks
To evaluate the variability of multi-phase flow properties of porous media at
the pore scale, it is necessary to acquire a number of representative samples
of the void-solid structure. While modern x-ray computer tomography has made it
possible to extract three-dimensional images of the pore space, assessment of
the variability in the inherent material properties is often experimentally not
feasible. We present a novel method to reconstruct the solid-void structure of
porous media by applying a generative neural network that allows an implicit
description of the probability distribution represented by three-dimensional
image datasets. We show, by using an adversarial learning approach for neural
networks, that this method of unsupervised learning is able to generate
representative samples of porous media that honor their statistics. We
successfully compare measures of pore morphology, such as the Euler
characteristic, two-point statistics and directional single-phase permeability
of synthetic realizations with the calculated properties of a bead pack, Berea
sandstone, and Ketton limestone. Results show that GANs can be used to
reconstruct high-resolution three-dimensional images of porous media at
different scales that are representative of the morphology of the images used
to train the neural network. The fully convolutional nature of the trained
neural network allows the generation of large samples while maintaining
computational efficiency. Compared to classical stochastic methods of image
reconstruction, the implicit representation of the learned data distribution
can be stored and reused to generate multiple realizations of the pore
structure very rapidly.Comment: 21 pages, 20 figure
Graph signal processing for machine learning: A review and new perspectives
The effective representation, processing, analysis, and visualization of
large-scale structured data, especially those related to complex domains such
as networks and graphs, are one of the key questions in modern machine
learning. Graph signal processing (GSP), a vibrant branch of signal processing
models and algorithms that aims at handling data supported on graphs, opens new
paths of research to address this challenge. In this article, we review a few
important contributions made by GSP concepts and tools, such as graph filters
and transforms, to the development of novel machine learning algorithms. In
particular, our discussion focuses on the following three aspects: exploiting
data structure and relational priors, improving data and computational
efficiency, and enhancing model interpretability. Furthermore, we provide new
perspectives on future development of GSP techniques that may serve as a bridge
between applied mathematics and signal processing on one side, and machine
learning and network science on the other. Cross-fertilization across these
different disciplines may help unlock the numerous challenges of complex data
analysis in the modern age
Video Propagation Networks
We propose a technique that propagates information forward through video
data. The method is conceptually simple and can be applied to tasks that
require the propagation of structured information, such as semantic labels,
based on video content. We propose a 'Video Propagation Network' that processes
video frames in an adaptive manner. The model is applied online: it propagates
information forward without the need to access future frames. In particular we
combine two components, a temporal bilateral network for dense and video
adaptive filtering, followed by a spatial network to refine features and
increased flexibility. We present experiments on video object segmentation and
semantic video segmentation and show increased performance comparing to the
best previous task-specific methods, while having favorable runtime.
Additionally we demonstrate our approach on an example regression task of color
propagation in a grayscale video.Comment: Appearing in Computer Vision and Pattern Recognition, 2017 (CVPR'17
- …