63,590 research outputs found
Enabling Massive Deep Neural Networks with the GraphBLAS
Deep Neural Networks (DNNs) have emerged as a core tool for machine learning.
The computations performed during DNN training and inference are dominated by
operations on the weight matrices describing the DNN. As DNNs incorporate more
stages and more nodes per stage, these weight matrices may be required to be
sparse because of memory limitations. The GraphBLAS.org math library standard
was developed to provide high performance manipulation of sparse weight
matrices and input/output vectors. For sufficiently sparse matrices, a sparse
matrix library requires significantly less memory than the corresponding dense
matrix implementation. This paper provides a brief description of the
mathematics underlying the GraphBLAS. In addition, the equations of a typical
DNN are rewritten in a form designed to use the GraphBLAS. An implementation of
the DNN is given using a preliminary GraphBLAS C library. The performance of
the GraphBLAS implementation is measured relative to a standard dense linear
algebra library implementation. For various sizes of DNN weight matrices, it is
shown that the GraphBLAS sparse implementation outperforms a BLAS dense
implementation as the weight matrix becomes sparser.Comment: 10 pages, 7 figures, to appear in the 2017 IEEE High Performance
Extreme Computing (HPEC) conferenc
Similarity-Aware Spectral Sparsification by Edge Filtering
In recent years, spectral graph sparsification techniques that can compute
ultra-sparse graph proxies have been extensively studied for accelerating
various numerical and graph-related applications. Prior nearly-linear-time
spectral sparsification methods first extract low-stretch spanning tree from
the original graph to form the backbone of the sparsifier, and then recover
small portions of spectrally-critical off-tree edges to the spanning tree to
significantly improve the approximation quality. However, it is not clear how
many off-tree edges should be recovered for achieving a desired spectral
similarity level within the sparsifier. Motivated by recent graph signal
processing techniques, this paper proposes a similarity-aware spectral graph
sparsification framework that leverages efficient spectral off-tree edge
embedding and filtering schemes to construct spectral sparsifiers with
guaranteed spectral similarity (relative condition number) level. An iterative
graph densification scheme is introduced to facilitate efficient and effective
filtering of off-tree edges for highly ill-conditioned problems. The proposed
method has been validated using various kinds of graphs obtained from public
domain sparse matrix collections relevant to VLSI CAD, finite element analysis,
as well as social and data networks frequently studied in many machine learning
and data mining applications
Inferring short-term volatility indicators from Bitcoin blockchain
In this paper, we study the possibility of inferring early warning indicators
(EWIs) for periods of extreme bitcoin price volatility using features obtained
from Bitcoin daily transaction graphs. We infer the low-dimensional
representations of transaction graphs in the time period from 2012 to 2017
using Bitcoin blockchain, and demonstrate how these representations can be used
to predict extreme price volatility events. Our EWI, which is obtained with a
non-negative decomposition, contains more predictive information than those
obtained with singular value decomposition or scalar value of the total Bitcoin
transaction volume
On Sampling from the Gibbs Distribution with Random Maximum A-Posteriori Perturbations
In this paper we describe how MAP inference can be used to sample efficiently
from Gibbs distributions. Specifically, we provide means for drawing either
approximate or unbiased samples from Gibbs' distributions by introducing low
dimensional perturbations and solving the corresponding MAP assignments. Our
approach also leads to new ways to derive lower bounds on partition functions.
We demonstrate empirically that our method excels in the typical "high signal -
high coupling" regime. The setting results in ragged energy landscapes that are
challenging for alternative approaches to sampling and/or lower bounds
Adaptation and learning over networks for nonlinear system modeling
In this chapter, we analyze nonlinear filtering problems in distributed
environments, e.g., sensor networks or peer-to-peer protocols. In these
scenarios, the agents in the environment receive measurements in a streaming
fashion, and they are required to estimate a common (nonlinear) model by
alternating local computations and communications with their neighbors. We
focus on the important distinction between single-task problems, where the
underlying model is common to all agents, and multitask problems, where each
agent might converge to a different model due to, e.g., spatial dependencies or
other factors. Currently, most of the literature on distributed learning in the
nonlinear case has focused on the single-task case, which may be a strong
limitation in real-world scenarios. After introducing the problem and reviewing
the existing approaches, we describe a simple kernel-based algorithm tailored
for the multitask case. We evaluate the proposal on a simulated benchmark task,
and we conclude by detailing currently open problems and lines of research.Comment: To be published as a chapter in `Adaptive Learning Methods for
Nonlinear System Modeling', Elsevier Publishing, Eds. D. Comminiello and J.C.
Principe (2018
- …