Enabling Massive Deep Neural Networks with the GraphBLAS

Deep Neural Networks (DNNs) have emerged as a core tool for machine learning. The computations performed during DNN training and inference are dominated by operations on the weight matrices describing the DNN. As DNNs incorporate more stages and more nodes per stage, these weight matrices may be required to be sparse because of memory limitations. The GraphBLAS.org math library standard was developed to provide high performance manipulation of sparse weight matrices and input/output vectors. For sufficiently sparse matrices, a sparse matrix library requires significantly less memory than the corresponding dense matrix implementation. This paper provides a brief description of the mathematics underlying the GraphBLAS. In addition, the equations of a typical DNN are rewritten in a form designed to use the GraphBLAS. An implementation of the DNN is given using a preliminary GraphBLAS C library. The performance of the GraphBLAS implementation is measured relative to a standard dense linear algebra library implementation. For various sizes of DNN weight matrices, it is shown that the GraphBLAS sparse implementation outperforms a BLAS dense implementation as the weight matrix becomes sparser.Comment: 10 pages, 7 figures, to appear in the 2017 IEEE High Performance Extreme Computing (HPEC) conferenc

A normal form algorithm for the Brieskorn lattice

This article describes a normal form algorithm for the Brieskorn lattice of an isolated hypersurface singularity. It is the basis of efficient algorithms to compute the Bernstein-Sato polynomial, the complex monodromy, and Hodge-theoretic invariants of the singularity such as the spectral pairs and good bases of the Brieskorn lattice. The algorithm is a variant of Buchberger's normal form algorithm for power series rings using the idea of partial standard bases and adic convergence replacing termination.Comment: 23 pages, 1 figure, 4 table

Tropical bounds for eigenvalues of matrices

We show that for all k = 1,...,n the absolute value of the product of the k largest eigenvalues of an n-by-n matrix A is bounded from above by the product of the k largest tropical eigenvalues of the matrix |A| (entrywise absolute value), up to a combinatorial constant depending only on k and on the pattern of the matrix. This generalizes an inequality by Friedland (1986), corresponding to the special case k = 1.Comment: 17 pages, 1 figur

An Inverse Method for Policy-Iteration Based Algorithms

We present an extension of two policy-iteration based algorithms on weighted graphs (viz., Markov Decision Problems and Max-Plus Algebras). This extension allows us to solve the following inverse problem: considering the weights of the graph to be unknown constants or parameters, we suppose that a reference instantiation of those weights is given, and we aim at computing a constraint on the parameters under which an optimal policy for the reference instantiation is still optimal. The original algorithm is thus guaranteed to behave well around the reference instantiation, which provides us with some criteria of robustness. We present an application of both methods to simple examples. A prototype implementation has been done

Nearly Optimal Algorithms for the Decomposition of Multivariate Rational Functions and the Extended L\"uroth's Theorem

The extended L\"uroth's Theorem says that if the transcendence degree of \KK(\mathsf{f}_1,\dots,\mathsf{f}_m)/\KK is 1 then there exists f \in \KK(\underline{X}) such that \KK(\mathsf{f}_1,\dots,\mathsf{f}_m) is equal to \KK(f). In this paper we show how to compute $f$ with a probabilistic algorithm. We also describe a probabilistic and a deterministic algorithm for the decomposition of multivariate rational functions. The probabilistic algorithms proposed in this paper are softly optimal when $n$ is fixed and $d$ tends to infinity. We also give an indecomposability test based on gcd computations and Newton's polytope. In the last section, we show that we get a polynomial time algorithm, with a minor modification in the exponential time decomposition algorithm proposed by Gutierez-Rubio-Sevilla in 2001

A complete closed-form solution to a tropical extremal problem

A multidimensional extremal problem in the idempotent algebra setting is considered which consists in minimizing a nonlinear functional defined on a finite-dimensional semimodule over an idempotent semifield. The problem integrates two other known problems by combining their objective functions into one general function and includes these problems as particular cases. A new solution approach is proposed based on the analysis of linear inequalities and spectral properties of matrices. The approach offers a comprehensive solution to the problem in a closed form that involves performing simple matrix and vector operations in terms of idempotent algebra and provides a basis for the development of efficient computational algorithms and their software implementation.Comment: Proceedings of the 6th WSEAS European Computing Conference (ECC '12), Prague, Czech Republic, September 24-26, 201