39,268 research outputs found
Automatic differentiation in machine learning: a survey
Derivatives, mostly in the form of gradients and Hessians, are ubiquitous in
machine learning. Automatic differentiation (AD), also called algorithmic
differentiation or simply "autodiff", is a family of techniques similar to but
more general than backpropagation for efficiently and accurately evaluating
derivatives of numeric functions expressed as computer programs. AD is a small
but established field with applications in areas including computational fluid
dynamics, atmospheric sciences, and engineering design optimization. Until very
recently, the fields of machine learning and AD have largely been unaware of
each other and, in some cases, have independently discovered each other's
results. Despite its relevance, general-purpose AD has been missing from the
machine learning toolbox, a situation slowly changing with its ongoing adoption
under the names "dynamic computational graphs" and "differentiable
programming". We survey the intersection of AD and machine learning, cover
applications where AD has direct relevance, and address the main implementation
techniques. By precisely defining the main differentiation techniques and their
interrelationships, we aim to bring clarity to the usage of the terms
"autodiff", "automatic differentiation", and "symbolic differentiation" as
these are encountered more and more in machine learning settings.Comment: 43 pages, 5 figure
A Polyhedral Method to Compute All Affine Solution Sets of Sparse Polynomial Systems
To compute solutions of sparse polynomial systems efficiently we have to
exploit the structure of their Newton polytopes. While the application of
polyhedral methods naturally excludes solutions with zero components, an
irreducible decomposition of a variety is typically understood in affine space,
including also those components with zero coordinates. We present a polyhedral
method to compute all affine solution sets of a polynomial system. The method
enumerates all factors contributing to a generalized permanent. Toric solution
sets are recovered as a special case of this enumeration. For sparse systems as
adjacent 2-by-2 minors our methods scale much better than the techniques from
numerical algebraic geometry
Real root finding for equivariant semi-algebraic systems
Let be a real closed field. We consider basic semi-algebraic sets defined
by -variate equations/inequalities of symmetric polynomials and an
equivariant family of polynomials, all of them of degree bounded by .
Such a semi-algebraic set is invariant by the action of the symmetric group. We
show that such a set is either empty or it contains a point with at most
distinct coordinates. Combining this geometric result with efficient algorithms
for real root finding (based on the critical point method), one can decide the
emptiness of basic semi-algebraic sets defined by polynomials of degree
in time . This improves the state-of-the-art which is exponential
in . When the variables are quantified and the
coefficients of the input system depend on parameters , one
also demonstrates that the corresponding one-block quantifier elimination
problem can be solved in time
Sigref ā A Symbolic Bisimulation Tool Box
We present a uniform signature-based approach to compute the most popular bisimulations. Our approach is implemented symbolically using BDDs, which enables the handling of very large transition systems. Signatures for the bisimulations are built up from a few generic building blocks, which naturally correspond to efficient BDD operations. Thus, the definition of an appropriate signature is the key for a rapid development of algorithms for other types of bisimulation.
We provide experimental evidence of the viability of this approach by presenting computational results for many bisimulations on real-world instances. The experiments show cases where our framework can handle state spaces efficiently that are far too large to handle for any tool that requires an explicit state space description.
This work was partly supported by the German Research Council (DFG) as part of the Transregional Collaborative Research Center āAutomatic Verification and Analysis of Complex Systemsā (SFB/TR 14 AVACS). See www.avacs.org for more information
- ā¦