6,039 research outputs found
Automatic differentiation in machine learning: a survey
Derivatives, mostly in the form of gradients and Hessians, are ubiquitous in
machine learning. Automatic differentiation (AD), also called algorithmic
differentiation or simply "autodiff", is a family of techniques similar to but
more general than backpropagation for efficiently and accurately evaluating
derivatives of numeric functions expressed as computer programs. AD is a small
but established field with applications in areas including computational fluid
dynamics, atmospheric sciences, and engineering design optimization. Until very
recently, the fields of machine learning and AD have largely been unaware of
each other and, in some cases, have independently discovered each other's
results. Despite its relevance, general-purpose AD has been missing from the
machine learning toolbox, a situation slowly changing with its ongoing adoption
under the names "dynamic computational graphs" and "differentiable
programming". We survey the intersection of AD and machine learning, cover
applications where AD has direct relevance, and address the main implementation
techniques. By precisely defining the main differentiation techniques and their
interrelationships, we aim to bring clarity to the usage of the terms
"autodiff", "automatic differentiation", and "symbolic differentiation" as
these are encountered more and more in machine learning settings.Comment: 43 pages, 5 figure
Taming Numbers and Durations in the Model Checking Integrated Planning System
The Model Checking Integrated Planning System (MIPS) is a temporal least
commitment heuristic search planner based on a flexible object-oriented
workbench architecture. Its design clearly separates explicit and symbolic
directed exploration algorithms from the set of on-line and off-line computed
estimates and associated data structures. MIPS has shown distinguished
performance in the last two international planning competitions. In the last
event the description language was extended from pure propositional planning to
include numerical state variables, action durations, and plan quality objective
functions. Plans were no longer sequences of actions but time-stamped
schedules. As a participant of the fully automated track of the competition,
MIPS has proven to be a general system; in each track and every benchmark
domain it efficiently computed plans of remarkable quality. This article
introduces and analyzes the most important algorithmic novelties that were
necessary to tackle the new layers of expressiveness in the benchmark problems
and to achieve a high level of performance. The extensions include critical
path analysis of sequentially generated plans to generate corresponding optimal
parallel plans. The linear time algorithm to compute the parallel plan bypasses
known NP hardness results for partial ordering by scheduling plans with respect
to the set of actions and the imposed precedence relations. The efficiency of
this algorithm also allows us to improve the exploration guidance: for each
encountered planning state the corresponding approximate sequential plan is
scheduled. One major strength of MIPS is its static analysis phase that grounds
and simplifies parameterized predicates, functions and operators, that infers
knowledge to minimize the state description length, and that detects domain
object symmetries. The latter aspect is analyzed in detail. MIPS has been
developed to serve as a complete and optimal state space planner, with
admissible estimates, exploration engines and branching cuts. In the
competition version, however, certain performance compromises had to be made,
including floating point arithmetic, weighted heuristic search exploration
according to an inadmissible estimate and parameterized optimization
O'Mega: An Optimizing Matrix Element Generator
We sketch the architecture of O'Mega, a new optimizing compiler for tree
amplitudes in quantum field theory, and briefly describe its usage. O'Mega
generates the most efficient code currently available for scattering amplitudes
for many polarized particles in the Standard Model and its extensions.Comment: 29 pages, LaTe
Badger: Complexity Analysis with Fuzzing and Symbolic Execution
Hybrid testing approaches that involve fuzz testing and symbolic execution
have shown promising results in achieving high code coverage, uncovering subtle
errors and vulnerabilities in a variety of software applications. In this paper
we describe Badger - a new hybrid approach for complexity analysis, with the
goal of discovering vulnerabilities which occur when the worst-case time or
space complexity of an application is significantly higher than the average
case. Badger uses fuzz testing to generate a diverse set of inputs that aim to
increase not only coverage but also a resource-related cost associated with
each path. Since fuzzing may fail to execute deep program paths due to its
limited knowledge about the conditions that influence these paths, we
complement the analysis with a symbolic execution, which is also customized to
search for paths that increase the resource-related cost. Symbolic execution is
particularly good at generating inputs that satisfy various program conditions
but by itself suffers from path explosion. Therefore, Badger uses fuzzing and
symbolic execution in tandem, to leverage their benefits and overcome their
weaknesses. We implemented our approach for the analysis of Java programs,
based on Kelinci and Symbolic PathFinder. We evaluated Badger on Java
applications, showing that our approach is significantly faster in generating
worst-case executions compared to fuzzing or symbolic execution on their own
Julia: A Fresh Approach to Numerical Computing
Bridging cultures that have often been distant, Julia combines expertise from
the diverse fields of computer science and computational science to create a
new approach to numerical computing. Julia is designed to be easy and fast.
Julia questions notions generally held as "laws of nature" by practitioners of
numerical computing:
1. High-level dynamic programs have to be slow.
2. One must prototype in one language and then rewrite in another language
for speed or deployment, and
3. There are parts of a system for the programmer, and other parts best left
untouched as they are built by the experts.
We introduce the Julia programming language and its design --- a dance
between specialization and abstraction. Specialization allows for custom
treatment. Multiple dispatch, a technique from computer science, picks the
right algorithm for the right circumstance. Abstraction, what good computation
is really about, recognizes what remains the same after differences are
stripped away. Abstractions in mathematics are captured as code through another
technique from computer science, generic programming.
Julia shows that one can have machine performance without sacrificing human
convenience.Comment: 37 page
ALHEP symbolic algebra program for high-energy physics
ALHEP is the symbolic algebra program for high-energy physics. It deals with
amplitudes calculation, matrix element squaring, Wick theorem, dimensional
regularization, tensor reduction of loop integrals and simplification of final
expressions. The program output includes: Fortran code for differential cross
section, Mathematica files to view results and intermediate steps and TeX
source for Feynman diagrams. The PYTHIA interface is available. The project
website http://www.hep.by/alhep contains up-to-date executables, manual and
script examples.Comment: 33 pages, 4 figure
- …