157,797 research outputs found
Automatic differentiation in machine learning: a survey
Derivatives, mostly in the form of gradients and Hessians, are ubiquitous in
machine learning. Automatic differentiation (AD), also called algorithmic
differentiation or simply "autodiff", is a family of techniques similar to but
more general than backpropagation for efficiently and accurately evaluating
derivatives of numeric functions expressed as computer programs. AD is a small
but established field with applications in areas including computational fluid
dynamics, atmospheric sciences, and engineering design optimization. Until very
recently, the fields of machine learning and AD have largely been unaware of
each other and, in some cases, have independently discovered each other's
results. Despite its relevance, general-purpose AD has been missing from the
machine learning toolbox, a situation slowly changing with its ongoing adoption
under the names "dynamic computational graphs" and "differentiable
programming". We survey the intersection of AD and machine learning, cover
applications where AD has direct relevance, and address the main implementation
techniques. By precisely defining the main differentiation techniques and their
interrelationships, we aim to bring clarity to the usage of the terms
"autodiff", "automatic differentiation", and "symbolic differentiation" as
these are encountered more and more in machine learning settings.Comment: 43 pages, 5 figure
Defending Against Firmware Cyber Attacks on Safety-Critical Systems
In the past, it was not possible to update the underlying software in many industrial control devices. Engineering
teams had to ‘rip and replace’ obsolete components. However, the ability to make firmware updates has provided
significant benefits to the companies who use Programmable Logic Controllers (PLCs), switches, gateways and
bridges as well as an array of smart sensor/actuators. These updates include security patches when vulnerabilities are
identified in existing devices; they can be distributed by physical media but are increasingly downloaded over
Internet connections. These mechanisms pose a growing threat to the cyber security of safety-critical applications,
which are illustrated by recent attacks on safety-related infrastructures across the Ukraine. Subsequent sections
explain how malware can be distributed within firmware updates. Even when attackers cannot reverse engineer the
code necessary to disguise their attack, they can undermine a device by forcing it into a constant upload cycle where
the firmware installation never terminates. In this paper, we present means of mitigating the risks of firmware attack
on safety-critical systems as part of wider initiatives to secure national critical infrastructures. Technical solutions,
including firmware hashing, must be augmented by organizational measures to secure the supply chain within
individual plants, across companies and throughout safety-related industries
Automatic Differentiation of Rigid Body Dynamics for Optimal Control and Estimation
Many algorithms for control, optimization and estimation in robotics depend
on derivatives of the underlying system dynamics, e.g. to compute
linearizations, sensitivities or gradient directions. However, we show that
when dealing with Rigid Body Dynamics, these derivatives are difficult to
derive analytically and to implement efficiently. To overcome this issue, we
extend the modelling tool `RobCoGen' to be compatible with Automatic
Differentiation. Additionally, we propose how to automatically obtain the
derivatives and generate highly efficient source code. We highlight the
flexibility and performance of the approach in two application examples. First,
we show a Trajectory Optimization example for the quadrupedal robot HyQ, which
employs auto-differentiation on the dynamics including a contact model. Second,
we present a hardware experiment in which a 6 DoF robotic arm avoids a randomly
moving obstacle in a go-to task by fast, dynamic replanning
AI and OR in management of operations: history and trends
The last decade has seen a considerable growth in the use of Artificial Intelligence (AI) for operations management with the aim of finding solutions to problems that are increasing in complexity and scale. This paper begins by setting the context for the survey through a historical perspective of OR and AI. An extensive survey of applications of AI techniques for operations management, covering a total of over 1200 papers published from 1995 to 2004 is then presented. The survey utilizes Elsevier's ScienceDirect database as a source. Hence, the survey may not cover all the relevant journals but includes a sufficiently wide range of publications to make it representative of the research in the field. The papers are categorized into four areas of operations management: (a) design, (b) scheduling, (c) process planning and control and (d) quality, maintenance and fault diagnosis. Each of the four areas is categorized in terms of the AI techniques used: genetic algorithms, case-based reasoning, knowledge-based systems, fuzzy logic and hybrid techniques. The trends over the last decade are identified, discussed with respect to expected trends and directions for future work suggested
Implementation of the dissection theorem in cadence virtuoso
This paper describes a tool for the Cadence Virtuoso software that implements the Dissection Theorem (DT) or General Network Theorem (GNT) and its applications: the Extra Element Theorem (EET), Chain Theorem (CT) and General Feedback Theorem (GFT). The tool allows a circuit designer to gain additional circuit insight by providing all second- and third-level transfer functions of the DT. In particular, feedback networks are factored into their exact components, enabling a deeper insight into the structure of the loop gain, direct forward transmission and hence closed-loop behaviour
Recovering Grammar Relationships for the Java Language Specification
Grammar convergence is a method that helps discovering relationships between
different grammars of the same language or different language versions. The key
element of the method is the operational, transformation-based representation
of those relationships. Given input grammars for convergence, they are
transformed until they are structurally equal. The transformations are composed
from primitive operators; properties of these operators and the composed chains
provide quantitative and qualitative insight into the relationships between the
grammars at hand. We describe a refined method for grammar convergence, and we
use it in a major study, where we recover the relationships between all the
grammars that occur in the different versions of the Java Language
Specification (JLS). The relationships are represented as grammar
transformation chains that capture all accidental or intended differences
between the JLS grammars. This method is mechanized and driven by nominal and
structural differences between pairs of grammars that are subject to
asymmetric, binary convergence steps. We present the underlying operator suite
for grammar transformation in detail, and we illustrate the suite with many
examples of transformations on the JLS grammars. We also describe the
extraction effort, which was needed to make the JLS grammars amenable to
automated processing. We include substantial metadata about the convergence
process for the JLS so that the effort becomes reproducible and transparent
AD in Fortran, Part 1: Design
We propose extensions to Fortran which integrate forward and reverse
Automatic Differentiation (AD) directly into the programming model.
Irrespective of implementation technology, embedding AD constructs directly
into the language extends the reach and convenience of AD while allowing
abstraction of concepts of interest to scientific-computing practice, such as
root finding, optimization, and finding equilibria of continuous games.
Multiple different subprograms for these tasks can share common interfaces,
regardless of whether and how they use AD internally. A programmer can maximize
a function F by calling a library maximizer, XSTAR=ARGMAX(F,X0), which
internally constructs derivatives of F by AD, without having to learn how to
use any particular AD tool. We illustrate the utility of these extensions by
example: programs become much more concise and closer to traditional
mathematical notation. A companion paper describes how these extensions can be
implemented by a program that generates input to existing Fortran-based AD
tools
- …