5,893 research outputs found
Tradeoffs between Convergence Speed and Reconstruction Accuracy in Inverse Problems
Solving inverse problems with iterative algorithms is popular, especially for
large data. Due to time constraints, the number of possible iterations is
usually limited, potentially affecting the achievable accuracy. Given an error
one is willing to tolerate, an important question is whether it is possible to
modify the original iterations to obtain faster convergence to a minimizer
achieving the allowed error without increasing the computational cost of each
iteration considerably. Relying on recent recovery techniques developed for
settings in which the desired signal belongs to some low-dimensional set, we
show that using a coarse estimate of this set may lead to faster convergence at
the cost of an additional reconstruction error related to the accuracy of the
set approximation. Our theory ties to recent advances in sparse recovery,
compressed sensing, and deep learning. Particularly, it may provide a possible
explanation to the successful approximation of the l1-minimization solution by
neural networks with layers representing iterations, as practiced in the
learned iterative shrinkage-thresholding algorithm (LISTA).Comment: To appear in IEEE Transactions on Signal Processin
Efficiently testing local optimality and escaping saddles for ReLU networks
We provide a theoretical algorithm for checking local optimality and escaping
saddles at nondifferentiable points of empirical risks of two-layer ReLU
networks. Our algorithm receives any parameter value and returns: local
minimum, second-order stationary point, or a strict descent direction. The
presence of data points on the nondifferentiability of the ReLU divides the
parameter space into at most regions, which makes analysis difficult. By
exploiting polyhedral geometry, we reduce the total computation down to one
convex quadratic program (QP) for each hidden node, (in)equality tests,
and one (or a few) nonconvex QP. For the last QP, we show that our specific
problem can be solved efficiently, in spite of nonconvexity. In the benign
case, we solve one equality constrained QP, and we prove that projected
gradient descent solves it exponentially fast. In the bad case, we have to
solve a few more inequality constrained QPs, but we prove that the time
complexity is exponential only in the number of inequality constraints. Our
experiments show that either benign case or bad case with very few inequality
constraints occurs, implying that our algorithm is efficient in most cases.Comment: 23 pages, appeared at ICLR 201
Quadratically constrained quadratic programming for classification using particle swarms and applications
Particle swarm optimization is used in several combinatorial optimization
problems. In this work, particle swarms are used to solve quadratic programming
problems with quadratic constraints. The approach of particle swarms is an
example for interior point methods in optimization as an iterative technique.
This approach is novel and deals with classification problems without the use
of a traditional classifier. Our method determines the optimal hyperplane or
classification boundary for a data set. In a binary classification problem, we
constrain each class as a cluster, which is enclosed by an ellipsoid. The
estimation of the optimal hyperplane between the two clusters is posed as a
quadratically constrained quadratic problem. The optimization problem is solved
in distributed format using modified particle swarms. Our method has the
advantage of using the direction towards optimal solution rather than searching
the entire feasible region. Our results on the Iris, Pima, Wine, and Thyroid
datasets show that the proposed method works better than a neural network and
the performance is close to that of SVM.Comment: 17 pages, 3 figure
Robust Dynamic Locomotion via Reinforcement Learning and Novel Whole Body Controller
We propose a robust dynamic walking controller consisting of a dynamic
locomotion planner, a reinforcement learning process for robustness, and a
novel whole-body locomotion controller (WBLC). Previous approaches specify
either the position or the timing of steps, however, the proposed locomotion
planner simultaneously computes both of these parameters as locomotion outputs.
Our locomotion strategy relies on devising a reinforcement learning (RL)
approach for robust walking. The learned policy generates multi step walking
patterns, and the process is quick enough to be suitable for real-time
controls. For learning, we devise an RL strategy that uses a phase space
planner (PSP) and a linear inverted pendulum model to make the problem
tractable and very fast. Then, the learned policy is used to provide goal-based
commands to the WBLC, which calculates the torque commands to be executed in
full-humanoid robots. The WBLC combines multiple prioritized tasks and
calculates the associated reaction forces based on practical inequality
constraints. The novel formulation includes efficient calculation of the time
derivatives of various Jacobians. This provides high-fidelity dynamic control
of fast motions. More specifically, we compute the time derivative of the
Jacobian for various tasks and the Jacobian of the centroidal momentum task by
utilizing Lie group operators and operational space dynamics respectively. The
integration of RL-PSP and the WBLC provides highly robust, versatile, and
practical locomotion including steering while walking and handling push
disturbances of up to 520 N during an interval of 0.1 sec. Theoretical and
numerical results are tested through a 3D physics-based simulation of the
humanoid robot Valkyrie.Comment: 15 pages, 12 figure
Initialization-free Distributed Algorithms for Optimal Resource Allocation with Feasibility Constraints and its Application to Economic Dispatch of Power Systems
In this paper, the distributed resource allocation optimization problem is
investigated. The allocation decisions are made to minimize the sum of all the
agents' local objective functions while satisfying both the global network
resource constraint and the local allocation feasibility constraints. Here the
data corresponding to each agent in this separable optimization problem, such
as the network resources, the local allocation feasibility constraint, and the
local objective function, is only accessible to individual agent and cannot be
shared with others, which renders new challenges in this distributed
optimization problem. Based on either projection or differentiated projection,
two classes of continuous-time algorithms are proposed to solve this
distributed optimization problem in an initialization-free and scalable manner.
Thus, no re-initialization is required even if the operation environment or
network configuration is changed, making it possible to achieve a
"plug-and-play" optimal operation of networked heterogeneous agents. The
algorithm convergence is guaranteed for strictly convex objective functions,
and the exponential convergence is proved for strongly convex functions without
local constraints. Then the proposed algorithm is applied to the distributed
economic dispatch problem in power grids, to demonstrate how it can achieve the
global optimum in a scalable way, even when the generation cost, or system
load, or network configuration, is changing.Comment: 13 pages, 7 figure
Distributed coordination for nonsmooth convex optimization via saddle-point dynamics
This paper considers continuous-time coordination algorithms for networks of
agents that seek to collectively solve a general class of nonsmooth convex
optimization problems with an inherent distributed structure. Our algorithm
design builds on the characterization of the solutions of the nonsmooth convex
program as saddle points of an augmented Lagrangian. We show that the
associated saddle-point dynamics are asymptotically correct but, in general,
not distributed because of the presence of a global penalty parameter. This
motivates the design of a discontinuous saddle-point-like algorithm that enjoys
the same convergence properties and is fully amenable to distributed
implementation. Our convergence proofs rely on the identification of a novel
global Lyapunov function for saddle-point dynamics. This novelty also allows us
to identify mild convexity and regularity conditions on the objective function
that guarantee the exponential convergence rate of the proposed algorithms for
convex optimization problems subject to equality constraints. Various examples
illustrate our discussion.Comment: 20 page
Proximal Reinforcement Learning: A New Theory of Sequential Decision Making in Primal-Dual Spaces
In this paper, we set forth a new vision of reinforcement learning developed
by us over the past few years, one that yields mathematically rigorous
solutions to longstanding important questions that have remained unresolved:
(i) how to design reliable, convergent, and robust reinforcement learning
algorithms (ii) how to guarantee that reinforcement learning satisfies
pre-specified "safety" guarantees, and remains in a stable region of the
parameter space (iii) how to design "off-policy" temporal difference learning
algorithms in a reliable and stable manner, and finally (iv) how to integrate
the study of reinforcement learning into the rich theory of stochastic
optimization. In this paper, we provide detailed answers to all these questions
using the powerful framework of proximal operators.
The key idea that emerges is the use of primal dual spaces connected through
the use of a Legendre transform. This allows temporal difference updates to
occur in dual spaces, allowing a variety of important technical advantages. The
Legendre transform elegantly generalizes past algorithms for solving
reinforcement learning problems, such as natural gradient methods, which we
show relate closely to the previously unconnected framework of mirror descent
methods. Equally importantly, proximal operator theory enables the systematic
development of operator splitting methods that show how to safely and reliably
decompose complex products of gradients that occur in recent variants of
gradient-based temporal difference learning. This key technical innovation
makes it possible to finally design "true" stochastic gradient methods for
reinforcement learning. Finally, Legendre transforms enable a variety of other
benefits, including modeling sparsity and domain geometry. Our work builds
extensively on recent work on the convergence of saddle-point algorithms, and
on the theory of monotone operators.Comment: 121 page
Differentiating through Log-Log Convex Programs
We show how to efficiently compute the derivative (when it exists) of the
solution map of log-log convex programs (LLCPs). These are nonconvex, nonsmooth
optimization problems with positive variables that become convex when the
variables, objective functions, and constraint functions are replaced with
their logs. We focus specifically on LLCPs generated by disciplined geometric
programming, a grammar consisting of a set of atomic functions with known
log-log curvature and a composition rule for combining them. We represent a
parametrized LLCP as the composition of a smooth transformation of parameters,
a convex optimization problem, and an exponential transformation of the convex
optimization problem's solution. The derivative of this composition can be
computed efficiently, using recently developed methods for differentiating
through convex optimization problems. We implement our method in CVXPY, a
Python-embedded modeling language and rewriting system for convex optimization.
In just a few lines of code, a user can specify a parametrized LLCP, solve it,
and evaluate the derivative or its adjoint at a vector. This makes it possible
to conduct sensitivity analyses of solutions, given perturbations to the
parameters, and to compute the gradient of a function of the solution with
respect to the parameters. We use the adjoint of the derivative to implement
differentiable log-log convex optimization layers in PyTorch and TensorFlow.
Finally, we present applications to designing queuing systems and fitting
structured prediction models.Comment: Fix some typo
Level-set methods for convex optimization
Convex optimization problems arising in applications often have favorable
objective functions and complicated constraints, thereby precluding first-order
methods from being immediately applicable. We describe an approach that
exchanges the roles of the objective and constraint functions, and instead
approximately solves a sequence of parametric level-set problems. A
zero-finding procedure, based on inexact function evaluations and possibly
inexact derivative information, leads to an efficient solution scheme for the
original problem. We describe the theoretical and practical properties of this
approach for a broad range of problems, including low-rank semidefinite
optimization, sparse optimization, and generalized linear models for inference.Comment: 38 page
A Tropical Approach to Neural Networks with Piecewise Linear Activations
We present a new, unifying approach following some recent developments on the
complexity of neural networks with piecewise linear activations. We treat
neural network layers with piecewise linear activations as tropical
polynomials, which generalize polynomials in the so-called or
tropical algebra, with possibly real-valued exponents. Motivated by the
discussion in (arXiv:1402.1869), this approach enables us to refine their upper
bounds on linear regions of layers with ReLU or leaky ReLU activations to
, where are the
number of inputs and outputs, respectively. Additionally, we recover their
upper bounds on maxout layers. Our work follows a novel path, exclusively under
the lens of tropical geometry, which is independent of the improvements
reported in (arXiv:1611.01491, arXiv:1711.02114). Finally, we present a
geometric approach for effective counting of linear regions using random
sampling in order to avoid the computational overhead of exact counting
approachesComment: v2: Removed morphological perceptron section and added vertex
sampling section. Updated references. 18 pages, 7 figure
- …