36,653 research outputs found
Optimization under Uncertainty in the Era of Big Data and Deep Learning: When Machine Learning Meets Mathematical Programming
This paper reviews recent advances in the field of optimization under
uncertainty via a modern data lens, highlights key research challenges and
promise of data-driven optimization that organically integrates machine
learning and mathematical programming for decision-making under uncertainty,
and identifies potential research opportunities. A brief review of classical
mathematical programming techniques for hedging against uncertainty is first
presented, along with their wide spectrum of applications in Process Systems
Engineering. A comprehensive review and classification of the relevant
publications on data-driven distributionally robust optimization, data-driven
chance constrained program, data-driven robust optimization, and data-driven
scenario-based optimization is then presented. This paper also identifies
fertile avenues for future research that focuses on a closed-loop data-driven
optimization framework, which allows the feedback from mathematical programming
to machine learning, as well as scenario-based optimization leveraging the
power of deep learning techniques. Perspectives on online learning-based
data-driven multistage optimization with a learning-while-optimizing scheme is
presented
A nonmonotone spectral projected gradient method for large-scale topology optimization problems
An efficient gradient-based method to solve the volume constrained topology
optimization problems is presented. Each iterate of this algorithm is obtained
by the projection of a Barzilai-Borwein step onto the feasible set consisting
of box and one linear constraints (volume constraint). To ensure the global
convergence, an adaptive nonmonotone line search is performed along the
direction that is given by the current and projection point. The adaptive
cyclic reuse of the Barzilai-Borwein step is applied as the initial stepsize.
The minimum memory requirement, the guaranteed convergence property, and almost
only one function and gradient evaluations per iteration make this new method
very attractive within common alternative methods to solve large-scale optimal
design problems. Efficiency and feasibility of the presented method are
supported by numerical experiments
Asynchronous parallel primal-dual block coordinate update methods for affinely constrained convex programs
Recent several years have witnessed the surge of asynchronous (async-)
parallel computing methods due to the extremely big data involved in many
modern applications and also the advancement of multi-core machines and
computer clusters. In optimization, most works about async-parallel methods are
on unconstrained problems or those with block separable constraints.
In this paper, we propose an async-parallel method based on block coordinate
update (BCU) for solving convex problems with nonseparable linear constraint.
Running on a single node, the method becomes a novel randomized primal-dual BCU
with adaptive stepsize for multi-block affinely constrained problems. For these
problems, Gauss-Seidel cyclic primal-dual BCU needs strong convexity to have
convergence. On the contrary, merely assuming convexity, we show that the
objective value sequence generated by the proposed algorithm converges in
probability to the optimal value and also the constraint residual to zero. In
addition, we establish an ergodic convergence result, where is the
number of iterations. Numerical experiments are performed to demonstrate the
efficiency of the proposed method and significantly better speed-up performance
than its sync-parallel counterpart
A Low-Rank Coordinate-Descent Algorithm for Semidefinite Programming Relaxations of Optimal Power Flow
The alternating-current optimal power flow (ACOPF) is one of the best known
non-convex non-linear optimisation problems. We present a novel re-formulation
of ACOPF, which is based on lifting the rectangular power-voltage
rank-constrained formulation, and makes it possible to derive alternative SDP
relaxations. For those, we develop a first-order method based on the parallel
coordinate descent with a novel closed-form step based on roots of cubic
polynomials
Adaptive FISTA for Non-convex Optimization
In this paper we propose an adaptively extrapolated proximal gradient method,
which is based on the accelerated proximal gradient method (also known as
FISTA), however we locally optimize the extrapolation parameter by carrying out
an exact (or inexact) line search. It turns out that in some situations, the
proposed algorithm is equivalent to a class of SR1 (identity minus rank 1)
proximal quasi-Newton methods. Convergence is proved in a general non-convex
setting, and hence, as a byproduct, we also obtain new convergence guarantees
for proximal quasi-Newton methods. The efficiency of the new method is shown in
numerical experiments on a sparsity regularized non-linear inverse problem
Constrained Deep Learning using Conditional Gradient and Applications in Computer Vision
A number of results have recently demonstrated the benefits of incorporating
various constraints when training deep architectures in vision and machine
learning. The advantages range from guarantees for statistical generalization
to better accuracy to compression. But support for general constraints within
widely used libraries remains scarce and their broader deployment within many
applications that can benefit from them remains under-explored. Part of the
reason is that Stochastic gradient descent (SGD), the workhorse for training
deep neural networks, does not natively deal with constraints with global scope
very well. In this paper, we revisit a classical first order scheme from
numerical optimization, Conditional Gradients (CG), that has, thus far had
limited applicability in training deep models. We show via rigorous analysis
how various constraints can be naturally handled by modifications of this
algorithm. We provide convergence guarantees and show a suite of immediate
benefits that are possible -- from training ResNets with fewer layers but
better accuracy simply by substituting in our version of CG to faster training
of GANs with 50% fewer epochs in image inpainting applications to provably
better generalization guarantees using efficiently implementable forms of
recently proposed regularizers
Robust optimization of a broad class of heterogeneous vehicle routing problems under demand uncertainty
This paper studies robust variants of an extended model of the classical
Heterogeneous Vehicle Routing Problem (HVRP), where a mixed fleet of vehicles
with different capacities, availabilities, fixed costs and routing costs is
used to serve customers with uncertain demand. This model includes, as special
cases, all variants of the HVRP studied in the literature with fixed and
unlimited fleet sizes, accessibility restrictions at customer locations, as
well as multiple depots. Contrary to its deterministic counterpart, the goal of
the robust HVRP is to determine a minimum-cost set of routes and fleet
composition that remains feasible for all demand realizations from a
pre-specified uncertainty set. To solve this problem, we develop robust
versions of classical node- and edge-exchange neighborhoods that are commonly
used in local search and establish that efficient evaluation of the local moves
can be achieved for five popular classes of uncertainty sets. The proposed
local search is then incorporated in a modular fashion within two metaheuristic
algorithms to determine robust HVRP solutions. The quality of the metaheuristic
solutions is quantified using an integer programming model that provides lower
bounds on the optimal solution. An extensive computational study on literature
benchmarks shows that the proposed methods allow us to obtain high quality
robust solutions for different uncertainty sets and with minor additional
effort compared to deterministic solutions.Comment: 54 pages, 10 figures, 12 table
A Learning-based Power Management for Networked Microgrids Under Incomplete Information
This paper presents an approximate Reinforcement Learning (RL) methodology
for bi-level power management of networked Microgrids (MG) in electric
distribution systems. In practice, the cooperative agent can have limited or no
knowledge of the MG asset behavior and detailed models behind the Point of
Common Coupling (PCC). This makes the distribution systems unobservable and
impedes conventional optimization solutions for the constrained MG power
management problem. To tackle this challenge, we have proposed a bi-level RL
framework in a price-based environment. At the higher level, a cooperative
agent performs function approximation to predict the behavior of entities under
incomplete information of MG parametric models; while at the lower level, each
MG provides power-flow-constrained optimal response to price signals. The
function approximation scheme is then used within an adaptive RL framework to
optimize the price signal as the system load and solar generation change over
time. Numerical experiments have verified that, compared to previous works in
the literature, the proposed privacy-preserving learning model has better
adaptability and enhanced computational speed
Boosting Cloud Data Analytics using Multi-Objective Optimization
Data analytics in the cloud has become an integral part of enterprise
businesses. Big data analytics systems, however, still lack the ability to take
user performance goals and budgetary constraints for a task, collectively
referred to as task objectives, and automatically configure an analytic job to
achieve these objectives. This paper presents a data analytics optimizer that
can automatically determine a cluster configuration with a suitable number of
cores as well as other system parameters that best meet the task objectives. At
a core of our work is a principled multi-objective optimization (MOO) approach
that computes a Pareto optimal set of job configurations to reveal tradeoffs
between different user objectives, recommends a new job configuration that best
explores such tradeoffs, and employs novel optimizations to enable such
recommendations within a few seconds. We present efficient incremental
algorithms based on the notion of a Progressive Frontier for realizing our MOO
approach and implement them into a Spark-based prototype. Detailed experiments
using benchmark workloads show that our MOO techniques provide a 2-50x speedup
over existing MOO methods, while offering good coverage of the Pareto frontier.
When compared to Ottertune, a state-of-the-art performance tuning system, our
approach recommends configurations that yield 26\%-49\% reduction of running
time of the TPCx-BB benchmark while adapting to different application
preferences on multiple objectives
Particle Swarm Optimization: A survey of historical and recent developments with hybridization perspectives
Particle Swarm Optimization (PSO) is a metaheuristic global optimization
paradigm that has gained prominence in the last two decades due to its ease of
application in unsupervised, complex multidimensional problems which cannot be
solved using traditional deterministic algorithms. The canonical particle swarm
optimizer is based on the flocking behavior and social co-operation of birds
and fish schools and draws heavily from the evolutionary behavior of these
organisms. This paper serves to provide a thorough survey of the PSO algorithm
with special emphasis on the development, deployment and improvements of its
most basic as well as some of the state-of-the-art implementations. Concepts
and directions on choosing the inertia weight, constriction factor, cognition
and social weights and perspectives on convergence, parallelization, elitism,
niching and discrete optimization as well as neighborhood topologies are
outlined. Hybridization attempts with other evolutionary and swarm paradigms in
selected applications are covered and an up-to-date review is put forward for
the interested reader.Comment: 34 pages, 7 table
- …