11,556 research outputs found
Worst-Case Linear Discriminant Analysis as Scalable Semidefinite Feasibility Problems
In this paper, we propose an efficient semidefinite programming (SDP)
approach to worst-case linear discriminant analysis (WLDA). Compared with the
traditional LDA, WLDA considers the dimensionality reduction problem from the
worst-case viewpoint, which is in general more robust for classification.
However, the original problem of WLDA is non-convex and difficult to optimize.
In this paper, we reformulate the optimization problem of WLDA into a sequence
of semidefinite feasibility problems. To efficiently solve the semidefinite
feasibility problems, we design a new scalable optimization method with
quasi-Newton methods and eigen-decomposition being the core components. The
proposed method is orders of magnitude faster than standard interior-point
based SDP solvers.
Experiments on a variety of classification problems demonstrate that our
approach achieves better performance than standard LDA. Our method is also much
faster and more scalable than standard interior-point SDP solvers based WLDA.
The computational complexity for an SDP with constraints and matrices of
size by is roughly reduced from to
( in our case).Comment: 14 page
Localization of Control Synthesis Problem for Large-Scale Interconnected System Using IQC and Dissipativity Theories
The synthesis problem for the compositional performance certification of
interconnected systems is considered. A fairly unified description of control
synthesis problem is given using integral quadratic constraints (IQC) and
dissipativity. Starting with a given large-scale interconnected system and a
global performance objective, an optimization problem is formulated to search
for admissible dissipativity properties of each subsystems. Local control laws
are then synthesized to certify the relevant dissipativity properties.
Moreover, the term localization is introduced to describe a finite collection
of syntheses problems, for the local subsystems, which are a feasibility
certificate for the global synthesis problem. Consequently, the problem of
localizing the global problem to a smaller collection of disjointed sets of
subsystems, called groups, is considered. This works looks promising as another
way of looking at decentralized control and also as a way of doing performance
specifications for components in a large-scale system
Generalization Bounds in the Predict-then-Optimize Framework
The predict-then-optimize framework is fundamental in many practical
settings: predict the unknown parameters of an optimization problem, and then
solve the problem using the predicted values of the parameters. A natural loss
function in this environment is to consider the cost of the decisions induced
by the predicted parameters, in contrast to the prediction error of the
parameters. This loss function was recently introduced in Elmachtoub and Grigas
(2017) and referred to as the Smart Predict-then-Optimize (SPO) loss. In this
work, we seek to provide bounds on how well the performance of a prediction
model fit on training data generalizes out-of-sample, in the context of the SPO
loss. Since the SPO loss is non-convex and non-Lipschitz, standard results for
deriving generalization bounds do not apply.
We first derive bounds based on the Natarajan dimension that, in the case of
a polyhedral feasible region, scale at most logarithmically in the number of
extreme points, but, in the case of a general convex feasible region, have
linear dependence on the decision dimension. By exploiting the structure of the
SPO loss function and a key property of the feasible region, which we denote as
the strength property, we can dramatically improve the dependence on the
decision and feature dimensions. Our approach and analysis rely on placing a
margin around problematic predictions that do not yield unique optimal
solutions, and then providing generalization bounds in the context of a
modified margin SPO loss function that is Lipschitz continuous. Finally, we
characterize the strength property and show that the modified SPO loss can be
computed efficiently for both strongly convex bodies and polytopes with an
explicit extreme point representation.Comment: Preliminary version in NeurIPS 201
A sequential semidefinite programming method and an application in passive reduced-order modeling
We consider the solution of nonlinear programs with nonlinear
semidefiniteness constraints. The need for an efficient exploitation of the
cone of positive semidefinite matrices makes the solution of such nonlinear
semidefinite programs more complicated than the solution of standard nonlinear
programs. In particular, a suitable symmetrization procedure needs to be chosen
for the linearization of the complementarity condition. The choice of the
symmetrization procedure can be shifted in a very natural way to certain linear
semidefinite subproblems, and can thus be reduced to a well-studied problem.
The resulting sequential semidefinite programming (SSP) method is a
generalization of the well-known SQP method for standard nonlinear programs. We
present a sensitivity result for nonlinear semidefinite programs, and then
based on this result, we give a self-contained proof of local quadratic
convergence of the SSP method. We also describe a class of nonlinear
semidefinite programs that arise in passive reduced-order modeling, and we
report results of some numerical experiments with the SSP method applied to
problems in that class
A receding horizon generalization of pointwise min-norm controllers
Control Lyapunov functions (CLFs) are used in conjunction with receding horizon control to develop a new class of receding horizon control schemes. In the process, strong connections between the seemingly disparate approaches are revealed, leading to a unified picture that ties together the notions of pointwise min-norm, receding horizon, and optimal control. This framework is used to develop a CLF based receding horizon scheme, of which a special case provides an appropriate extension of Sontag's formula. The scheme is first presented as an idealized continuous-time receding horizon control law. The issue of implementation under discrete-time sampling is then discussed as a modification. These schemes are shown to possess a number of desirable theoretical and implementation properties. An example is provided, demonstrating their application to a nonlinear control problem. Finally, stronger connections to both optimal and pointwise min-norm control are proved
An extension of the projected gradient method to a Banach space setting with application in structural topology optimization
For the minimization of a nonlinear cost functional under convex
constraints the relaxed projected gradient process is
a well known method. The analysis is classically performed in a Hilbert space
. We generalize this method to functionals which are differentiable in a
Banach space. Thus it is possible to perform e.g. an gradient method if
is only differentiable in . We show global convergence using
Armijo backtracking in and allow the inner product and the scaling
to change in every iteration. As application we present a
structural topology optimization problem based on a phase field model, where
the reduced cost functional is differentiable in . The
presented numerical results using the inner product and a pointwise
chosen metric including second order information show the expected mesh
independency in the iteration numbers. The latter yields an additional, drastic
decrease in iteration numbers as well as in computation time. Moreover we
present numerical results using a BFGS update of the inner product for
further optimization problems based on phase field models
- …