50,151 research outputs found
MM Algorithms for Geometric and Signomial Programming
This paper derives new algorithms for signomial programming, a generalization
of geometric programming. The algorithms are based on a generic principle for
optimization called the MM algorithm. In this setting, one can apply the
geometric-arithmetic mean inequality and a supporting hyperplane inequality to
create a surrogate function with parameters separated. Thus, unconstrained
signomial programming reduces to a sequence of one-dimensional minimization
problems. Simple examples demonstrate that the MM algorithm derived can
converge to a boundary point or to one point of a continuum of minimum points.
Conditions under which the minimum point is unique or occurs in the interior of
parameter space are proved for geometric programming. Convergence to an
interior point occurs at a linear rate. Finally, the MM framework easily
accommodates equality and inequality constraints of signomial type. For the
most important special case, constrained quadratic programming, the MM
algorithm involves very simple updates.Comment: 16 pages, 1 figur
Constrained Consensus
We present distributed algorithms that can be used by multiple agents to
align their estimates with a particular value over a network with time-varying
connectivity. Our framework is general in that this value can represent a
consensus value among multiple agents or an optimal solution of an optimization
problem, where the global objective function is a combination of local agent
objective functions. Our main focus is on constrained problems where the
estimate of each agent is restricted to lie in a different constraint set.
To highlight the effects of constraints, we first consider a constrained
consensus problem and present a distributed ``projected consensus algorithm''
in which agents combine their local averaging operation with projection on
their individual constraint sets. This algorithm can be viewed as a version of
an alternating projection method with weights that are varying over time and
across agents. We establish convergence and convergence rate results for the
projected consensus algorithm. We next study a constrained optimization problem
for optimizing the sum of local objective functions of the agents subject to
the intersection of their local constraint sets. We present a distributed
``projected subgradient algorithm'' which involves each agent performing a
local averaging operation, taking a subgradient step to minimize its own
objective function, and projecting on its constraint set. We show that, with an
appropriately selected stepsize rule, the agent estimates generated by this
algorithm converge to the same optimal solution for the cases when the weights
are constant and equal, and when the weights are time-varying but all agents
have the same constraint set.Comment: 35 pages. Included additional results, removed two subsections, added
references, fixed typo
Cycles in adversarial regularized learning
Regularized learning is a fundamental technique in online optimization,
machine learning and many other fields of computer science. A natural question
that arises in these settings is how regularized learning algorithms behave
when faced against each other. We study a natural formulation of this problem
by coupling regularized learning dynamics in zero-sum games. We show that the
system's behavior is Poincar\'e recurrent, implying that almost every
trajectory revisits any (arbitrarily small) neighborhood of its starting point
infinitely often. This cycling behavior is robust to the agents' choice of
regularization mechanism (each agent could be using a different regularizer),
to positive-affine transformations of the agents' utilities, and it also
persists in the case of networked competition, i.e., for zero-sum polymatrix
games.Comment: 22 pages, 4 figure
- …