17 research outputs found
Adaptive Relaxed ADMM: Convergence Theory and Practical Implementation
Many modern computer vision and machine learning applications rely on solving
difficult optimization problems that involve non-differentiable objective
functions and constraints. The alternating direction method of multipliers
(ADMM) is a widely used approach to solve such problems. Relaxed ADMM is a
generalization of ADMM that often achieves better performance, but its
efficiency depends strongly on algorithm parameters that must be chosen by an
expert user. We propose an adaptive method that automatically tunes the key
algorithm parameters to achieve optimal performance without user oversight.
Inspired by recent work on adaptivity, the proposed adaptive relaxed ADMM
(ARADMM) is derived by assuming a Barzilai-Borwein style linear gradient. A
detailed convergence analysis of ARADMM is provided, and numerical results on
several applications demonstrate fast practical convergence.Comment: CVPR 201
Small-Sample Inferred Adaptive Recoding for Batched Network Coding
Batched network coding is a low-complexity network coding solution to
feedbackless multi-hop wireless packet network transmission with packet loss.
The data to be transmitted is encoded into batches where each of which consists
of a few coded packets. Unlike the traditional forwarding strategy, the
intermediate network nodes have to perform recoding, which generates recoded
packets by network coding operations restricted within the same batch. Adaptive
recoding is a technique to adapt the fluctuation of packet loss by optimizing
the number of recoded packets per batch to enhance the throughput. The input
rank distribution, which is a piece of information regarding the batches
arriving at the node, is required to apply adaptive recoding. However, this
distribution is not known in advance in practice as the incoming link's channel
condition may change from time to time. On the other hand, to fully utilize the
potential of adaptive recoding, we need to have a good estimation of this
distribution. In other words, we need to guess this distribution from a few
samples so that we can apply adaptive recoding as soon as possible. In this
paper, we propose a distributionally robust optimization for adaptive recoding
with a small-sample inferred prediction of the input rank distribution. We
develop an algorithm to efficiently solve this optimization with the support of
theoretical guarantees that our optimization's performance would constitute as
a confidence lower bound of the optimal throughput with high probability.Comment: 7 pages, 2 figures, accepted in ISIT-21, appendix adde
Acceleration of the PDHGM on strongly convex subspaces
We propose several variants of the primal-dual method due to Chambolle and
Pock. Without requiring full strong convexity of the objective functions, our
methods are accelerated on subspaces with strong convexity. This yields mixed
rates, with respect to initialisation and with respect to
the dual sequence, and the residual part of the primal sequence. We demonstrate
the efficacy of the proposed methods on image processing problems lacking
strong convexity, such as total generalised variation denoising and total
variation deblurring
Block-proximal methods with spatially adapted acceleration
We study and develop (stochastic) primal--dual block-coordinate descent
methods for convex problems based on the method due to Chambolle and Pock. Our
methods have known convergence rates for the iterates and the ergodic gap:
if each block is strongly convex, if no convexity is
present, and more generally a mixed rate for strongly convex
blocks, if only some blocks are strongly convex. Additional novelties of our
methods include blockwise-adapted step lengths and acceleration, as well as the
ability to update both the primal and dual variables randomly in blocks under a
very light compatibility condition. In other words, these variants of our
methods are doubly-stochastic. We test the proposed methods on various image
processing problems, where we employ pixelwise-adapted acceleration