Solving monotone stochastic variational inequalities and complementarity problems by progressive hedging

The concept of a stochastic variational inequality has recently been articulated in a new way that is able to cover, in particular, the optimality conditions for a multistage stochastic programming problem. One of the long-standing methods for solving such an optimization problem under convexity is the progressive hedging algorithm. That approach is demonstrated here to be applicable also to solving multistage stochastic variational inequality problems under monotonicity, thus increasing the range of applications for progressive hedging. Stochastic complementarity problems as a special case are explored numerically in a linear two-stage formulation

A prediction-correction ADMM for multistage stochastic variational inequalities

The multistage stochastic variational inequality is reformulated into a variational inequality with separable structure through introducing a new variable. The prediction-correction ADMM which was originally proposed in [B.-S. He, L.-Z. Liao and M.-J. Qian, J. Comput. Math., 24 (2006), 693--710] for solving deterministic variational inequalities in finite dimensional spaces is adapted to solve the multistage stochastic variational inequality. Weak convergence of the sequence generated by that algorithm is proved under the conditions of monotonicity and Lipschitz continuity. When the sample space is a finite set, the corresponding multistage stochastic variational inequality is defined on a finite dimensional Hilbert space and the strong convergence of the sequence naturally holds true. A numerical example in that case is given to show the efficiency of the algorithm

Two-stage quadratic games under uncertainty and their solution by progressive hedging algorithms

A model of a two-stage N-person noncooperative game under uncertainty is studied, in which at the first stage each player solves a quadratic program parameterized by other players’ decisions and then at the second stage the player solves a recourse quadratic program parameterized by the realization of a random vector, the second-stage decisions of other players, and the first-stage decisions of all players. The problem of finding a Nash equilibrium of this game is shown to be equivalent to a stochastic linear complementarity problem. A linearly convergent progressive hedging algorithm is proposed for finding a Nash equilibrium if the resulting complementarity problem is monotone. For the nonmonotone case, it is shown that, as long as the complementarity problem satisfies an additional elicitability condition, the progressive hedging algorithm can be modified to find a local Nash equilibrium at a linear rate. The elicitability condition is reminiscent of the sufficient second-order optimality condition in nonlinear programming. Various numerical experiments indicate that the progressive hedging algorithms are efficient for mid-sized problems. In particular, the numerical results include a comparison with the best response method that is commonly adopted in the literature

A Novel Euler's Elastica based Segmentation Approach for Noisy Images via using the Progressive Hedging Algorithm

Euler's Elastica based unsupervised segmentation models have strong capability of completing the missing boundaries for existing objects in a clean image, but they are not working well for noisy images. This paper aims to establish a Euler's Elastica based approach that properly deals with random noises to improve the segmentation performance for noisy images. We solve the corresponding optimization problem via using the progressive hedging algorithm (PHA) with a step length suggested by the alternating direction method of multipliers (ADMM). Technically, all the simplified convex versions of the subproblems derived from the major framework of PHA can be obtained by using the curvature weighted approach and the convex relaxation method. Then an alternating optimization strategy is applied with the merits of using some powerful accelerating techniques including the fast Fourier transform (FFT) and generalized soft threshold formulas. Extensive experiments have been conducted on both synthetic and real images, which validated some significant gains of the proposed segmentation models and demonstrated the advantages of the developed algorithm

The Elicited Progressive Decoupling Algorithm: A Note on the Rate of Convergence and a Preliminary Numerical Experiment on the Choice of Parameters

The paper studies the progressive decoupling algorithm (PDA) of Rockafellar and focuses on the elicited version of the algorithm. Based on a generalized Yosida-regularization of Spingarn’s partial inverse of an elicitable operator, it is shown that the elicited progressive decoupling algorithm (EPDA), in a particular nonmonotone setting, linearly converges at a rate that could be viewed as the rate of a rescaled PDA, which may provide certain guidance to the selection of the parameters in computational practice. A preliminary numerical experiment shows that the choice of the elicitation constant has an impact on the efficiency of the EPDA. It is also observed that the influence of the elicitation constant is generally weaker than the proximal constant in the algorithm

A Model of Multistage Risk-Averse Stochastic Optimization and its Solution by Scenario-Based Decomposition Algorithms

Stochastic optimization models based on risk-averse measures are of essential importance in financial management and business operations. This paper studies new algorithms for a popular class of these models, namely, the mean-deviation models in multistage decision making under uncertainty. It is argued that these types of problems enjoy a scenario-decomposable structure, which could be utilized in an efficient progressive hedging procedure. In case that linkage constraints arise in reformulations of the original problem, a Lagrange progressive hedging algorithm could be utilized to solve the reformulated problem. Convergence results of the algorithms are obtained based on the recent development of the Lagrangian form of stochastic variational inequalities. Numerical results are provided to show the effectiveness of the proposed algorithms

A Gradually Reinforced Sample-Average-Approximation Differentiable Homotopy Method for a System of Stochastic Equations

This paper intends to apply the sample-average-approximation (SAA) scheme to solve a system of stochastic equations (SSE), which has many applications in a variety of fields. The SAA is an effective paradigm to address risks and uncertainty in stochastic models from the perspective of Monte Carlo principle. Nonetheless, a numerical conflict arises from the sample size of SAA when one has to make a tradeoff between the accuracy of solutions and the computational cost. To alleviate this issue, we incorporate a gradually reinforced SAA scheme into a differentiable homotopy method and develop a gradually reinforced sample-average-approximation (GRSAA) differentiable homotopy method in this paper. By introducing a series of continuously differentiable functions of the homotopy parameter $t$ ranging between zero and one, we establish a differentiable homotopy system, which is able to gradually increase the sample size of SAA as $t$ descends from one to zero. The set of solutions to the homotopy system contains an everywhere smooth path, which starts from an arbitrary point and ends at a solution to the SAA with any desired accuracy. The GRSAA differentiable homotopy method serves as a bridge to link the gradually reinforced SAA scheme and a differentiable homotopy method and retains the nice property of global convergence the homotopy method possesses while greatly reducing the computational cost for attaining a desired solution to the original SSE. Several numerical experiments further confirm the effectiveness and efficiency of the proposed method