Explore Aggressively, Update Conservatively: Stochastic Extragradient Methods with Variable Stepsize Scaling

Owing to their stability and convergence speed, extragradient methods have become a staple for solving large-scale saddle-point problems in machine learning. The basic premise of these algorithms is the use of an extrapolation step before performing an update; thanks to this exploration step, extra-gradient methods overcome many of the non-convergence issues that plague gradient descent/ascent schemes. On the other hand, as we show in this paper, running vanilla extragradient with stochastic gradients may jeopardize its convergence, even in simple bilinear models. To overcome this failure, we investigate a double stepsize extragradient algorithm where the exploration step evolves at a more aggressive time-scale compared to the update step. We show that this modification allows the method to converge even with stochastic gradients, and we derive sharp convergence rates under an error bound condition.Comment: In Advances in Neural Information Processing Systems 33 (NeurIPS 2020); 29 pages, 5 figure

Monotone Inclusions, Acceleration and Closed-Loop Control

We propose and analyze a new dynamical system with a closed-loop control law in a Hilbert space $\mathcal{H}$, aiming to shed light on the acceleration phenomenon for \textit{monotone inclusion} problems, which unifies a broad class of optimization, saddle point and variational inequality (VI) problems under a single framework. Given $A: \mathcal{H} \rightrightarrows \mathcal{H}$ that is maximal monotone, we propose a closed-loop control system that is governed by the operator $I - (I + \lambda(t)A)^{-1}$, where a feedback law $\lambda(\cdot)$ is tuned by the resolution of the algebraic equation $\lambda(t)\|(I + \lambda(t)A)^{-1}x(t) - x(t)\|^{p-1} = \theta$ for some $\theta > 0$. Our first contribution is to prove the existence and uniqueness of a global solution via the Cauchy-Lipschitz theorem. We present a simple Lyapunov function for establishing the weak convergence of trajectories via the Opial lemma and strong convergence results under additional conditions. We then prove a global ergodic convergence rate of $O(t^{-(p+1)/2})$ in terms of a gap function and a global pointwise convergence rate of $O(t^{-p/2})$ in terms of a residue function. Local linear convergence is established in terms of a distance function under an error bound condition. Further, we provide an algorithmic framework based on the implicit discretization of our system in a Euclidean setting, generalizing the large-step HPE framework. Although the discrete-time analysis is a simplification and generalization of existing analyses for a bounded domain, it is largely motivated by the above continuous-time analysis, illustrating the fundamental role that the closed-loop control plays in acceleration in monotone inclusion. A highlight of our analysis is a new result concerning $p^{th}$-order tensor algorithms for monotone inclusion problems, complementing the recent analysis for saddle point and VI problems.Comment: Accepted by Mathematics of Operations Research; 42 Page

Iterative Methods for the Elasticity Imaging Inverse Problem

Cancers of the soft tissue reign among the deadliest diseases throughout the world and effective treatments for such cancers rely on early and accurate detection of tumors within the interior of the body. One such diagnostic tool, known as elasticity imaging or elastography, uses measurements of tissue displacement to reconstruct the variable elasticity between healthy and unhealthy tissue inside the body. This gives rise to a challenging parameter identification inverse problem, that of identifying the Lamé parameter μ in a system of partial differential equations in linear elasticity. Due to the near incompressibility of human tissue, however, common techniques for solving the direct and inverse problems are rendered ineffective due to a phenomenon known as the “locking effect”. Alternative methods, such as mixed finite element methods, must be applied to overcome this complication. Using these methods, this work reposes the problem as a generalized saddle point problem along with a presentation of several optimization formulations, including the modified output least squares (MOLS), energy output least squares (EOLS), and equation error (EE) frameworks, for solving the elasticity imaging inverse problem. Subsequently, numerous iterative optimization methods, including gradient, extragradient, and proximal point methods, are explored and applied to solve the related optimization problem. Implementations of all of the iterative techniques under consideration are applied to all of the developed optimization frameworks using a representative numerical example in elasticity imaging. A thorough analysis and comparison of the methods is subsequently presented

An Improvement of Global Error Bound for the Generalized Nonlinear Complementarity Problem over a Polyhedral Cone

We consider the global error bound for the generalized nonlinear complementarity problem over a polyhedral cone (GNCP). By a new technique, we establish an easier computed global error bound for the GNCP under weaker conditions, which improves the result obtained by for GNCP

Global Error Bound Estimation for the Generalized Nonlinear Complementarity Problem over a Closed Convex Cone

The global error bound estimation for the generalized nonlinear complementarity problem over a closed convex cone (GNCP) is considered. To obtain a global error bound for the GNCP, we first develop an equivalent reformulation of the problem. Based on this, a global error bound for the GNCP is established. The results obtained in this paper can be taken as an extension of previously known results

How to increase the impact of disaster relief: A study of transportation rates, framework agreements and product distribution

Due to restricted budgets of relief organizations, costs of hiring transportation service providers steer distribution decisions and limit the impact of disaster relief. To improve the success of future humanitarian operations, it is of paramount importance to understand this relationship in detail and to identify mitigation actions, always considering the interdependencies between multiple independent actors in humanitarian logistics. In this paper, we develop a game-theoretic model in order to investigate the influence of transportation costs on distribution decisions in long-term relief operations and to evaluate measures for improving the fulfillment of beneficiary needs. The equilibrium of the model is a Generalized Nash Equilibrium, which has had few applications in the supply chain context to date. We formulate it, utilizing the construct of a Variational Equilibrium, as a Variational Inequality and perform numerical simulations in order to study the effects of three interventions: an increase in carrier competition, a reduction of transportation costs and an extension of framework agreements. The results yield important implications for policy makers and humanitarian organizations (HOs). Increasing the number of preselected carriers strengthens the bargaining power of HOs and improves impact up to a certain limit. The limit is reached when carriers set framework rates equal to transportation unit costs. Reductions of transportation costs have a consistently positive, but decreasing marginal benefit without any upper bound. They provide the highest benefit when the bargaining power of HOs is weak. On the contrary, extending framework agreements enables most improvements when the bargaining power of HOs is strong

Simulação numérica de problemas de contato estrutural com elementos mortar-based de alta-ordem

Orientadores: Marco Lúcio Bittencourt, Sérgio Persival Baroncini ProençaTese (doutorado) - Universidade Estadual de Campinas, Faculdade de Engenharia MecânicaResumo: No presente trabalho, apresenta-se um novo elemento de contato do tipo mortar de alta ordem para solução de problemas bi e tridimensionais de contato com atrito, considerando pequenas e grandes deformações. Considera-se o modelo Neo-Hookeano de material hiperelástico compressível isotrópico. O mapeamento das superfícies curvas dos elementos de contato foi realizado com NURBS (Non-Uniform Rational B-Splines). Verifica-se o desempenho do elemento em pequenas e grandes deformações com soluções conhecidas na literatura, e apresentam-se estudos de precisão de solução e tempo de processamento para refinamentos p e h. Como aplicações em dinâmica de contato, apresentam-se soluções para problemas de impacto estrutural com o elemento de contato de alta ordem. Os resultados comparativos mostram que a interpolação de alta ordem é uma estratégia com desempenho superior para os problemas de contato analisados, melhorando a precisão da solução das tensões e forças geradas pelo contato com um menor tempo de processamentoAbstract: In the present work, we present a new high-order mortar-based contact element for solution of two and three-dimensional frictional contact problems, considering small and large deformations. The Neo-Hookean isotropic compressible hyperelastic material model was considered. The mapping of curved surfaces of elements was performed with Non-Uniform Rational B-Splines (NURBS). We verify the behavior of the element in small and large deformation with known solutions in the literature and present studies of accuracy and processing time of the contact elements for h- and p-refinements. As applications in dynamic contact, we present solutions for structural impact problems with high-order contact element. The comparative results show that the high-order interpolation is a strategy with superior performance for the contact problems analysed improving the solution accuracy of the stresses and forces generated by contact with a lower processing timeDoutoradoMecanica dos Sólidos e Projeto MecanicoDoutor em Engenharia Mecânica2013/10523-0FAPES

EFFECT – Efficient finite element code

The theme of this dissertation is the finite element method applied to mechanical structures. A new finite element program is developed that, besides executing different types of structural analysis, also allows the calculation of the derivatives of structural performances using the continuum method of design sensitivities analysis, with the purpose of allowing, in combination with the mathematical programming algorithms found in the commercial software MATLAB, to solve structural optimization problems. The program is called EFFECT – Efficient Finite Element Code. The object-oriented programming paradigm and specifically the C ++ programming language are used for program development. The main objective of this dissertation is to design EFFECT so that it can constitute, in this stage of development, the foundation for a program with analysis capacities similar to other open source finite element programs. In this first stage, 6 elements are implemented for linear analysis: 2-dimensional truss (Truss2D), 3-dimensional truss (Truss3D), 2-dimensional beam (Beam2D), 3-dimensional beam (Beam3D), triangular shell element (Shell3Node) and quadrilateral shell element (Shell4Node). The shell elements combine two distinct elements, one for simulating the membrane behavior and the other to simulate the plate bending behavior. The non-linear analysis capability is also developed, combining the corotational formulation with the Newton-Raphson iterative method, but at this stage is only avaiable to solve problems modeled with Beam2D elements subject to large displacements and rotations, called nonlinear geometric problems. The design sensitivity analysis capability is implemented in two elements, Truss2D and Beam2D, where are included the procedures and the analytic expressions for calculating derivatives of displacements, stress and volume performances with respect to 5 different design variables types. Finally, a set of test examples were created to validate the accuracy and consistency of the result obtained from EFFECT, by comparing them with results published in the literature or obtained with the ANSYS commercial finite element code

Methods for computer aided inspection of geometric tolerances

This thesis investigates computational methods for assessing tolerance specifications of geometric features in a context of computer aided inspection. It is concerned with checking the sampled features for containment within tolerance zones specified at the design stage, not with explicit shape measurement. The significance of this difference is highlighted when two or more features are to be inspected in combination. The approach adopted is to express the tolerance information as a set of inequality constraints and then to seek efficient methods for determining the feasibility of the set, that is whether all the constraints can be simultaneously satisfied. Roundness inspection is used to introduce all the concepts of the new formulations. By linearisation of the constraints, a standard approximation in roundness measurement, a new algorithm is implemented which provides a “GO-NOGO” result of inspection by checking for feasibility in a highly efficient way. This algorithmic approach is then extended to other inspection situations where naturally linear constraints or valid linearisation occur. Since there are many inspection cases where linearisation is not appropriate, non-linear optimisation techniques are then investigated for their effectiveness in feasibility testing. The inspection of arrays of circular features is used here as a typical test case. Genetic search methods are explored as a possible alternative to formal non-linear programming and guidelines for their efficient use for this problem are proposed. These methods are then compared and contrasted with formal methods, particularly generalised reduced gradient (GRG) and sequential quadratic programming (SQP). The linear algorithm is shown to be the most efficient when it can be used, although all techniques were fast enough for on-line use with modest sized data sets. Currently all the non-linear methods are too expensive for routine use on large data sets. GRG is recommended as having the most favourable combination of good and bad features, but there is some evidence that genetic search might be relatively more efficient for more complex inspection problems