121 research outputs found

    Well-Behavior, Well-Posedness and Nonsmooth Analysis

    Get PDF
    AMS subject classification: 90C30, 90C33.We survey the relationships between well-posedness and well-behavior. The latter notion means that any critical sequence (xn) of a lower semicontinuous function f on a Banach space is minimizing. Here “critical” means that the remoteness of the subdifferential ∂f(xn) of f at xn (i.e. the distance of 0 to ∂f(xn)) converges to 0. The objective function f is not supposed to be convex or smooth and the subdifferential ∂ is not necessarily the usual Fenchel subdifferential. We are thus led to deal with conditions ensuring that a growth property of the subdifferential (or the derivative) of a function implies a growth property of the function itself. Both qualitative questions and quantitative results are considered

    Differentiable Game Mechanics

    Get PDF
    Deep learning is built on the foundational guarantee that gradient descent on an objective function converges to local minima. Unfortunately, this guarantee fails in settings, such as generative adversarial nets, that exhibit multiple interacting losses. The behavior of gradient-based methods in games is not well understood -- and is becoming increasingly important as adversarial and multi-objective architectures proliferate. In this paper, we develop new tools to understand and control the dynamics in n-player differentiable games. The key result is to decompose the game Jacobian into two components. The first, symmetric component, is related to potential games, which reduce to gradient descent on an implicit function. The second, antisymmetric component, relates to Hamiltonian games, a new class of games that obey a conservation law akin to conservation laws in classical mechanical systems. The decomposition motivates Symplectic Gradient Adjustment (SGA), a new algorithm for finding stable fixed points in differentiable games. Basic experiments show SGA is competitive with recently proposed algorithms for finding stable fixed points in GANs -- while at the same time being applicable to, and having guarantees in, much more general cases.Comment: JMLR 2019, journal version of arXiv:1802.0564

    Symbolic approaches and artificial intelligence algorithms for solving multi-objective optimisation problems

    Get PDF
    Problems that have more than one objective function are of great importance in engineering sciences and many other disciplines. This class of problems are known as multi-objective optimisation problems (or multicriteria). The difficulty here lies in the conflict between the various objective functions. Due to this conflict, one cannot find a single ideal solution which simultaneously satisfies all the objectives. But instead one can find the set of Pareto-optimal solutions (Pareto-optimal set) and consequently the Pareto-optimal front is established. Finding these solutions plays an important role in multi-objective optimisation problems and mathematically the problem is considered to be solved when the Pareto-optimal set, i.e. the set of all compromise solutions is found. The Pareto-optimal set may contain information that can help the designer make a decision and thus arrive at better trade-off solutions. The aim of this research is to develop new multi-objective optimisation symbolic algorithms capable of detecting relationship(s) among decision variables that can be used for constructing the analytical formula of Pareto-optimal front based on the extension of the current optimality conditions. A literature survey of theoretical and evolutionary computation techniques for handling multiple objectives, constraints and variable interaction highlights a lack of techniques to handle variable interaction. This research, therefore, focuses on the development of techniques for detecting the relationships between the decision variables (variable interaction) in the presence of multiple objectives and constraints. It attempts to fill the gap in this research by formally extending the theoretical results (optimality conditions). The research then proposes first-order multi-objective symbolic algorithm or MOSA-I and second-order multi-objective symbolic algorithm or MOSA-II that are capable of detecting the variable interaction. The performance of these algorithms is analysed and compared to a current state-of-the-art optimisation algorithm using popular test problems. The performance of the MOSA-II algorithm is finally validated using three appropriately chosen problems from literature. In this way, this research proposes a fully tested and validated methodology for dealing with multi-objective optimisation problems. In conclusion, this research proposes two new symbolic algorithms that are used for identifying the variable interaction responsible for constructing Pareto-optimal front among objectives in multi-objective optimisation problems. This is completed based on a development and relaxation of the first and second-order optimality conditions of Karush-Kuhn-Tucker.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

    Distributed Forward-Backward Methods for Ring Networks

    Get PDF
    In this work, we propose and analyse forward-backward-type algorithms for finding a zero of the sum of finitely many monotone operators, which are not based on reduction to a two operator inclusion in the product space. Each iteration of the studied algorithms requires one resolvent evaluation per set-valued operator, one forward evaluation per cocoercive operator, and two forward evaluations per monotone operator. Unlike existing methods, the structure of the proposed algorithms are suitable for distributed, decentralised implementation in ring networks without needing global summation to enforce consensus between nodes.Comment: 19 page
    corecore