An Interior-Point Path-Following Method to Compute Stationary Equilibria in Stochastic Games

Subgame perfect equilibrium in stationary strategies (SSPE) is the most important solution concept used in applications of stochastic games, which makes it imperative to develop efficient numerical methods to compute an SSPE. For this purpose, this paper develops an interior-point path-following method (IPM), which remedies a number of issues with the existing method called stochastic linear tracing procedure (SLTP). The homotopy system of IPM is derived from the optimality conditions of an artificial barrier game, whose objective function is a combination of the original payoff function and a logarithmic term. Unlike SLTP, the starting stationary strategy profile can be arbitrarily chosen and IPM does not need switching between different systems of equations. The use of a perturbation term makes IPM applicable to all stochastic games, whereas SLTP only works for a generic stochastic game. A transformation of variables reduces the number of equations and variables of by roughly one half. Numerical results show that our method is more than three times as efficient as SLTP

Supervised Random Walks: Predicting and Recommending Links in Social Networks

Predicting the occurrence of links is a fundamental problem in networks. In the link prediction problem we are given a snapshot of a network and would like to infer which interactions among existing members are likely to occur in the near future or which existing interactions are we missing. Although this problem has been extensively studied, the challenge of how to effectively combine the information from the network structure with rich node and edge attribute data remains largely open. We develop an algorithm based on Supervised Random Walks that naturally combines the information from the network structure with node and edge level attributes. We achieve this by using these attributes to guide a random walk on the graph. We formulate a supervised learning task where the goal is to learn a function that assigns strengths to edges in the network such that a random walker is more likely to visit the nodes to which new links will be created in the future. We develop an efficient training algorithm to directly learn the edge strength estimation function. Our experiments on the Facebook social graph and large collaboration networks show that our approach outperforms state-of-the-art unsupervised approaches as well as approaches that are based on feature extraction

Deflation for semismooth equations

Variational inequalities can in general support distinct solutions. In this paper we study an algorithm for computing distinct solutions of a variational inequality, without varying the initial guess supplied to the solver. The central idea is the combination of a semismooth Newton method with a deflation operator that eliminates known solutions from consideration. Given one root of a semismooth residual, deflation constructs a new problem for which a semismooth Newton method will not converge to the known root, even from the same initial guess. This enables the discovery of other roots. We prove the effectiveness of the deflation technique under the same assumptions that guarantee locally superlinear convergence of a semismooth Newton method. We demonstrate its utility on various finite- and infinite-dimensional examples drawn from constrained optimization, game theory, economics and solid mechanics.Comment: 24 pages, 3 figure

Melding the Data-Decisions Pipeline: Decision-Focused Learning for Combinatorial Optimization

Creating impact in real-world settings requires artificial intelligence techniques to span the full pipeline from data, to predictive models, to decisions. These components are typically approached separately: a machine learning model is first trained via a measure of predictive accuracy, and then its predictions are used as input into an optimization algorithm which produces a decision. However, the loss function used to train the model may easily be misaligned with the end goal, which is to make the best decisions possible. Hand-tuning the loss function to align with optimization is a difficult and error-prone process (which is often skipped entirely). We focus on combinatorial optimization problems and introduce a general framework for decision-focused learning, where the machine learning model is directly trained in conjunction with the optimization algorithm to produce high-quality decisions. Technically, our contribution is a means of integrating common classes of discrete optimization problems into deep learning or other predictive models, which are typically trained via gradient descent. The main idea is to use a continuous relaxation of the discrete problem to propagate gradients through the optimization procedure. We instantiate this framework for two broad classes of combinatorial problems: linear programs and submodular maximization. Experimental results across a variety of domains show that decision-focused learning often leads to improved optimization performance compared to traditional methods. We find that standard measures of accuracy are not a reliable proxy for a predictive model's utility in optimization, and our method's ability to specify the true goal as the model's training objective yields substantial dividends across a range of decision problems.Comment: Full version of paper accepted at AAAI 201

Liquidity constraints and time non-separable preferences: simulating models with large state spaces

This paper presents an alternative method for the stochastic simulation of nonlinear and possibly non-differentiable models with large state spaces. We compare our method to other existing methods, and show that the accuracy is satisfactory. We then use the method to analyze the features of an intertemporal optimizing consumption-saving model, when the utility function is time non-separable and when liquidity constraints are imposed. Two non-separabilities are studied, habit persistence and durability of the commodity. As the model has no closed-form solution, we compute deterministic and stochastic solution paths. It enables us to compare income and consumption volatility, and to describe the density of consumption under the different hypotheses on the utility function

Computing Perfect Stationary Equilibria in Stochastic Games

The notion of stationary equilibrium is one of the most crucial solution concepts in stochastic games. However, a stochastic game can have multiple stationary equilibria, some of which may be unstable or counterintuitive. As a refinement of stationary equilibrium, we extend the concept of perfect equilibrium in strategic games to stochastic games and formulate the notion of perfect stationary equilibrium (PeSE). To further promote its applications, we develop a differentiable homotopy method to compute such an equilibrium. We incorporate vanishing logarithmic barrier terms into the payoff functions, thereby constituting a logarithmic-barrier stochastic game. As a result of this barrier game, we attain a continuously differentiable homotopy system. To reduce the number of variables in the homotopy system, we eliminate the Bellman equations through a replacement of variables and derive an equivalent system. We use the equivalent system to establish the existence of a smooth path, which starts from an arbitrary total mixed strategy profile and ends at a PeSE. Extensive numerical experiments further affirm the effectiveness and efficiency of the method