Distributed Multi-Agent Optimization with State-Dependent Communication

We study distributed algorithms for solving global optimization problems in which the objective function is the sum of local objective functions of agents and the constraint set is given by the intersection of local constraint sets of agents. We assume that each agent knows only his own local objective function and constraint set, and exchanges information with the other agents over a randomly varying network topology to update his information state. We assume a state-dependent communication model over this topology: communication is Markovian with respect to the states of the agents and the probability with which the links are available depends on the states of the agents. In this paper, we study a projected multi-agent subgradient algorithm under state-dependent communication. The algorithm involves each agent performing a local averaging to combine his estimate with the other agents' estimates, taking a subgradient step along his local objective function, and projecting the estimates on his local constraint set. The state-dependence of the communication introduces significant challenges and couples the study of information exchange with the analysis of subgradient steps and projection errors. We first show that the multi-agent subgradient algorithm when used with a constant stepsize may result in the agent estimates to diverge with probability one. Under some assumptions on the stepsize sequence, we provide convergence rate bounds on a "disagreement metric" between the agent estimates. Our bounds are time-nonhomogeneous in the sense that they depend on the initial starting time. Despite this, we show that agent estimates reach an almost sure consensus and converge to the same optimal solution of the global optimization problem with probability one under different assumptions on the local constraint sets and the stepsize sequence

Social networks : rational learning and information aggregation

Thesis (Ph. D.)--Massachusetts Institute of Technology, Sloan School of Management, Operations Research Center, 2009.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Cataloged from student-submitted PDF version of thesis.Includes bibliographical references (p. 137-140).This thesis studies the learning problem of a set of agents connected via a general social network. We address the question of how dispersed information spreads in social networks and whether the information is efficiently aggregated in large societies. The models developed in this thesis allow us to study the learning behavior of rational agents embedded in complex networks. We analyze the perfect Bayesian equilibrium of a dynamic game where each agent sequentially receives a signal about an underlying state of the world, observes the past actions of a stochastically-generated neighborhood of individuals, and chooses one of two possible actions. The stochastic process generating the neighborhoods defines the network topology (social network). We characterize equilibria for arbitrary stochastic and deterministic social networks and characterize the conditions under which there will be asymptotic learning -- that is, the conditions under which, as the social network becomes large, the decisions of the individuals converge (in probability) to the right action. We show that when private beliefs are unbounded (meaning that the implied likelihood ratios are unbounded), there will be asymptotic learning as long as there is some minimal amount of expansion in observations. This result therefore establishes that, with unbounded private beliefs, there will be asymptotic learning in almost all reasonable social networks. Furthermore, we provide bounds on the speed of learning for some common network topologies. We also analyze when learning occurs when the private beliefs are bounded.(cont.) We show that asymptotic learning does not occur in many classes of network topologies, but, surprisingly, it happens in a family of stochastic networks that has infinitely many agents observing the actions of neighbors that are not sufficiently persuasive. Finally, we characterize equilibria in a generalized environment with heterogeneity of preferences and show that, contrary to a nave intuition, greater diversity (heterogeneity) 3 facilitates asymptotic learning when agents observe the full history of past actions. In contrast, we show that heterogeneity of preferences hinders information aggregation when each agent observes only the action of a single neighbor.by Ilan Lobel.Ph.D

Contextual Inverse Optimization: Offline and Online Learning

We study the problems of offline and online contextual optimization with feedback information, where instead of observing the loss, we observe, after-the-fact, the optimal action an oracle with full knowledge of the objective function would have taken. We aim to minimize regret, which is defined as the difference between our losses and the ones incurred by an all-knowing oracle. In the offline setting, the decision-maker has information available from past periods and needs to make one decision, while in the online setting, the decision-maker optimizes decisions dynamically over time based a new set of feasible actions and contextual functions in each period. For the offline setting, we characterize the optimal minimax policy, establishing the performance that can be achieved as a function of the underlying geometry of the information induced by the data. In the online setting, we leverage this geometric characterization to optimize the cumulative regret. We develop an algorithm that yields the first regret bound for this problem that is logarithmic in the time horizon

Reducing Marketplace Interference Bias Via Shadow Prices

Marketplace companies rely heavily on experimentation when making changes to the design or operation of their platforms. The workhorse of experimentation is the randomized controlled trial (RCT), or A/B test, in which users are randomly assigned to treatment or control groups. However, marketplace interference causes the Stable Unit Treatment Value Assumption (SUTVA) to be violated, leading to bias in the standard RCT metric. In this work, we propose techniques for platforms to run standard RCTs and still obtain meaningful estimates despite the presence of marketplace interference. We specifically consider a generalized matching setting, in which the platform explicitly matches supply with demand via a linear programming algorithm. Our first proposal is for the platform to estimate the value of global treatment and global control via optimization. We prove that this approach is unbiased in the fluid limit. Our second proposal is to compare the average shadow price of the treatment and control groups rather than the total value accrued by each group. We prove that this technique corresponds to the correct first-order approximation (in a Taylor series sense) of the value function of interest even in a finite-size system. We then use this result to prove that, under reasonable assumptions, our estimator is less biased than the RCT estimator. At the heart of our result is the idea that it is relatively easy to model interference in matching-driven marketplaces since, in such markets, the platform intermediates the spillover.Comment: v3: new results on additional estimator (Two-LP), asymmetric experiments, and generalized settin

Bayesian Learning in Social Networks

We study the (perfect Bayesian) equilibrium of a model of learning over a general so- cial network. Each individual receives a signal about the underlying state of the world, observes the past actions of a stochastically-generated neighborhood of individuals, and chooses one of two possible actions. The stochastic process generating the neighborhoods de¯nes the network topology (social network). The special case where each individual observes all past actions has been widely studied in the literature. We characterize pure-strategy equilibria for arbitrary stochastic and deterministic social networks and characterize the conditions under which there will be asymptotic learning|that is, the conditions under which, as the social network becomes large, individuals converge (in probability) to taking the right action. We show that when private beliefs are unbounded (meaning that the implied likelihood ratios are unbounded), there will be asymptotic learning as long as there is some minimal amount of \expansion in observations". Our main theorem shows that when the probability that each individual observes some other individual from the recent past converges to one as the social network becomes large, un- bounded private beliefs are su±cient to ensure asymptotic learning. This theorem there- fore establishes that, with unbounded private beliefs, there will be asymptotic learning in almost all reasonable social networks. We also show that for most network topologies, when private beliefs are bounded, there will not be asymptotic learning. In addition, in contrast to the special case where all past actions are observed, asymptotic learning is possible even with bounded beliefs in certain stochastic network topologies.NYU, Stern, Center for Digital Economy Researc

Bayesian Learning in Social Networks

We study the perfect Bayesian equilibrium of a model of learning over a general social network. Each individual receives a signal about the underlying state of the world, observes the past actions of a stochastically-generated neighborhood of individuals, and chooses one of two possible actions. The stochastic process generating the neighborhoods defines the network topology (social network). The special case where each individual observes all past actions has been widely studied in the literature. We characterize pure-strategy equilibria for arbitrary stochastic and deterministic social networks and characterize the conditions under which there will be asymptotic learning -- that is, the conditions under which, as the social network becomes large, individuals converge (in probability) to taking the right action. We show that when private beliefs are unbounded (meaning that the implied likelihood ratios are unbounded), there will be asymptotic learning as long as there is some minimal amount of "expansion in observations". Our main theorem shows that when the probability that each individual observes some other individual from the recent past converges to one as the social network becomes large, unbounded private beliefs are sufficient to ensure asymptotic learning. This theorem therefore establishes that, with unbounded private beliefs, there will be asymptotic learning an almost all reasonable social networks. We also show that for most network topologies, when private beliefs are bounded, there will not be asymptotic learning. In addition, in contrast to the special case where all past actions are observed, asymptotic learning is possible even with bounded beliefs in certain stochastic network topologies.

Bayesian Learning in Social Networks

We study the (perfect Bayesian) equilibrium of a model of learning over a general so- cial network. Each individual receives a signal about the underlying state of the world, observes the past actions of a stochastically-generated neighborhood of individuals, and chooses one of two possible actions. The stochastic process generating the neighborhoods de¯nes the network topology (social network). The special case where each individual observes all past actions has been widely studied in the literature. We characterize pure-strategy equilibria for arbitrary stochastic and deterministic social networks and characterize the conditions under which there will be asymptotic learning|that is, the conditions under which, as the social network becomes large, individuals converge (in probability) to taking the right action. We show that when private beliefs are unbounded (meaning that the implied likelihood ratios are unbounded), there will be asymptotic learning as long as there is some minimal amount of \expansion in observations". Our main theorem shows that when the probability that each individual observes some other individual from the recent past converges to one as the social network becomes large, un- bounded private beliefs are su±cient to ensure asymptotic learning. This theorem there- fore establishes that, with unbounded private beliefs, there will be asymptotic learning in almost all reasonable social networks. We also show that for most network topologies, when private beliefs are bounded, there will not be asymptotic learning. In addition, in contrast to the special case where all past actions are observed, asymptotic learning is possible even with bounded beliefs in certain stochastic network topologies.NYU, Stern, Center for Digital Economy Researc

The benefit of sequentiality in social networks *

Abstract This paper examines the benefit of sequentiality in the social networks. We adopt the elegant theoretical framework proposed by We then examine the structure of optimal mechanism and allow for arbitrary sequence of players' moves. We show that starting from any fixed sequence, the aggregate contribution always goes up while making simultaneous-moving players move sequentially. This suggests a robust rule of thumbs -any local modification towards the sequential-move game is beneficial. Pushing this idea to the extreme, the optimal sequence turns out to be a chain structure, i.e., players should move one by one. Our results continue to hold when either players exhibit strategic substitutes instead or the network designer's goal is to maximize the players' aggregate payoff rather than the aggregate contribution