Distributed optimization algorithms for multihop wireless networks

Recent technological advances in low-cost computing and communication hardware design have led to the feasibility of large-scale deployments of wireless ad hoc and sensor networks. Due to their wireless and decentralized nature, multihop wireless networks are attractive for a variety of applications. However, these properties also pose significant challenges to their developers and therefore require new types of algorithms. In cases where traditional wired networks usually rely on some kind of centralized entity, in multihop wireless networks nodes have to cooperate in a distributed and self-organizing manner. Additional side constraints, such as energy consumption, have to be taken into account as well. This thesis addresses practical problems from the domain of multihop wireless networks and investigates the application of mathematically justified distributed algorithms for solving them. Algorithms that are based on a mathematical model of an underlying optimization problem support a clear understanding of the assumptions and restrictions that are necessary in order to apply the algorithm to the problem at hand. Yet, the algorithms proposed in this thesis are simple enough to be formulated as a set of rules for each node to cooperate with other nodes in the network in computing optimal or approximate solutions. Nodes communicate with their neighbors by sending messages via wireless transmissions. Neither the size nor the number of messages grows rapidly with the size of the network. The thesis represents a step towards a unified understanding of the application of distributed optimization algorithms to problems from the domain of multihop wireless networks. The problems considered serve as examples for related problems and demonstrate the design methodology of obtaining distributed algorithms from mathematical optimization methods

Truthful Equilibria in Dynamic Bayesian Games

This paper characterizes an equilibrium payoff subset for dynamic Bayesian games as discounting vanishes. Monitoring is imperfect, transitions may depend on actions, types may be correlated and values may be interdependent. The focus is on equilibria in which players report truthfully. The characterization generalizes that for repeated games, reducing the analysis to static Bayesian games with transfers. With independent private values, the restriction to truthful equilibria is without loss, except for the punishment level; if players withhold their information during punishment-like phases, a folk theorem obtains

Truthful Equilibria in Dynamic Bayesian Games

This paper characterizes an equilibrium payoff subset for Markovian games with private information as discounting vanishes. Monitoring is imperfect, transitions may depend on actions, types be correlated and values interdependent. The focus is on equilibria in which players report truthfully. The characterization generalizes that for repeated games, reducing the analysis to static Bayesian games with transfers. With correlated types, results from mechanism design apply, yielding a folk theorem. With independent private values, the restriction to truthful equilibria is without loss, except for the punishment level; if players withhold their information during punishment-like phases, a “folk” theorem obtains also

A model predictive control approach to a class of multiplayer minmax differential games

In this dissertation, we consider a class of two-team adversarial differential games in which there are multiple mobile dynamic agents on each team. We describe such games in terms of semi-infinite minmax Model Predictive Control (MPC) problems, and present a numerical optimization technique for efficiently solving them. We also describe the implementation of the solution method in both indoor and outdoor robotic testbeds. Our solution method requires one to solve a sequence of Quadratic Programs (QPs), which together efficiently solve the original semi-infinite min- max MPC problem. The solution method separates the problem into two subproblems called the inner and outer subproblems, respectively. The inner subproblem is based on a constrained nonlinear numerical optimization technique called the Phase I-Phase II method, and we develop a customized version of this method. The outer subproblem is about judiciously initializing the inner subproblems to achieve overall convergence; our method guarantees exponential convergence. We focus on a specific semi-infinite minmax MPC problem called the harbor defense problem. First, we present foundational work on this problem in a formulation containing a single defender and single intruder. We next extend the basic formulation to various advanced scenarios that include cases in which there are multiple defenders and intruders, and also ones that include varying assumptions about intruder strategies. Another main contribution is that we implemented our solution method for the harbor defense problem on both real-time indoor and outdoor testbeds, and demonstrated its computational effectiveness. The indoor testbed is a custom-built robotic testbed named HoTDeC (Hovercraft Testbed for Decentralized Control). The outdoor testbed involved full-sized US Naval Academy patrol ships, and the experiment was conducted in Chesapeake Bay in collaboration with the US Naval Academy. The scenario used involved one ship (the intruder) being commanded by a human pilot, and the defender ship being controlled automatically by our semi-infinite minmax MPC algorithm.The results of several experiments are presented. Finally, we present an efficient algorithm for solving a class of matrix games, and show how this approach can be directly used to effectively solve our original continuous space semi-infinite minmax problem using an adaptive approximation

The cybernetics of nonzero sum games – the prisoner's dilemma reinterpreted as a pure conflict game with nature, with empirical applications

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.In this thesis a new solution concept is developed for n-player, nonzero sum games. The solution concept is based in reinterpreting the n-player nonzero sum game into 2-player zero sum games. The n-player nonzero sum game is first rewritten as an n + 1 player coalition game. The definition of zero sum payment is that one player pays the other what he gets in a given outcome (coalition of the n + 1 player game). Who pays whom depends on the coalition. More than one 2-player zero sum interpretation game always results from the procedure, and criteria are established to select one of the zero sum interpretation games. The solution concept defines results identical to the minimax concept when applied directly to zero sum 2-player games. When applied to 2-player prisoner’s dilemma games, the solution procedure assigns mixed strategies to the prisoners, thereby “resolving” the dilemma. The mixed strategies vary with the payoffs (up to a linear transformation). For prisoner’s dilemma matrices which have been used in large numbers of gaming experiments, the solution concept predicts dynamically, i.e., by play number, the “fraction of cooperative choices” for (approximately) the first 30 plays. In addition, the mixed strategy appears in a game between each subject (prisoner) and the n + 1st player (district attorney), suggesting that the subjects have been playing against the experimenter. Empirical evidence for this conclusion is given. A theorem is proved for n-player prisoner’s dilemma games. Game theory is reviewed to show the roots of this solution concept in the heuristic use of zero sum n-player games in the von Neumann and Morgenstern theory, and in rational decision making models, e.g., “games against Nature.” The empirical and formal difficulties of the equilibrium point solution concept for nonzero sum games are discussed. Detailed connections between game theory and cybernetics are discussed

On the Provision of Public Goods on Networks: Incentives, Exit Equilibrium, and Applications to Cyber .

Attempts to improve the state of cyber-security have been on the rise over the past years. The importance of incentivizing better security decisions by users in the current landscape is two-fold: it not only helps users protect themselves against attacks, but also provides positive externalities to others interacting with them, as a protected user is less likely to become compromised and be used to propagate attacks against other entities. Therefore, security can be viewed as a public good. This thesis takes a game-theoretic approach to understanding the theoretical underpinnings of users' incentives in the provision of public goods, and in particular, cyber-security. We analyze the strategic interactions of users in the provision of security as a non-excludable public good. We propose the notion of exit equilibrium to describe users' outside options from mechanisms for incentivizing the adoption of better security decisions, and use it to highlight the crucial effect of outside options on the design of incentive mechanisms for improving the state of cyber-security. We further focus on the general problem of public good provision games on networks. We identify necessary and sufficient conditions on the structure of the network for the existence and uniqueness of the Nash equilibrium in these games. We show that previous results in the literature can be recovered as special cases of our result. We provide a graph-theoretical interpretation of users' efforts at the Nash equilibria, Pareto efficient outcomes, and semi-cooperative equilibria of these games, by linking users' effort decisions to their centralities in the interaction network. Using this characterization, we separate the effects of users' dependencies and influences (outgoing and incoming edges, respectively) on their effort levels, and uncover an alternating effect over walks of different length in the network. We also propose the design of inter-temporal incentives in a particular type of security games, namely, security information sharing agreement. We show that either public or private assessments can be used in designing incentives for participants to disclose their information in these agreements. Finally, we present a method for crowdsourcing reputation that can be useful in attaining assessments of users' efforts in security games.PhDElectrical Engineering: SystemsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/133328/1/naghizad_1.pd

Frequent Monitoring in Repeated Games under Brownian Uncertainty

This paper studies frequent monitoring in a simple infinitely repeated game with imperfect public information and discounting, where players observe the state of a continuous time Brownian process at moments in time of length Δ. It shows that efficient strongly symmetric perfect public equilibrium payoffs can be achieved with imperfect public monitoring when players monitor each other at the highest frequency, i.e. Δ→0. The approach proposed places distinct initial conditions on the process, which depend on the unknown action profile simultaneously and privately decided by the players at the beginning of each period of the game. The strong decreasing effect on the expected immediate gains from deviation when the interval between actions shrinks, and the associated increase precision of the public signals, make the result possible in the limit. The existence of a positive monotonic relation between payoffs and monitoring intensity is also found.Repeated Games; Frequent Monitoring; Imperfect Public Monitoring; Brownian Motion; Moral Hazard

On-Farm Water Management Game With Heuristic Capabilities

A modern computer-based simulation tool (WaterMan) in the form of a game for on-farm water management was developed for application in training events for farmers, students, and irrigators. The WaterMan game utilizes an interactive framework, thereby allowing the user to develop scenarios and test alternatives in a convenient, risk-free environment. It includes a comprehensive soil water and salt balance calculation algorithm. It also employs heuristic capabilities for modeling all of the important aspects of on-farm water management, and to provide reasonable scores and advice to the trainees. Random events (both favorable and unfavorable) and different strategic decisions are included in the game for more realism and to provide an appropriate level of challenge according to player performance. Thus, the ability to anticipate the player skill level, and to reply with random events appropriate to the anticipated level, is provided by the heuristic capabilities used in the software. These heuristic features were developed based on a combination of two artificial intelligence approaches: (1) a pattern recognition approach; and (2) reinforcement learning based on a Markov Decision Processes approach, specifically, the Q-learning method. These two approaches were combined in a new way to account for the difference in the effect of actions taken by the player and action taken by the system on the game world. The reward function for the Q-learning method was modified to reflect the anticipated type of the WaterMan game as what is referred to as a partially competitive and partially cooperative game. Twenty-two different persons classified under three major categories (1) practicing farmers; (2) persons without an irrigation background; and (3) persons with an irrigation background, were observed while playing the game, and each of them filled out a questionnaire about the game. The technical module of the game was validated in two ways: through conducting mass balance calculations for soil water content and salt content over a period of simulation time, and through comparing the WaterMan technical module output data in calculating the irrigation requirements and the use of irrigation scheduling recommendations with those obtained from the same set of input data to the FAO CropWat 8 software. The testing results and the technical validation outcomes demonstrate the high performance of the WaterMan game as a heuristic training tool for on-farm water management