A game theoretic approach to UAV routing and information collection

In recent times, the use of Unmanned aerial vehicles (UAVs) for tasks which involve high endurance or perilous environments, has become increasingly vital. A typical problem is that of information collection, in particular when multiple UAVs are involved, which prompts an important problem of routing these UAVs through the search environment with the goal of maximizing the collected information. Most of the previous line of work assumes a centralized control and full communication among the UAVs, thus posing this as an optimization problem solved via centralized solutions. However, in applications where communication is infeasible, each UAV must individually solve the problem. Assuming a natural scenario of UAVs being compensated for the collected information makes them self-interested agents trying to maximize their payoffs. Consequently, our game-theoretic approach is a natural fit. While our game model is primarily based on the game model used in a previous work, it is also significantly generalized, incorporating interesting facets of information fusion and multi-modality-composed information. This game is closely related to the well-studied classes of congestion-type and resource selection games, but cannot be cast into these classes unless certain critical constraints are relaxed. Our contribution to this literature, is a result on existence of pure Nash equilibria via existence of the Finite Improvement Property, which applies to any singleton congestion-type games having a certain class of payoff functions. Finally, to our best knowledge, our results providing theoretically guaranteed tight bounds on the Price of anarchy and Price of stability, are the first such results in the literature involving a game theoretic approach to UAV routing

Resource allocation in networks via coalitional games

The main goal of this dissertation is to manage resource allocation in network engineering problems and to introduce efficient cooperative algorithms to obtain high performance, ensuring fairness and stability. Specifically, this dissertation introduces new approaches for resource allocation in Orthogonal Frequency Division Multiple Access (OFDMA) wireless networks and in smart power grids by casting the problems to the coalitional game framework and by providing a constructive iterative algorithm based on dynamic learning theory.  Software Engineering (Software)Algorithms and the Foundations of Software technolog

Routing choices in intelligent transport systems

Road congestion is a phenomenon that can often be avoided; roads become popular, travel times increase, which could be mitigated with better coordination mechanisms. The choice of route, mode of transport, and departure time all play a crucial part in controlling congestion levels. Technology, such as navigation applications, have the ability to influence these decisions and play an essential role in congestion reduction. To predict vehicles' routing behaviours, we model the system as a game with rational players. Players choose a path between origin and destination nodes in a network. Each player seeks to minimise their own journey time, often leading to inefficient equilibria with poor social welfare. Traffic congestion motivates the results in this thesis. However, the results also hold true for many other applications where congestion occurs, e.g. power grid demand. Coordinating route selection to reduce congestion constitutes a social dilemma for vehicles. In sequential social dilemmas, players' strategies need to balance their vulnerability to exploitation from their opponents and to learn to cooperate to achieve maximal payouts. We address this trade-off between mathematical safety and cooperation of strategies in social dilemmas to motivate our proposed algorithm, a safe method of achieving cooperation in social dilemmas, including route choice games. Many vehicles use navigation applications to help plan their journeys, but these provide only partial information about the routes available to them. We find a class of networks for which route information distribution cannot harm the receiver's expected travel times. Additionally, we consider a game where players always follow the route chosen by an application or where vehicle route selection is controlled by a route planner, such as autonomous vehicles. We show that having multiple route planners controlling vehicle routing leads to inefficient equilibria. We calculate the Price of Anarchy (PoA) for polynomial function travel times and show that multiagent reinforcement learning algorithms suffer from the predicted Price of Anarchy when controlling vehicle routing. Finally, we equip congestion games with waiting times at junctions to model the properties of traffic lights at intersections. Here, we show that Braess' paradox can be avoided by implementing traffic light cycles and establish the PoA for realistic waiting times. By employing intelligent traffic lights that use myopic learning, such as multi-agent reinforcement learning, we prove a natural reward function guarantees convergence to equilibrium. Moreover, we highlight the impact of multi-agent reinforcement learning traffic lights on the fairness of journey times to vehicles

Improving Energy Efficiency and Security for Pervasive Computing Systems

Pervasive computing systems are comprised of various personal mobile devices connected by the wireless networks. Pervasive computing systems have gained soaring popularity because of the rapid proliferation of the personal mobile devices. The number of personal mobile devices increased steeply over years and will surpass world population by 2016.;However, the fast development of pervasive computing systems is facing two critical issues, energy efficiency and security assurance. Power consumption of personal mobile devices keeps increasing while the battery capacity has been hardly improved over years. at the same time, a lot of private information is stored on and transmitted from personal mobile devices, which are operating in very risky environment. as such, these devices became favorite targets of malicious attacks. Without proper solutions to address these two challenging problems, concerns will keep rising and slow down the advancement of pervasive computing systems.;We select smartphones as the representative devices in our energy study because they are popular in pervasive computing systems and their energy problem concerns users the most in comparison with other devices. We start with the analysis of the power usage pattern of internal system activities, and then identify energy bugs for improving energy efficiency. We also investigate into the external communication methods employed on smartphones, such as cellular networks and wireless LANs, to reduce energy overhead on transmissions.;As to security, we focus on implantable medical devices (IMDs) that are specialized for medical purposes. Malicious attacks on IMDs may lead to serious damages both in the cyber and physical worlds. Unlike smartphones, simply borrowing existing security solutions does not work on IMDs because of their limited resources and high requirement of accessibility. Thus, we introduce an external device to serve as the security proxy for IMDs and ensure that IMDs remain accessible to save patients\u27 lives in certain emergency situations when security credentials are not available

Applications of Game Theory to Multi-Agent Coordination Problems in Communication Networks

Recent years there has been a growing interest in the study of distributed control mechanisms for use in communication networks. A fundamental assumption in these models is that the participants in the network are willing to cooperate with the system. However, there are many instances where the incentives to cooperate is missing. Then, the agents may seek to achieve their own private interests by behaving strategically. Often, such selfish choices lead to inefficient equilibrium state of the system, commonly known as the tragedy of commons in Economics terminology. Now, one may ask the following question: how can the system be led to the socially optimal state in spite of selfish behaviors of its participants? The traditional control design framework fails to provide an answer as it does not take into account of selfish and strategic behavior of the agents. The use of game theoretical methods to achieve coordination in such network systems is appealing, as it naturally captures the idea of rational agents taking locally optimal decisions. In this thesis, we explore several instances of coordination problems in communication networks that can be analyzed using game theoretical methods. We study one coordination problem each, from each layer of TCP/IP reference model - the network model used in the current Internet architecture. First, we consider societal agents taking decisions on whether to obtain content legally or illegally, and tie their behavior to questions of performance of content distribution networks. We show that revenue sharing with peers promote performance and revenue extraction from content distribution networks. Next, we consider a transport layer problem where applications compete against each other to meet their performance objectives by selfishly picking congestion controllers. We establish that tolling schemes that incentivize applications to choose one of several different virtual networks catering to particular needs yields higher system value. Hence, we propose the adoption of such virtual networks. We address a network layer question in third problem. How do the sources in a wireless network split their traffic over the available set of paths to attain the lowest possible number of transmissions per unit time? We develop a two level distributed controller that attains the optimal traffic split. Finally, we study mobile applications competing for channel access in a cellular network. We show that the mechanism where base station conducting sequence of second price auctions and providing channel access to the winner achieves the benefits of the state of art solution, Largest Queue First policy

Automated Markets and Trading Agents

Computer automation has the potential, just starting to be realized, of transforming the design and operation of markets, and the behaviors of agents trading in them. We discuss the possibilities for automating markets, presenting a broad conceptual framework covering resource allocation as well as enabling marketplace services such as search and transaction execution. One of the most intriguing opportunities is provided by markets implementing computationally sophisticated negotiation mechanisms, for example combinatorial auctions. An important theme that emerges from the literature is the centrality of design decisions about matching the domain of goods over which a mechanism operates to the domain over which agents have preferences. When the match is imperfect (as is almost inevitable), the market game induced by the mechanism is analytically intractable, and the literature provides an incomplete characterization of rational bidding policies. A review of the literature suggests that much of our existing knowledge comes from computational simulations, including controlled studies of abstract market designs (e.g., simultaneous ascending auctions), and research tournaments comparing agent strategies in a variety of market scenarios. An empirical game-theoretic methodology combines the advantages of simulation, agent-based modeling, and statistical and game-theoretic analysis.http://deepblue.lib.umich.edu/bitstream/2027.42/49510/1/ace_galleys.pd

Optimisation des Systèmes Partiellement Observables dans les Réseaux Sans-fil (Théorie des jeux, Auto-adaptation et Apprentissage)

La dernière décennie a vu l'émergence d'Internet et l'apparition des applications multimédia qui requièrent de plus en plus de bande passante, ainsi que des utilisateurs qui exigent une meilleure qualité de service. Dans cette perspective, beaucoup de travaux ont été effectués pour améliorer l'utilisation du spectre sans fil.Le sujet de ma thèse de doctorat porte sur l'application de la théorie des jeux, la théorie des files d'attente et l'apprentissage dans les réseaux sans fil,en particulier dans des environnements partiellement observables. Nous considérons différentes couches du modèle OSI. En effet, nous étudions l'accès opportuniste au spectre sans fil à la couche MAC en utilisant la technologie des radios cognitifs (CR). Par la suite, nous nous concentrons sur le contrôle de congestion à la couche transport, et nous développons des mécanismes de contrôle de congestion pour le protocole TCP.Since delay-sensitive and bandwidth-intense multimedia applications have emerged in the Internet, the demand for network resources has seen a steady increase during the last decade. Specifically, wireless networks have become pervasive and highly populated.These motivations are behind the problems considered in this dissertation.The topic of my PhD is about the application of game theory, queueing theory and learning techniques in wireless networks under some QoS constraints, especially in partially observable environments.We consider different layers of the protocol stack. In fact, we study the Opportunistic Spectrum Access (OSA) at the Medium Access Control (MAC) layer through Cognitive Radio (CR) approaches.Thereafter, we focus on the congestion control at the transport layer, and we develop some congestion control mechanisms under the TCP protocol.The roadmap of the research is as follows. Firstly, we focus on the MAC layer, and we seek for optimal OSA strategies in CR networks. We consider that Secondary Users (SUs) take advantage of opportunities in licensed channels while ensuring a minimum level of QoS. In fact, SUs have the possibility to sense and access licensed channels, or to transmit their packets using a dedicated access (like 3G). Therefore, a SU has two conflicting goals: seeking for opportunities in licensed channels, but spending energy for sensing those channels, or transmitting over the dedicated channel without sensing, but with higher transmission delay. We model the slotted and the non-slotted systems using a queueing framework. Thereafter, we analyze the non-cooperative behavior of SUs, and we prove the existence of a Nash equilibrium (NE) strategy. Moreover, we measure the gap of performance between the centralized and the decentralized systems using the Price of Anarchy (PoA).Even if the OSA at the MAC layer was deeply investigated in the last decade, the performance of SUs, such as energy consumption or Quality of Service (QoS) guarantee, was somehow ignored. Therefore, we study the OSA taking into account energy consumption and delay. We consider, first, one SU that access opportunistically licensed channels, or transmit its packets through a dedicated channel. Due to the partial spectrum sensing, the state of the spectrum is partially observable. Therefore, we use the Partially Observable Markov Decision Process (POMDP) framework to design an optimal OSA policy for SUs. Specifically, we derive some structural properties of the value function, and we prove that the optimal OSA policy has a threshold structure.Thereafter, we extend the model to the context of multiple SUs. We study the non-cooperative behavior of SUs and we prove the existence of a NE. Moreover, we highlight a paradox in this situation: more opportunities in the licensed spectrum may lead to worst performances for SUs. Thereafter, we focus on the study of spectrum management issues. In fact, we introduce a spectrum manager to the model, and we analyze the hierarchical game between the network manager and SUs.Finally, we focus on the transport layer and we study the congestion control for wireless networks under some QoS and Quality of Experience (QoE) constraints. Firstly, we propose a congestion control algorithm that takes into account applications' parameters and multimedia quality. In fact, we consider that network users maximize their expected multimedia quality by choosing the congestion control strategy. Since users ignore the congestion status at bottleneck links, we use a POMDP framework to determine the optimal congestion control strategy.Thereafter, we consider a subjective measure of the multimedia quality, and we propose a QoE-based congestion control algorithm. This algorithm bases on QoE feedbacks from receivers in order to adapt the congestion window size. Note that the proposed algorithms are designed based on some learning methods in order to face the complexity of solving POMDP problems.AVIGNON-Bib. numérique (840079901) / SudocSudocFranceF