Learning for Cross-layer Resource Allocation in the Framework of Cognitive Wireless Networks

The framework of cognitive wireless networks is expected to endow wireless devices with a cognition-intelligence ability with which they can efficiently learn and respond to the dynamic wireless environment. In this dissertation, we focus on the problem of developing cognitive network control mechanisms without knowing in advance an accurate network model. We study a series of cross-layer resource allocation problems in cognitive wireless networks. Based on model-free learning, optimization and game theory, we propose a framework of self-organized, adaptive strategy learning for wireless devices to (implicitly) build the understanding of the network dynamics through trial-and-error. The work of this dissertation is divided into three parts. In the first part, we investigate a distributed, single-agent decision-making problem for real-time video streaming over a time-varying wireless channel between a single pair of transmitter and receiver. By modeling the joint source-channel resource allocation process for video streaming as a constrained Markov decision process, we propose a reinforcement learning scheme to search for the optimal transmission policy without the need to know in advance the details of network dynamics. In the second part of this work, we extend our study from the single-agent to a multi-agent decision-making scenario, and study the energy-efficient power allocation problems in a two-tier, underlay heterogeneous network and in a self-sustainable green network. For the heterogeneous network, we propose a stochastic learning algorithm based on repeated games to allow individual macro- or femto-users to find a Stackelberg equilibrium without flooding the network with local action information. For the self-sustainable green network, we propose a combinatorial auction mechanism that allows mobile stations to adaptively choose the optimal base station and sub-carrier group for transmission only from local payoff and transmission strategy information. In the third part of this work, we study a cross-layer routing problem in an interweaved Cognitive Radio Network (CRN), where an accurate network model is not available and the secondary users that are distributed within the CRN only have access to local action/utility information. In order to develop a spectrum-aware routing mechanism that is robust against potential insider attackers, we model the uncoordinated interaction between CRN nodes in the dynamic wireless environment as a stochastic game. Through decomposition of the stochastic routing game, we propose two stochastic learning algorithm based on a group of repeated stage games for the secondary users to learn the best-response strategies without the need of information flooding

Blockchain-based distributive auction for relay-assisted secure communications

Physical layer security (PLS) is considered as a promising technique to prevent information eavesdropping in wireless systems. In this context, cooperative relaying has emerged as a robust solution for achieving PLS due to multipath diversity and relatively lower transmission power. However, relays or the relay operators in the practical environment are unwilling for service provisioning unless they are incentivized for their cost of services. Thus, it is required to jointly consider network economics and relay cooperation to improve system efficiency. In this paper, we consider the problem of joint network economics and PLS using cooperative relaying and jamming. Based on the double auction theory, we model the interaction between transmitters seeking for a particular level of secure transmission of information and relay operators for suitable relay and jammer assignment, in a multiple source-destination networks. In addition, theoretical analyses are presented to justify that the proposed auction mechanism satisfies the desirable economic properties of individual rationality, budget balance, and truthfulness. As the participants in the traditional centralized auction framework may take selfish actions or collude with each other, we propose a decentralized and trustless auction framework based on blockchain technology. In particular, we exploit the smart contract feature of blockchain to construct a completely autonomous framework, where all the participants are financially enforced by smart contract terms. The security properties of the proposed framework are also discussed

Systems-compatible Incentives

Originally, the Internet was a technological playground, a collaborative endeavor among researchers who shared the common goal of achieving communication. Self-interest used not to be a concern, but the motivations of the Internet's participants have broadened. Today, the Internet consists of millions of commercial entities and nearly 2 billion users, who often have conflicting goals. For example, while Facebook gives users the illusion of access control, users do not have the ability to control how the personal data they upload is shared or sold by Facebook. Even in BitTorrent, where all users seemingly have the same motivation of downloading a file as quickly as possible, users can subvert the protocol to download more quickly without giving their fair share. These examples demonstrate that protocols that are merely technologically proficient are not enough. Successful networked systems must account for potentially competing interests. In this dissertation, I demonstrate how to build systems that give users incentives to follow the systems' protocols. To achieve incentive-compatible systems, I apply mechanisms from game theory and auction theory to protocol design. This approach has been considered in prior literature, but unfortunately has resulted in few real, deployed systems with incentives to cooperate. I identify the primary challenge in applying mechanism design and game theory to large-scale systems: the goals and assumptions of economic mechanisms often do not match those of networked systems. For example, while auction theory may assume a centralized clearing house, there is no analog in a decentralized system seeking to avoid single points of failure or centralized policies. Similarly, game theory often assumes that each player is able to observe everyone else's actions, or at the very least know how many other players there are, but maintaining perfect system-wide information is impossible in most systems. In other words, not all incentive mechanisms are systems-compatible. The main contribution of this dissertation is the design, implementation, and evaluation of various systems-compatible incentive mechanisms and their application to a wide range of deployable systems. These systems include BitTorrent, which is used to distribute a large file to a large number of downloaders, PeerWise, which leverages user cooperation to achieve lower latencies in Internet routing, and Hoodnets, a new system I present that allows users to share their cellular data access to obtain greater bandwidth on their mobile devices. Each of these systems represents a different point in the design space of systems-compatible incentives. Taken together, along with their implementations and evaluations, these systems demonstrate that systems-compatibility is crucial in achieving practical incentives in real systems. I present design principles outlining how to achieve systems-compatible incentives, which may serve an even broader range of systems than considered herein. I conclude this dissertation with what I consider to be the most important open problems in aligning the competing interests of the Internet's participants

Game Theory in Communications:a Study of Two Scenarios

Multi-user communication theory typically studies the fundamental limits of communication systems, and considers communication schemes that approach or even achieve these limits. The functioning of many such schemes assumes that users always cooperate, even when it is not in their own best interest. In practice, this assumption need not be fulfilled, as rational communication participants are often only interested in maximizing their own communication experience, and may behave in an undesirable manner from the system's point of view. Thus, communication systems may operate differently than intended if the behavior of individual participants is not taken into account. In this thesis, we study how users make decisions in wireless settings, by considering their preferences and how they interact with each other. We investigate whether the outcomes of their decisions are desirable, and, if not, what can be done to improve them. In particular, we focus on two related issues. The first is the decision-making of communication users in the absence of any central authority, which we consider in the context of the Gaussian multiple access channel. The second is the pricing of wireless resources, which we consider in the context of the competition of wireless service providers for users who are not contractually tied to any provider, but free to choose the one offering the best tradeoff of parameters. In the first part of the thesis, we model the interaction of self-interested users in a Gaussian multiple access channel using non-cooperative game theory. We demonstrate that the lack of infrastructure leads to an inefficient outcome for users who interact only once, specifically due to the lack of coordination between users. Using evolutionary game theory, we show that this inefficient outcome would also arise as a result of repeated interaction of many individuals over time. On the other hand, if the users correlate their decoding schedule with the outcome of some publicly observed (pseudo) random variable, the resulting outcome is efficient. This shows that sometimes it takes very little intervention on the part of the system planner to make sure that users choose a desirable operating point. In the second part of the thesis, we consider the competition of wireless service providers for users who are free to choose their service provider based on their channel parameters and the resource price. We model this situation as a two-stage game where the providers announce unit resource prices in the first stage and the users choose how much resource they want to purchase from each provider in the second stage. Under fairly general conditions, we show that the competitive interaction of users and providers results in socially optimal resource allocation. We also provide a decentralized primal-dual algorithm and prove its convergence to the socially optimal outcome

Trustnet: a Trust and Reputation Management System in Distributed Environments

With emerging Internet-scale open content and resource sharing, social networks, and complex cyber-physical systems, trust issues become prominent. Despite their rigorous foundations, conventional network security theories and mechanisms are inadequate at addressing such loosely-defined security issues in decentralized open environments.In this dissertation, we propose a trust and reputation management system architecture and protocols (TrustNet), aimed to define and promote trust as a first-class system parameter on par with communication, computation, and storage performance metrics. To achieve such a breakthrough, we need a fundamentally new design paradigm to seamlessly integrate trust into system design. Our TrustNet initiative represents a bold effort to approach this ultimate goal. TrustNet is built on the top of underlying P2P and mobile ad hoc network layer and provides trust services to higher level applications and middleware. Following the TrustNet architecture, we design, implement, and analyze trust rating, trust aggregation, and trust management strategies. Especially, we propose three trust dissemination protocols and algorithms to meet the urgent needs and explicitly define and formulate end-to-end trust. We formulate trust management problems and propose the H-Trust, VectorTrust, and cTrust scheme to handle trust establishment and aggregation issues. We model trust relations as a trust graph in distributed environment to enhance accuracy and efficiency of trust establishment among peers. Leveraging the distributed Bellman-Ford algorithm, stochastic Markov chain process and H-Index algorithm for fast and lightweight aggregation of trust scores, our scheme are decentralized and self-configurable trust aggregation schemes.To evaluate TrustNet management strategies, we simulated our proposed protocols in both unstructured P2P network and mobile ad hoc network to analyze and simulate trust relationships. We use software generated data as well as real world data sets. Particularly, the student contact patterns on the NUS campus is used as our trust communication model. The simulation results demonstrate the features of trust relationship dissemination in real environments and the efficiency, accuracy, scalability and robustness of the TrustNet system.Computer Science Departmen

Automated Algorithmic Machine-to-Machine Negotiation for Lane Changes Performed by Driverless Vehicles at the Edge of the Internet of Things

This dissertation creates and examines algorithmic models for automated machine-to-machine negotiation in localized multi-agent systems at the edge of the Internet of Things. It provides an implementation of two such models for unsupervised resource allocation for the application domain of autonomous vehicle traffic as it pertains to lane changing and speed setting selection. The first part concerns negotiation via abstract argumentation. A general model for the arbitration of conflict based on abstract argumentation is outlined and then applied to a scenario where autonomous vehicles on a multi-lane highway use expert systems in consultation with private objectives to form arguments and use them to compete for lane positions. The conflict resolution component of the resulting argumentation framework is augmented with social voting to achieve a community supported conflict-free outcome. The presented model heralds a step toward independent negotiation through automated argumentation in distributed multi-agent systems. Many other cyber-physical environments embody stages for opposing positions that may benefit from this type of tool for collaboration. The second part deals with game-theoretic negotiation through mechanism design. It outlines a mechanism providing resource allocation for a fee and applies it to autonomous vehicle traffic. Vehicular agents apply for speed and lane assignments with sealed bids containing their private feasible action valuations determined within the context of their governing objective. A truth-inducing mechanism implementing an incentive-compatible strategyproof social choice functions achieves a socially optimal outcome. The model can be adapted to many application fields through the definition of a domain-appropriate operation to be used by the allocation function of the mechanism. Both presented prototypes conduct operations at the edge of the Internet of Things. They can be applied to agent networks in just about any domain where the sharing of resources is required. The social voting argumentation approach is a minimal but powerful tool facilitating the democratic process when a community makes decisions on the sharing or rationing of common-pool assets. The mechanism design model can create social welfare maximizing allocations for multiple or multidimensional resources