Learning-based Decision Making in Wireless Communications

Fueled by emerging applications and exponential increase in data traffic, wireless networks have recently grown significantly and become more complex. In such large-scale complex wireless networks, it is challenging and, oftentimes, infeasible for conventional optimization methods to quickly solve critical decision-making problems. With this motivation, in this thesis, machine learning methods are developed and utilized for obtaining optimal/near-optimal solutions for timely decision making in wireless networks. Content caching at the edge nodes is a promising technique to reduce the data traffic in next-generation wireless networks. In this context, we in the first part of the thesis study content caching at the wireless network edge using a deep reinforcement learning framework with Wolpertinger architecture. Initially, we develop a learning-based caching policy for a single base station aiming at maximizing the long-term cache hit rate. Then, we extend this study to a wireless communication network with multiple edge nodes. In particular, we propose deep actor-critic reinforcement learning based policies for both centralized and decentralized content caching. Next, with the purpose of making efficient use of limited spectral resources, we develop a deep actor-critic reinforcement learning based framework for dynamic multichannel access. We consider both a single-user case and a scenario in which multiple users attempt to access channels simultaneously. In the single-user model, in order to evaluate the performance of the proposed channel access policy and the framework\u27s tolerance against uncertainty, we explore different channel switching patterns and different switching probabilities. In the case of multiple users, we analyze the probabilities of each user accessing channels with favorable channel conditions and the probability of collision. Following the analysis of the proposed learning-based dynamic multichannel access policy, we consider adversarial attacks on it. In particular, we propose two adversarial policies, one based on feed-forward neural networks and the other based on deep reinforcement learning policies. Both attack strategies aim at minimizing the accuracy of a deep reinforcement learning based dynamic channel access agent, and we demonstrate and compare their performances. Next, anomaly detection as an active hypothesis test problem is studied. Specifically, we study deep reinforcement learning based active sequential testing for anomaly detection. We assume that there is an unknown number of abnormal processes at a time and the agent can only check with one sensor in each sampling step. To maximize the confidence level of the decision and minimize the stopping time concurrently, we propose a deep actor-critic reinforcement learning framework that can dynamically select the sensor based on the posterior probabilities. Separately, we also regard the detection of threshold crossing as an anomaly detection problem, and analyze it via hierarchical generative adversarial networks (GANs). In the final part of the thesis, to address state estimation and detection problems in the presence of noisy sensor observations and probing costs, we develop a soft actor-critic deep reinforcement learning framework. Moreover, considering Byzantine attacks, we design a GAN-based framework to identify the Byzantine sensors. To evaluate the proposed framework, we measure the performance in terms of detection accuracy, stopping time, and the total probing cost needed for detection

Belief Refinement Approaches to Communication and Inference Problems

This dissertation considers a problem where a single agent or a group of agents aim to estimate/learn unknown (possibly time-varying) parameters of interest despite making noisy observations. The agents take a Bayesian-like approach by maintaining a posterior probability distribution or “belief" over a parameter space conditioned on past observations. The agents aim to iteratively refine their belief over the parameter space as new information is acquired from their private observations or through collaboration with other agents. In particular, the agents aim to ensure that sufficient belief is assigned in neighborhoods centered around the true parameter with high probability or “reliability". In the context of communication problems considered in this dissertation, the agents may be active, i.e., agents may additionally take actions which provide new observations. Furthermore, agents may employ an adaptive strategy, i.e., using their past actions and the resulting observations, agents can adaptively choose actions to control the concentration of the belief. When the agents are active, we propose and analyze adaptive belief refinement approaches to obtain belief concentration on the unknown parameter with high reliability. In a different context, namely that of decentralized inference, we consider passive agents. Here, agents face an additional challenge due to the statistical insufficiency of their private observations to learn the unknown parameter. While individual agents’ observations are not informative enough, we assume that the agents’ observations are collectively informative to learn the unknown parameter. Here, we propose and analyze decentralized belief refining strategies to collaboratively obtain belief concentration on the unknown parameter. In the first part of this dissertation, we consider active strategies that are extensions of the posterior matching strategy (PM) introduced by Horstein, which is a generalization of the well-known binary search algorithm. We propose and analyze PM based strategies in the context of modern communication systems, namely the problem of establishing initial access in mm-Wave communication and spectrum sensing for Cognitive Radio. We propose and analyze channel coding strategies for real-time streaming and control applications. The second part of the dissertation investigates the belief refinement approaches for decentralized learning. In particular, it focusing on developing and analyzing a decentralized learning rule for statistical hypothesis testing and its application to decentralized machine learning

Airborne Directional Networking: Topology Control Protocol Design

This research identifies and evaluates the impact of several architectural design choices in relation to airborne networking in contested environments related to autonomous topology control. Using simulation, we evaluate topology reconfiguration effectiveness using classical performance metrics for different point-to-point communication architectures. Our attention is focused on the design choices which have the greatest impact on reliability, scalability, and performance. In this work, we discuss the impact of several practical considerations of airborne networking in contested environments related to autonomous topology control modeling. Using simulation, we derive multiple classical performance metrics to evaluate topology reconfiguration effectiveness for different point-to-point communication architecture attributes for the purpose of qualifying protocol design elements

Decentralization in messaging applications with support for contactless interaction

Peer-to-peer communication has increasingly been gaining prevalence in people’s daily lives, with its widespread adoption being catalysed by technological advances. Although there have been strides for the inclusion of disabled individuals to ease communication between peers, people who suffer arm/hand impairments have little to no support in regular mainstream applications to efficiently communicate with other individuals. Additionally, as centralized systems have come into scrutiny regarding privacy and security, the development of alternative, decentralized solutions have increased, a movement pioneered by Bitcoin that culminated in the blockchain technology and its variants. Aiming towards expanding inclusivity in the messaging applications panorama, this project showcases an alternative on contactless human-computer interaction with support for disabled individuals with focus on the decentralized backend counterpart. Users of the application partake in a decentralized network based on a distributed hash table that is designed for secure communication (granted by a custom cryptographic messaging protocol) and exchange of data between peers. Such system is both resilient to tampering attacks and central points of failure (akin to blockchains), as well as having no long-term restrictions regarding scalability prospects, something that is a recurring issue in blockchain-based platforms. The conducted experiments showcase a level of performance similar to mainstream centralized approaches, outperforming blockchain-based decentralized applications on the delay between sending and receiving messages.A comunicação ponto-a-ponto tem cada vez mais ganhado prevalência na vida contemporânea de pessoas, tendo a sua adoção sido catalisada pelos avanços tecnológicos. Embora tenham havido desenvolvimentos relativamente à inclusão de indivíduos com deficiência para facilitar a comunicação entre pessoas, as que sofrem imparidades no braço/mão têm um suporte escasso em aplicações convencionais para comunicar de forma eficiente com outros sujeitos. Adicionalmente, à medida que sistemas centralizados têm atraído ceticismo relativamente à sua privacidade e segurança, o desenvolvimento de soluções descentralizadas e alternativas têm aumentado, um movimento iniciado pela Bitcoin que culminou na tecnologia de blockchain e as suas variantes. Tendo como objectivo expandir a inclusão no panorama de aplicações de messaging, este projeto pretende demonstrar uma alternativa na interação humano-computador sem contacto direto físico e com suporte para indivíduos com deficiência, com foco no componente backend decentralizado. Utilizadores da aplicação são inseridos num sistema decentralizado baseado numa hash table distribuída que foi desenhado para comunicação segura (providenciado por um protocolo de messaging criptográfico customizado) e para troca de dados entre utilizadores. Tal sistema é tanto resiliente a ataques de adulteração de dados como também a pontos centrais de falha (presente em blockains), não tendo adicionalmente restrições ao nível de escabilidade a longo-prazo, algo que é um problem recorrente em plataformas baseadas em blockchain. As avaliações e experiências realizadas neste projeto demonstram um nível de performance semelhante a abordagens centralizadas convencionais, tendo uma melhor prestação que aplicações descentralizadas baseadas em blockchain no que toca à diferença no tempo entre enviar e receber mensagens