Towards Thompson Sampling for Complex Bayesian Reasoning

Paper III, IV, and VI are not available as a part of the dissertation due to the copyright.Thompson Sampling (TS) is a state-of-art algorithm for bandit problems set in a Bayesian framework. Both the theoretical foundation and the empirical efficiency of TS is wellexplored for plain bandit problems. However, the Bayesian underpinning of TS means that TS could potentially be applied to other, more complex, problems as well, beyond the bandit problem, if suitable Bayesian structures can be found. The objective of this thesis is the development and analysis of TS-based schemes for more complex optimization problems, founded on Bayesian reasoning. We address several complex optimization problems where the previous state-of-art relies on a relatively myopic perspective on the problem. These includes stochastic searching on the line, the Goore game, the knapsack problem, travel time estimation, and equipartitioning. Instead of employing Bayesian reasoning to obtain a solution, they rely on carefully engineered rules. In all brevity, we recast each of these optimization problems in a Bayesian framework, introducing dedicated TS based solution schemes. For all of the addressed problems, the results show that besides being more effective, the TS based approaches we introduce are also capable of solving more adverse versions of the problems, such as dealing with stochastic liars.publishedVersio

User grouping and power allocation in NOMA systems: a novel semi-supervised reinforcement learning-based solution

Author's accepted manuscriptIn this paper, we present a pioneering solution to the problem of user grouping and power allocation in non-orthogonal multiple access (NOMA) systems. The problem is highly pertinent because NOMA is a well-recognized technique for future mobile radio systems. The salient and difcult issues associated with NOMA systems involve the task of grouping users together into the prespecifed time slots, which are augmented with the question of determining how much power should be allocated to the respective users. This problem is, in and of itself, NP-hard. Our solution is the frst reported reinforcement learning (RL)-based solution, which attempts to resolve parts of this issue. In particular, we invoke the object migration automaton (OMA) and one of its variants to resolve the grouping in NOMA systems. Furthermore, unlike the solutions reported in the literature, we do not assume prior knowledge of the channels’ distributions, nor of their coefcients, to achieve the grouping/partitioning. Thereafter, we use the consequent groupings to heuristically infer the power allocation. The simulation results that we have obtained confrm that our learning scheme can follow the dynamics of the channel coefcients efciently, and that the solution is able to resolve the issue dynamicallyacceptedVersio

Sequential Design for Optimal Stopping Problems

We propose a new approach to solve optimal stopping problems via simulation. Working within the backward dynamic programming/Snell envelope framework, we augment the methodology of Longstaff-Schwartz that focuses on approximating the stopping strategy. Namely, we introduce adaptive generation of the stochastic grids anchoring the simulated sample paths of the underlying state process. This allows for active learning of the classifiers partitioning the state space into the continuation and stopping regions. To this end, we examine sequential design schemes that adaptively place new design points close to the stopping boundaries. We then discuss dynamic regression algorithms that can implement such recursive estimation and local refinement of the classifiers. The new algorithm is illustrated with a variety of numerical experiments, showing that an order of magnitude savings in terms of design size can be achieved. We also compare with existing benchmarks in the context of pricing multi-dimensional Bermudan options.Comment: 24 page

User Grouping and Power Allocation in NOMA Systems : A Reinforcement Learning-Based Solution

Author's accepted manuscript.Available from 05/09/2021.acceptedVersio

Learning Automata-Based Object Partitioning with Pre-Specified Cardinalities

Master's thesis in Information- and communication technology (IKT591)The Object Migrating Automata (OMA) has been used as a powerful AI-based tool to resolve real-life partitioning problems. Apart from its original version, variants and enhancements that invoke the pursuit concept of Learning Automata, and the phenomena of transitivity, have more recently been used to improve its power. The single major handicap that it possesses is the fact that the number of the objects in each partition must be equal. This thesis deals with the task of relaxing this constraint. Thus, in this thesis, we will consider the problem of designing OMA-based schemes when the number of the objects can be unequal, but prespecified. By opening ourselves to this less-constrained version, we encounter a few problems that deal with the implementation of the inter-partition migration of the objects. This thesis considers how these problems can be solved, and in essence, presents the design, implementation and testing of two OMA-based methods and all its variants, that include the pursuit and transitivity phenomena

Review on Radio Resource Allocation Optimization in LTE/LTE-Advanced using Game Theory

Recently, there has been a growing trend toward ap-plying game theory (GT) to various engineering fields in order to solve optimization problems with different competing entities/con-tributors/players. Researches in the fourth generation (4G) wireless network field also exploited this advanced theory to overcome long term evolution (LTE) challenges such as resource allocation, which is one of the most important research topics. In fact, an efficient de-sign of resource allocation schemes is the key to higher performance. However, the standard does not specify the optimization approach to execute the radio resource management and therefore it was left open for studies. This paper presents a survey of the existing game theory based solution for 4G-LTE radio resource allocation problem and its optimization

Intelligent Learning Automata-based Strategies Applied to Personalized Service Provisioning in Pervasive Environments

Doktorgradsavhandling i informasjons- og kommunikasjonsteknologi, Universitetet i Agder, Grimstad, 201

Rank-aware, Approximate Query Processing on the Semantic Web

Search over the Semantic Web corpus frequently leads to queries having large result sets. So, in order to discover relevant data elements, users must rely on ranking techniques to sort results according to their relevance. At the same time, applications oftentimes deal with information needs, which do not require complete and exact results. In this thesis, we face the problem of how to process queries over Web data in an approximate and rank-aware fashion