68,967 research outputs found
Adaptive reinforcement learning for heterogeneous network selection
Next generation 5G mobile wireless networks will consist of multiple technologies for devices
to access the network at the edge. One of the keys to 5G is therefore the ability for
device to intelligently select its Radio Access Technology (RAT). Current fully distributed
algorithms for RAT selection although guaranteeing convergence to equilibrium states,
are often slow, require high exploration times and may converge to undesirable equilibria.
In this dissertation, we propose three novel reinforcement learning (RL) frameworks
to improve the efficiency of existing distributed RAT selection algorithms in a heterogeneous
environment, where users may potentially apply a number of different RAT selection
procedures. Although our research focuses on solutions for RAT selection in the
current and future mobile wireless networks, the proposed solutions in this dissertation
are general and suitable to apply for any large scale distributed multi-agent systems.
In the first framework, called RL with Non-positive Regret, we propose a novel adaptive
RL for multi-agent non-cooperative repeated games. The main contribution is to use both
positive and negative regrets in RL to improve the convergence speed and fairness of
the well-known regret-based RL procedure. Significant improvements in performance
compared to other related algorithms in the literature are demonstrated.
In the second framework, called RL with Network-Assisted Feedback (RLNF), our core
contribution is to develop a network feedback model that uses network-assisted information
to improve the performance of the distributed RL for RAT selection. RLNF guarantees
no-regret payoff in the long-run for any user adopting it, regardless of what other users
might do and so can work in an environment where not all users use the same learning
strategy. This is an important implementation advantage as RLNF can be implemented
within current mobile network standards.
In the third framework, we propose a novel adaptive RL-based mechanism for RAT selection
that can effectively handle user mobility. The key contribution is to leverage forgetting
methods to rapidly react to the changes in the radio conditions when users move.
We show that our solution improves the performance of wireless networks and converges
much faster when users move compared to the non-adaptive solutions. Another objective of the research is to study the impact of various network models on the
performance of different RAT selection approaches. We propose a unified benchmark to
compare the performances of different algorithms under the same computational environment.
The comparative studies reveal that among all the important network parameters
that influence the performance of RAT selection algorithms, the number of base stations
that a user can connect to has the most significant impact. This finding provides some
guidelines for the proper design of RAT selection algorithms for future 5G. Our evaluation
benchmark can serve as a reference for researchers, network developers, and engineers.
Overall, the thesis provides different reinforcement learning frameworks to improve the
efficiency of current fully distributed algorithms for heterogeneous RAT selection. We
prove the convergence of the proposed reinforcement learning procedures using the differential
inclusion (DI) technique. The theoretical analyses demonstrate that the use of
DI not only provides an effective method to study the convergence properties of adaptive
procedures in game-theoretic learning, but also yields a much more concise and extensible
proof as compared to the classical approaches.Thesis (Ph.D.) -- University of Adelaide, School of Electrical and Electronic Engineering, 201
Automated construction of a hierarchy of self-organized neural network classifiers
This paper documents an effort to design and implement a neural network-based, automatic classification system which dynamically constructs and trains a decision tree. The system is a combination of neural network and decision tree technology. The decision tree is constructed to partition a large classification problem into smaller problems. The neural network modules then solve these smaller problems. We used a variant of the Fuzzy ARTMAP neural network which can be trained much more quickly than traditional neural networks. The research extends the concept of self-organization from within the neural network to the overall structure of the dynamically constructed decision hierarchy. The primary advantage is avoidance of manual tedium and subjective bias in constructing decision hierarchies. Additionally, removing the need for manual construction of the hierarchy opens up a large class of potential classification applications. When tested on data from real-world images, the automatically generated hierarchies performed slightly better than an intuitive (handbuilt) hierarchy. Because the neural networks at the nodes of the decision hierarchy are solving smaller problems, generalization performance can really be improved if the number of features used to solve these problems is reduced. Algorithms for automatically selecting which features to use for each individual classification module were also implemented. We were able to achieve the same level of performance as in previous manual efforts, but in an efficient, automatic manner. The technology developed has great potential in a number of commercial areas, including data mining, pattern recognition, and intelligent interfaces for personal computer applications. Sample applications include: fraud detection, bankruptcy prediction, data mining agent, scalable object recognition system, email agent, resource librarian agent, and a decision aid agent
- …