A survey of machine learning techniques applied to self organizing cellular networks

In this paper, a survey of the literature of the past fifteen years involving Machine Learning (ML) algorithms applied to self organizing cellular networks is performed. In order for future networks to overcome the current limitations and address the issues of current cellular systems, it is clear that more intelligence needs to be deployed, so that a fully autonomous and flexible network can be enabled. This paper focuses on the learning perspective of Self Organizing Networks (SON) solutions and provides, not only an overview of the most common ML techniques encountered in cellular networks, but also manages to classify each paper in terms of its learning solution, while also giving some examples. The authors also classify each paper in terms of its self-organizing use-case and discuss how each proposed solution performed. In addition, a comparison between the most commonly found ML algorithms in terms of certain SON metrics is performed and general guidelines on when to choose each ML algorithm for each SON function are proposed. Lastly, this work also provides future research directions and new paradigms that the use of more robust and intelligent algorithms, together with data gathered by operators, can bring to the cellular networks domain and fully enable the concept of SON in the near future

Mathematical optimization techniques for demand management in smart grids

The electricity supply industry has been facing significant challenges in terms of meeting the projected demand for energy, environmental issues, security, reliability and integration of renewable energy. Currently, most of the power grids are based on many decades old vertical hierarchical infrastructures where the electric power flows in one direction from the power generators to the consumer side and the grid monitoring information is handled only at the operation side. It is generally believed that a fundamental evolution in electric power generation and supply system is required to make the grids more reliable, secure and efficient. This is generally recognised as the development of smart grids. Demand management is the key to the operational efficiency and reliability of smart grids. Facilitated by the two-way information flow and various optimization mechanisms, operators benefit from real time dynamic load monitoring and control while consumers benefit from optimised use of energy. In this thesis, various mathematical optimization techniques and game theoretic frameworks have been proposed for demand management in order to achieve efficient home energy consumption scheduling and optimal electric vehicle (EV) charging. A consumption scheduling technique is proposed to minimise the peak consumption load. The proposed technique is able to schedule the optimal operation time for appliances according to the power consumption patterns of the individual appliances. A game theoretic consumption optimization framework is proposed to manage the scheduling of appliances of multiple residential consumers in a decentralised manner, with the aim of achieving minimum cost of energy for consumers. The optimization incorporates integration of locally generated and stored renewable energy in order to minimise dependency on conventional energy. In addition to the appliance scheduling, a mean field game theoretic optimization framework is proposed for electric vehicles to manage their charging. In particular, the optimization considers a charging station where a large number of EVs are charged simultaneously during a flexible period of time. The proposed technique provides the EVs an optimal charging strategy in order to minimise the cost of charging. The performances of all these new proposed techniques have been demonstrated using Matlab based simulation studies

Optimization of Mobility Parameters using Fuzzy Logic and Reinforcement Learning in Self-Organizing Networks

In this thesis, several optimization techniques for next-generation wireless networks are proposed to solve different problems in the field of Self-Organizing Networks and heterogeneous networks. The common basis of these problems is that network parameters are automatically tuned to deal with the specific problem. As the set of network parameters is extremely large, this work mainly focuses on parameters involved in mobility management. In addition, the proposed self-tuning schemes are based on Fuzzy Logic Controllers (FLC), whose potential lies in the capability to express the knowledge in a similar way to the human perception and reasoning. In addition, in those cases in which a mathematical approach has been required to optimize the behavior of the FLC, the selected solution has been Reinforcement Learning, since this methodology is especially appropriate for learning from interaction, which becomes essential in complex systems such as wireless networks. Taking this into account, firstly, a new Mobility Load Balancing (MLB) scheme is proposed to solve persistent congestion problems in next-generation wireless networks, in particular, due to an uneven spatial traffic distribution, which typically leads to an inefficient usage of resources. A key feature of the proposed algorithm is that not only the parameters are optimized, but also the parameter tuning strategy. Secondly, a novel MLB algorithm for enterprise femtocells scenarios is proposed. Such scenarios are characterized by the lack of a thorough deployment of these low-cost nodes, meaning that a more efficient use of radio resources can be achieved by applying effective MLB schemes. As in the previous problem, the optimization of the self-tuning process is also studied in this case. Thirdly, a new self-tuning algorithm for Mobility Robustness Optimization (MRO) is proposed. This study includes the impact of context factors such as the system load and user speed, as well as a proposal for coordination between the designed MLB and MRO functions. Fourthly, a novel self-tuning algorithm for Traffic Steering (TS) in heterogeneous networks is proposed. The main features of the proposed algorithm are the flexibility to support different operator policies and the adaptation capability to network variations. Finally, with the aim of validating the proposed techniques, a dynamic system-level simulator for Long-Term Evolution (LTE) networks has been designed

Adaptive reinforcement learning for heterogeneous network selection

Next generation 5G mobile wireless networks will consist of multiple technologies for devices to access the network at the edge. One of the keys to 5G is therefore the ability for device to intelligently select its Radio Access Technology (RAT). Current fully distributed algorithms for RAT selection although guaranteeing convergence to equilibrium states, are often slow, require high exploration times and may converge to undesirable equilibria. In this dissertation, we propose three novel reinforcement learning (RL) frameworks to improve the efficiency of existing distributed RAT selection algorithms in a heterogeneous environment, where users may potentially apply a number of different RAT selection procedures. Although our research focuses on solutions for RAT selection in the current and future mobile wireless networks, the proposed solutions in this dissertation are general and suitable to apply for any large scale distributed multi-agent systems. In the first framework, called RL with Non-positive Regret, we propose a novel adaptive RL for multi-agent non-cooperative repeated games. The main contribution is to use both positive and negative regrets in RL to improve the convergence speed and fairness of the well-known regret-based RL procedure. Significant improvements in performance compared to other related algorithms in the literature are demonstrated. In the second framework, called RL with Network-Assisted Feedback (RLNF), our core contribution is to develop a network feedback model that uses network-assisted information to improve the performance of the distributed RL for RAT selection. RLNF guarantees no-regret payoff in the long-run for any user adopting it, regardless of what other users might do and so can work in an environment where not all users use the same learning strategy. This is an important implementation advantage as RLNF can be implemented within current mobile network standards. In the third framework, we propose a novel adaptive RL-based mechanism for RAT selection that can effectively handle user mobility. The key contribution is to leverage forgetting methods to rapidly react to the changes in the radio conditions when users move. We show that our solution improves the performance of wireless networks and converges much faster when users move compared to the non-adaptive solutions. Another objective of the research is to study the impact of various network models on the performance of different RAT selection approaches. We propose a unified benchmark to compare the performances of different algorithms under the same computational environment. The comparative studies reveal that among all the important network parameters that influence the performance of RAT selection algorithms, the number of base stations that a user can connect to has the most significant impact. This finding provides some guidelines for the proper design of RAT selection algorithms for future 5G. Our evaluation benchmark can serve as a reference for researchers, network developers, and engineers. Overall, the thesis provides different reinforcement learning frameworks to improve the efficiency of current fully distributed algorithms for heterogeneous RAT selection. We prove the convergence of the proposed reinforcement learning procedures using the differential inclusion (DI) technique. The theoretical analyses demonstrate that the use of DI not only provides an effective method to study the convergence properties of adaptive procedures in game-theoretic learning, but also yields a much more concise and extensible proof as compared to the classical approaches.Thesis (Ph.D.) -- University of Adelaide, School of Electrical and Electronic Engineering, 201

6G—Enabling the New Smart City: A Survey

Smart cities and 6G are technological areas that have the potential to transform the way we live and work in the years to come. Until this transformation comes into place, there is the need, underlined by research and market studies, for a critical reassessment of the entire wireless communication sector for smart cities, which should include the IoT infrastructure, economic factors that could improve their adoption rate, and strategies that enable smart city operations. Therefore, from a technical point of view, a series of stringent issues, such as interoperability, data privacy, security, the digital divide, and implementation issues have to be addressed. Notably, to concentrate the scrutiny on smart cities and the forthcoming influence of 6G, the groundwork laid by the current 5G, with its multifaceted role and inherent limitations within the domain of smart cities, is embraced as a foundational standpoint. This examination culminates in a panoramic exposition, extending beyond the mere delineation of the 6G standard toward the unveiling of the extensive gamut of potential applications that this emergent standard promises to introduce to the smart cities arena. This paper provides an update on the SC ecosystem around the novel paradigm of 6G, aggregating a series of enabling technologies accompanied by the descriptions of their roles and specific employment schemes