Evaluating the Robustness of Resource Allocations Obtained through Performance Modeling with Stochastic Process Algebra

Recent developments in the field of parallel and distributed computing has led to a proliferation of solving large and computationally intensive mathematical, science, or engineering problems, that consist of several parallelizable parts and several non-parallelizable (sequential) parts. In a parallel and distributed computing environment, the performance goal is to optimize the execution of parallelizable parts of an application on concurrent processors. This requires efficient application scheduling and resource allocation for mapping applications to a set of suitable parallel processors such that the overall performance goal is achieved. However, such computational environments are often prone to unpredictable variations in application (problem and algorithm) and system characteristics. Therefore, a robustness study is required to guarantee a desired level of performance. Given an initial workload, a mapping of applications to resources is considered to be robust if that mapping optimizes execution performance and guarantees a desired level of performance in the presence of unpredictable perturbations at runtime. In this research, a stochastic process algebra, Performance Evaluation Process Algebra (PEPA), is used for obtaining resource allocations via a numerical analysis of performance modeling of the parallel execution of applications on parallel computing resources. The PEPA performance model is translated into an underlying mathematical Markov chain model for obtaining performance measures. Further, a robustness analysis of the allocation techniques is performed for finding a robustmapping from a set of initial mapping schemes. The numerical analysis of the performance models have confirmed similarity with the simulation results of earlier research available in existing literature. When compared to direct experiments and simulations, numerical models and the corresponding analyses are easier to reproduce, do not incur any setup or installation costs, do not impose any prerequisites for learning a simulation framework, and are not limited by the complexity of the underlying infrastructure or simulation libraries

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.Comment: 46 pages, 22 fig

Learning algorithms for the control of routing in integrated service communication networks

There is a high degree of uncertainty regarding the nature of traffic on future integrated service networks. This uncertainty motivates the use of adaptive resource allocation policies that can take advantage of the statistical fluctuations in the traffic demands. The adaptive control mechanisms must be 'lightweight', in terms of their overheads, and scale to potentially large networks with many traffic flows. Adaptive routing is one form of adaptive resource allocation, and this thesis considers the application of Stochastic Learning Automata (SLA) for distributed, lightweight adaptive routing in future integrated service communication networks. The thesis begins with a broad critical review of the use of Artificial Intelligence (AI) techniques applied to the control of communication networks. Detailed simulation models of integrated service networks are then constructed, and learning automata based routing is compared with traditional techniques on large scale networks. Learning automata are examined for the 'Quality-of-Service' (QoS) routing problem in realistic network topologies, where flows may be routed in the network subject to multiple QoS metrics, such as bandwidth and delay. It is found that learning automata based routing gives considerable blocking probability improvements over shortest path routing, despite only using local connectivity information and a simple probabilistic updating strategy. Furthermore, automata are considered for routing in more complex environments spanning issues such as multi-rate traffic, trunk reservation, routing over multiple domains, routing in high bandwidth-delay product networks and the use of learning automata as a background learning process. Automata are also examined for routing of both 'real-time' and 'non-real-time' traffics in an integrated traffic environment, where the non-real-time traffic has access to the bandwidth 'left over' by the real-time traffic. It is found that adopting learning automata for the routing of the real-time traffic may improve the performance to both real and non-real-time traffics under certain conditions. In addition, it is found that one set of learning automata may route both traffic types satisfactorily. Automata are considered for the routing of multicast connections in receiver-oriented, dynamic environments, where receivers may join and leave the multicast sessions dynamically. Automata are shown to be able to minimise the average delay or the total cost of the resulting trees using the appropriate feedback from the environment. Automata provide a distributed solution to the dynamic multicast problem, requiring purely local connectivity information and a simple updating strategy. Finally, automata are considered for the routing of multicast connections that require QoS guarantees, again in receiver-oriented dynamic environments. It is found that the distributed application of learning automata leads to considerably lower blocking probabilities than a shortest path tree approach, due to a combination of load balancing and minimum cost behaviour

Reinforcement Learning

Brains rule the world, and brain-like computation is increasingly used in computers and electronic devices. Brain-like computation is about processing and interpreting data or directly putting forward and performing actions. Learning is a very important aspect. This book is on reinforcement learning which involves performing actions to achieve a goal. The first 11 chapters of this book describe and extend the scope of reinforcement learning. The remaining 11 chapters show that there is already wide usage in numerous fields. Reinforcement learning can tackle control tasks that are too complex for traditional, hand-designed, non-learning controllers. As learning computers can deal with technical complexities, the tasks of human operators remain to specify goals on increasingly higher levels. This book shows that reinforcement learning is a very dynamic area in terms of theory and applications and it shall stimulate and encourage new research in this field

Evolution of High Throughput Satellite Systems: Vision, Requirements, and Key Technologies

High throughput satellites (HTS), with their digital payload technology, are expected to play a key role as enablers of the upcoming 6G networks. HTS are mainly designed to provide higher data rates and capacities. Fueled by technological advancements including beamforming, advanced modulation techniques, reconfigurable phased array technologies, and electronically steerable antennas, HTS have emerged as a fundamental component for future network generation. This paper offers a comprehensive state-of-the-art of HTS systems, with a focus on standardization, patents, channel multiple access techniques, routing, load balancing, and the role of software-defined networking (SDN). In addition, we provide a vision for next-satellite systems that we named as extremely-HTS (EHTS) toward autonomous satellites supported by the main requirements and key technologies expected for these systems. The EHTS system will be designed such that it maximizes spectrum reuse and data rates, and flexibly steers the capacity to satisfy user demand. We introduce a novel architecture for future regenerative payloads while summarizing the challenges imposed by this architecture

Multi-agent system implementation in demand response: A literature review and bibliometric evaluation

This research provides a comprehensive literature overview and bibliometric evaluation of multi-agent system (MAS) implementation in energy demand response (DR) to identify gaps. The review encompasses 39 relevant papers from searches in three academic databases, focusing on studies published from 2012 to the middle of 2023. The review includes MAS frameworks, optimization algorithms, communication protocols, market structures and evaluation methodologies. Bibliometric analysis of 587 documents from the search on the Scopus database identified prolific authors, influential articles and collaborative networks within the field. The findings reveal growing research interest in implementing an MAS for DR, focusing on integrating intelligent agents into electricity grids to enable effective load management and enhance grid stability. Additionally, the review outlines potential research directions, including exploring advanced MAS techniques, interoperability challenges, policy implications and the integration of renewable energy sources

Development Schemes of Electric Vehicle Charging Protocols and Implementation of Algorithms for Fast Charging under Dynamic Environments Leading towards Grid-to-Vehicle Integration

This thesis focuses on the development of electric vehicle (EV) charging protocols under a dynamic environment using artificial intelligence (AI), to achieve Vehicle-to-Grid (V2G) integration and promote automobile electrification. The proposed framework comprises three major complementary steps. Firstly, the DC fast charging scheme is developed under different ambient conditions such as temperature and relative humidity. Subsequently, the transient performance of the controller is improved while implementing the proposed DC fast charging scheme. Finally, various novel techno-economic scenarios and case studies are proposed to integrate EVs with the utility grid. The proposed novel scheme is composed of hierarchical stages; In the first stage, an investigation of the temperature or/and relative humidity impact on the charging process is implemented using the constant current-constant voltage (CC-CV) protocol. Where the relative humidity impact on the charging process was not investigated or mentioned in the literature survey. This was followed by the feedforward backpropagation neural network (FFBP-NN) classification algorithm supported by the statistical analysis of an instant charging current sample of only 10 seconds at any ambient condition. Then the FFBP-NN perfectly estimated the EV’s battery terminal voltage, charging current, and charging interval time with an error of 1% at the corresponding temperature and relative humidity. Then, a nonlinear identification model of the lithium-polymer ion battery dynamic behaviour is introduced based on the Hammerstein-Wiener (HW) model with an experimental error of 1.1876%. Compared with the CC-CV fast charging protocol, intelligent novel techniques based on the multistage charging current protocol (MSCC) are proposed using the Cuckoo optimization algorithm (COA). COA is applied to the Hierarchical technique (HT) and the Conditional random technique (CRT). Compared with the CC-CV charging protocol, an improvement in the charging efficiency of 8% and 14.1% was obtained by the HT and the CRT, respectively, in addition to a reduction in energy losses of 7.783% and 10.408% and a reduction in charging interval time of 18.1% and 22.45%, respectively. The stated charging protocols have been implemented throughout a smart charger. The charger comprises a DC-DC buck converter controlled by an artificial neural network predictive controller (NNPC), trained and supported by the long short-term memory neural network (LSTM). The LSTM network model was utilized in the offline forecasting of the PV output power, which was fed to the NNPC as the training data. The NNPC–LSTM controller was compared with the fuzzy logic (FL) and the conventional PID controllers and perfectly ensured that the optimum transient performance with a minimum battery terminal voltage ripple reached 1 mV with a very high-speed response of 1 ms in reaching the predetermined charging current stages. Finally, to alleviate the power demand pressure of the proposed EV charging framework on the utility grid, a novel smart techno-economic operation of an electric vehicle charging station (EVCS) in Egypt controlled by the aggregator is suggested based on a hierarchical model of multiple scenarios. The deterministic charging scheduling of the EVs is the upper stage of the model to balance the generated and consumed power of the station. Mixed-integer linear programming (MILP) is used to solve the first stage, where the EV charging peak demand value is reduced by 3.31% (4.5 kW). The second challenging stage is to maximize the EVCS profit whilst minimizing the EV charging tariff. In this stage, MILP and Markov Decision Process Reinforcement Learning (MDP-RL) resulted in an increase in EVCS revenue by 28.88% and 20.10%, respectively. Furthermore, the grid-to-vehicle (G2V) and vehicle-to-grid (V2G) technologies are applied to the stochastic EV parking across the day, controlled by the aggregator to alleviate the utility grid load demand. The aggregator determined the number of EVs that would participate in the electric power trade and sets the charging/discharging capacity level for each EV. The proposed model minimized the battery degradation cost while maximizing the revenue of the EV owner and minimizing the utility grid load demand based on the genetic algorithm (GA). The implemented procedure reduced the degradation cost by an average of 40.9256%, increased the EV SOC by 27%, and ensured an effective grid stabilization service by shaving the load demand to reach a predetermined grid average power across the day where the grid load demand decreased by 26.5% (371 kW)