Age of Semantics in Cooperative Communications: To Expedite Simulation Towards Real via Offline Reinforcement Learning

The age of information metric fails to correctly describe the intrinsic semantics of a status update. In an intelligent reflecting surface-aided cooperative relay communication system, we propose the age of semantics (AoS) for measuring semantics freshness of the status updates. Specifically, we focus on the status updating from a source node (SN) to the destination, which is formulated as a Markov decision process (MDP). The objective of the SN is to maximize the expected satisfaction of AoS and energy consumption under the maximum transmit power constraint. To seek the optimal control policy, we first derive an online deep actor-critic (DAC) learning scheme under the on-policy temporal difference learning framework. However, implementing the online DAC in practice poses the key challenge in infinitely repeated interactions between the SN and the system, which can be dangerous particularly during the exploration. We then put forward a novel offline DAC scheme, which estimates the optimal control policy from a previously collected dataset without any further interactions with the system. Numerical experiments verify the theoretical results and show that our offline DAC scheme significantly outperforms the online DAC scheme and the most representative baselines in terms of mean utility, demonstrating strong robustness to dataset quality.Comment: This work has been submitted to the IEEE for possible publicatio

Allocation of Communication and Computation Resources in Mobile Networks

Konvergence komunikačních a výpočetních technologií vedlo k vzniku Multi-Access Edge Computing (MEC). MEC poskytuje výpočetní výkon na tzv. hraně mobilních sítí (základnové stanice, jádro mobilní sítě), který lze využít pro optimalizaci mobilních sítí v reálném čase. Optimalizacev reálném čase je umožněna díky nízkému komunikačnímu zpoždění například v porovnání s Mobile Cloud Computing (MCC). Optimalizace mobilních sítí vyžaduje informace o mobilní síti od uživatelských zařízeních, avšak sběr těchto informací využívá komunikační prostředky, které jsou využívány i pro přenos uživatelských dat. Zvyšující se počet uživatelských zařízení, senzorů a taktéž komunikace vozidel tvoří překážku pro sběr informací o mobilních sítích z důvodu omezeného množství komunikačních prostředků. Tudíž je nutné navrhnout řešení, která umožní sběr těchto informací pro potřeby optimalizace mobilních sítí. V této práci je navrženo řešení pro komunikaci vysokého počtu zařízeních, které je postaveno na využití přímé komunikace mezi zařízeními. Pro motivování uživatelů, pro využití přeposílání dat pomocí přímé komunikace mezi uživateli je navrženo přidělování komunikačních prostředků jenž vede na přirozenou spolupráci uživatelů. Dále je provedena analýza spotřeby energie při využití přeposílání dat pomocí přímé komunikace mezi uživateli pro ukázání jejích výhod z pohledu spotřeby energie. Pro další zvýšení počtu komunikujících zařízení je využito mobilních létajících základových stanic (FlyBS). Pro nasazení FlyBS je navržen algoritmus, který hledá pozici FlyBS a asociaci uživatel k FlyBS pro zvýšení spokojenosti uživatelů s poskytovanými datovými propustnostmi. MEC lze využít nejen pro optimalizaci mobilních sítí z pohledu mobilních operátorů, ale taktéž uživateli mobilních sítí. Tito uživatelé mohou využít MEC pro přenost výpočetně náročných úloh z jejich mobilních zařízeních do MEC. Z důvodu mobility uživatel je nutné nalézt vhodně přidělení komunikačních a výpočetních prostředků pro uspokojení uživatelských požadavků. Tudíž je navržen algorithmus pro výběr komunikační cesty mezi uživatelem a MEC, jenž je posléze rozšířen o přidělování výpočetných prostředků společně s komunikačními prostředky. Navržené řešení vede k snížení komunikačního zpoždění o desítky procent.The convergence of communication and computing in the mobile networks has led to an introduction of the Multi-Access Edge Computing (MEC). The MEC combines communication and computing resources at the edge of the mobile network and provides an option to optimize the mobile network in real-time. This is possible due to close proximity of the computation resources in terms of communication delay, in comparison to the Mobile Cloud Computing (MCC). The optimization of the mobile networks requires information about the mobile network and User Equipment (UE). Such information, however, consumes a significant amount of communication resources. The finite communication resources along with the ever increasing number of the UEs and other devices, such as sensors, vehicles pose an obstacle for collecting the required information. Therefore, it is necessary to provide solutions to enable the collection of the required mobile network information from the UEs for the purposes of the mobile network optimization. In this thesis, a solution to enable communication of a large number of devices, exploiting Device-to-Device (D2D) communication for data relaying, is proposed. To motivate the UEs to relay data of other UEs, we propose a resource allocation algorithm that leads to a natural cooperation of the UEs. To show, that the relaying is not only beneficial from the perspective of an increased number of UEs, we provide an analysis of the energy consumed by the D2D communication. To further increase the number of the UEs we exploit a recent concept of the flying base stations (FlyBSs), and we develop a joint algorithm for a positioning of the FlyBS and an association of the UEs to increase the UEs satisfaction with the provided data rates. The MEC can be exploited not only for processing of the collected data to optimize the mobile networks, but also by the mobile users. The mobile users can exploit the MEC for the computation offloading, i.e., transferring the computation from their UEs to the MEC. However, due to the inherent mobility of the UEs, it is necessary to determine communication and computation resource allocation in order to satisfy the UEs requirements. Therefore, we first propose a solution for a selection of the communication path between the UEs and the MEC (communication resource allocation). Then, we also design an algorithm for joint communication and computation resource allocation. The proposed solution then lead to a reduction in the computation offloading delay by tens of percent

Multimedia delivery in the future internet

The term “Networked Media” implies that all kinds of media including text, image, 3D graphics, audio and video are produced, distributed, shared, managed and consumed on-line through various networks, like the Internet, Fiber, WiFi, WiMAX, GPRS, 3G and so on, in a convergent manner [1]. This white paper is the contribution of the Media Delivery Platform (MDP) cluster and aims to cover the Networked challenges of the Networked Media in the transition to the Future of the Internet. Internet has evolved and changed the way we work and live. End users of the Internet have been confronted with a bewildering range of media, services and applications and of technological innovations concerning media formats, wireless networks, terminal types and capabilities. And there is little evidence that the pace of this innovation is slowing. Today, over one billion of users access the Internet on regular basis, more than 100 million users have downloaded at least one (multi)media file and over 47 millions of them do so regularly, searching in more than 160 Exabytes1 of content. In the near future these numbers are expected to exponentially rise. It is expected that the Internet content will be increased by at least a factor of 6, rising to more than 990 Exabytes before 2012, fuelled mainly by the users themselves. Moreover, it is envisaged that in a near- to mid-term future, the Internet will provide the means to share and distribute (new) multimedia content and services with superior quality and striking flexibility, in a trusted and personalized way, improving citizens’ quality of life, working conditions, edutainment and safety. In this evolving environment, new transport protocols, new multimedia encoding schemes, cross-layer inthe network adaptation, machine-to-machine communication (including RFIDs), rich 3D content as well as community networks and the use of peer-to-peer (P2P) overlays are expected to generate new models of interaction and cooperation, and be able to support enhanced perceived quality-of-experience (PQoE) and innovative applications “on the move”, like virtual collaboration environments, personalised services/ media, virtual sport groups, on-line gaming, edutainment. In this context, the interaction with content combined with interactive/multimedia search capabilities across distributed repositories, opportunistic P2P networks and the dynamic adaptation to the characteristics of diverse mobile terminals are expected to contribute towards such a vision. Based on work that has taken place in a number of EC co-funded projects, in Framework Program 6 (FP6) and Framework Program 7 (FP7), a group of experts and technology visionaries have voluntarily contributed in this white paper aiming to describe the status, the state-of-the art, the challenges and the way ahead in the area of Content Aware media delivery platforms

Energy efficiency in wireless communication

This era would probably be recognized as the information age, hence as a paramount milestone in the progress of mankind, by the future historians. One of the most significant achievements of this age is, making it possible to transmit and receive information effectively and reliably via wireless radio technology. The demand of wireless communication is increasing in a never-resting pace, imposing bigger challenge not only on service providers but also on innovators and researches to innovate out-of-the-box technologies. These challenges include faster data communication over seamless, reliable and cost effective wireless networks, utilizing the limited physical radio resources as well as considering the environmental impact caused by the increasing energy consumption. The ever-expanding wireless communication infrastructure is withdrawing higher energy than ever, raising the need for finding more efficient systems. The challenge of developing efficient wireless systems can be addressed on several levels, starting from device electronics, up to the network-level architecture and protocols. The anticipated gains of achieving such efficiency is the key feature of extending mobile devices' battery life and reducing environmental and economic impacts of wireless communication infrastructure. Therefore energy efficient designs are urgently needed from both environmental and economic aspects of wireless networks. In this research, we explore the field of energy efficiency in MAC and Physical layers of wireless networks in order to enhance the performance and reliability of future wireless networks as well as to reduce its environmental footprint. In the first part of this research, we analyse the energy efficiency of two mostly used modulation techniques, namely MQAM and MFSK, for short range wireless transmissions, up to a few $100$s of meters, and propose optimum rate adaptation to minimize the energy dissipation during transmissions. Energy consumed for transmitting the data over a distance to maintain a prescribed error probability together with the circuit energy have been considered in our work. We provide novel results for optimal rate adaptation for improved energy efficiency. Our results indicate that the energy efficiency can be significantly improved by performing optimal rate adaptation given the radio and channel parameters, and furthermore we identify the maximum distance where optimal rate adaptation can be performed beyond which the optimum rate then becomes the same as the minimum data rate. In the second part of this research, we propose energy efficient algorithm for cellular base stations. In cellular networks, the base stations are the most energy consuming parts, which consume approximately $60-80\%$ of the total energy. Hence control and optimization of energy consumption at base stations should be at the heart of any green radio engineering scheme. Sleep mode implementation in base stations has proven to be a very good approach for the energy efficiency of cellular BSs. Therefore, we have proposed a novel strategy for improving energy efficiency on ternary state transceivers for cellular BSs. We consider transceivers that are capable of switching between sleep, stand-by and active modes whenever required. We have modelled these ternary state transceivers as a three-state Markov model and have presented an algorithm based on Markov model to intelligently switch among the states of the transceivers based on the offered traffic whilst maintaining a prescribed minimum rate per user. We consider a typical macro BS with state changeable transceivers and our results show that it is possible to improve the energy efficiency of the BS by approximately $40\%$ using the proposed MDP based algorithm. In the third part of this research, we propose energy efficient algorithm for aerial base stations. Recently aerial base stations are investigated to provide wireless coverage to terrestrial radio terminals. The advantages of using aerial platforms in providing wireless coverage are many including larger coverage in remote areas, better line-of-sight conditions etc. Energy is a scarce resource for aerial base stations, hence the wise management of energy is quite beneficial for the aerial network. In this context, we study the means of reducing the total energy consumption by designing and implementing an energy efficient aerial base station. Sleep mode implementation in base stations (BSs) has proven to be a very good approach for improving the energy efficiency; therefore we propose a novel strategy for further improving energy efficiency by considering ternary state transceivers of aerial base stations. Using the three state model we propose a Markovian Decision process (MDP) based algorithm to switch between the states for improving the energy efficiency of the aerial base station. The MDP based approach intelligently switches between the states of the transceivers based on the offered traffic whilst maintaining a prescribed minimum channel rate per user. Our simulation results show that there is a around $40\%$ gain in the energy efficiency when using our proposed MDP algorithm together with the three-state transceiver model for the base station compared to the always active mode. We have also shown the energy-delay trade-off in order to design an efficient aerial base station. In the final part of our work, we propose a novel energy efficient handover algorithm, based on Markov decision process (MDP) for the two-tier LTE network, towards reducing power transmissions at the mobile terminal side. The proposed policy is LTE backward-compatible, as it can be employed by suitably adapting a prescribed SNR target and standard LTE measurements. Simulation results reveal that compared to the widely adopted policy based on strongest cell and another energy efficient policy, our proposed policy can greatly reduce the power consumption at the LTE mobile terminals. Most of our works presented in this dissertation has been published in conference proceeding and some of them are currently undergoing a review process for journals. These publications will be highlighted and identified at the end of the first chapter of this dissertation

A hybrid intelligent model for network selection in the industrial Internet of Things

Industrial Internet of Things (IIoT) plays an important role in increasing productivity and efficiency in heterogeneous wireless networks. However, different domains such as industrial wireless scenarios, small cell domains and vehicular ad hoc networks (VANET) require an efficient machine learning/intelligent algorithm to process the vertical handover decision that can maintain mobile terminals (MTs) in the preferable networks for a sufficient duration of time. The preferred quality of service parameters can be differentiated from all the other MTs. Hence, in this paper, the problem with the vertical handoff (VHO) decision is articulated as the process of the Markov decision aimed to maximize the anticipated total rewards as well as to minimize the handoffs’ average count. A rewards function is designed to evaluate the QoS at the point of when the connections take place, as that is where the policy decision for a stationary deterministic handoff can be established. The proposed hybrid model merges the biogeography-based optimization (BBO) with the Markov decision process (MDP). The MDP is utilized to establish the radio access technology (RAT) selection’s probability that behaves as an input to the BBO process. Therefore, the BBO determines the best RAT using the described multi-point algorithm in the heterogeneous network. The numerical findings display the superiority of this paper’s proposed schemes in comparison with other available algorithms. The findings shown that the MDP-BBO algorithm is able to outperform other algorithms in terms of number of handoffs, bandwidth availability, and decision delays. Our algorithm displayed better expected total rewards as well as a reduced average account of handoffs compared to current approaches. Simulation results obtained from Monte-Carlo experiments prove validity of the proposed model