Deep Reinforcement Learning-based Content Migration for Edge Content Delivery Networks with Vehicular Nodes

With the explosive demands for data, content delivery networks are facing ever-increasing challenges to meet end-users quality-of-experience requirements, especially in terms of delay. Content can be migrated from surrogate servers to local caches closer to end-users to address delay challenges. Unfortunately, these local caches have limited capacities, and when they are fully occupied, it may sometimes be necessary to remove their lower-priority content to accommodate higher-priority content. At other times, it may be necessary to return previously removed content to local caches. Downloading this content from surrogate servers is costly from the perspective of network usage, and potentially detrimental to the end-user QoE in terms of delay. In this paper, we consider an edge content delivery network with vehicular nodes and propose a content migration strategy in which local caches offload their contents to neighboring edge caches whenever feasible, instead of removing their contents when they are fully occupied. This process ensures that more contents remain in the vicinity of end-users. However, selecting which contents to migrate and to which neighboring cache to migrate is a complicated problem. This paper proposes a deep reinforcement learning approach to minimize the cost. Our simulation scenarios realized up to a 70% reduction of content access delay cost compared to conventional strategies with and without content migration

Deep Learning Techniques for Mobility Prediction and Management in Mobile Networks

Trajectory prediction is an important research topic in modern mobile networks (e.g., 5G and beyond 5G) to enhance the network quality of service by accurately predicting the future locations of mobile users, such as pedestrians and vehicles, based on their past mobility patterns. A trajectory is defined as the sequence of locations the user visits over time. The primary objective of this thesis is to improve the modeling of mobility data and establish personalized, scalable, collective-intelligent, distributed, and strategic trajectory prediction techniques that can effectively adapt to the dynamics of urban environments in order to facilitate the optimal delivery of mobility-aware network services. Our proposed approaches aim to increase the accuracy of trajectory prediction while minimizing communication and computational costs leading to more efficient mobile networks. The thesis begins by introducing a personalized trajectory prediction technique using deep learning and reinforcement learning. It adapts the neural network architecture to capture the distinct characteristics of mobile users’ data. Furthermore, it introduces advanced anticipatory handover management and dynamic service migration techniques that optimize network management using our high-performance trajectory predictor. This approach ensures seamless connectivity and proactively migrates network services, enhancing the quality of service in dense wireless networks. The second contribution of the thesis introduces cluster-level prediction to extend the reinforcement learning-based trajectory prediction, addressing scalability challenges in large-scale networks. Cluster-level trajectory prediction leverages users’ similarities within clusters to train only a few representatives. This enables efficient transfer learning of pre-trained mobility models and reduces computational overhead enhancing the network scalability. The third contribution proposes a collaborative social-aware multi-agent trajectory prediction technique that accounts for the interactions between multiple intra-cluster agents in a dynamic urban environment, increasing the prediction accuracy but decreasing the algorithm complexity and computational resource usage. The fourth contribution proposes a federated learning-driven multi-agent trajectory prediction technique that leverages the collaborative power of multiple local data sources in a decentralized manner to enhance user privacy and improve the accuracy of trajectory prediction while jointly minimizing computational and communication costs. The fifth contribution proposes a game theoretic non-cooperative multi-agent prediction technique that considers the strategic behaviors among competitive inter-cluster mobile users. The proposed approaches are evaluated on small-scale and large-scale location-based mobility datasets, where locations could be GPS coordinates or cellular base station IDs. Our experiments demonstrate that our proposed approaches outperform state-of-the-art trajectory prediction methods making significant contributions to the field of mobile networks

Navigation of micro-swimmers in steady flow: the importance of symmetries

Marine microorganisms must cope with complex flow patterns and even turbulence as they navigate the ocean. To survive they must avoid predation and find efficient energy sources. A major difficulty in analysing possible survival strategies is that the time series of environmental cues in non-linear flow is complex, and that it depends on the decisions taken by the organism. One way of determining and evaluating optimal strategies is reinforcement learning. In a proof-of-principle study, Colabrese et al. [Phys. Rev. Lett. (2017)] used this method to find out how a micro-swimmer in a vortex flow can navigate towards the surface as quickly as possible, given a fixed swimming speed. The swimmer measured its instantaneous swimming direction and the local flow vorticity in the laboratory frame, and reacted to these cues by swimming either left, right, up, or down. However, usually a motile microorganism measures the local flow rather than global information, and it can only react in relation to the local flow, because in general it cannot access global information (such as up or down in the laboratory frame). Here we analyse optimal strategies with local signals and actions that do not refer to the laboratory frame. We demonstrate that symmetry-breaking is required in order to learn vertical migration in a meaningful way. Using reinforcement learning we analyse the emerging strategies for different sets of environmental cues that microorganisms are known to measure.Comment: 20 pages, 5 figure

A Review on Computational Intelligence Techniques in Cloud and Edge Computing

Cloud computing (CC) is a centralized computing paradigm that accumulates resources centrally and provides these resources to users through Internet. Although CC holds a large number of resources, it may not be acceptable by real-time mobile applications, as it is usually far away from users geographically. On the other hand, edge computing (EC), which distributes resources to the network edge, enjoys increasing popularity in the applications with low-latency and high-reliability requirements. EC provides resources in a decentralized manner, which can respond to users’ requirements faster than the normal CC, but with limited computing capacities. As both CC and EC are resource-sensitive, several big issues arise, such as how to conduct job scheduling, resource allocation, and task offloading, which significantly influence the performance of the whole system. To tackle these issues, many optimization problems have been formulated. These optimization problems usually have complex properties, such as non-convexity and NP-hardness, which may not be addressed by the traditional convex optimization-based solutions. Computational intelligence (CI), consisting of a set of nature-inspired computational approaches, recently exhibits great potential in addressing these optimization problems in CC and EC. This article provides an overview of research problems in CC and EC and recent progresses in addressing them with the help of CI techniques. Informative discussions and future research trends are also presented, with the aim of offering insights to the readers and motivating new research directions

Jeffery's orbits and microswimmers in flows: A theoretical review

In this review, we provide a theoretical introduction to Jeffery's equations for the orientation dynamics of an axisymmetric object in a flow at low Reynolds number, and review recent theoretical extensions and applications to the motions of self-propelled particles, so-called microswimmers, in external flows. Bacteria colonize human organs and medical devices even with flowing fluid, microalgae occasionally cause huge harmful toxic blooms in lakes and oceans, and recent artificial microrobots can migrate in flows generated in well-designed microfluidic chambers. The Jeffery equations, a simple set of ordinary differential equation, provide a useful building block in modeling, analyzing, and understanding these microswimmer dynamics in a flow current, in particular when incorporating the impact of the swimmer shape since the equations contain a shape parameter as a single scalar, known as the Bretherton parameter. The particle orientation forms a closed orbit when situated in a simple shear, and this non-uniform periodic rotational motion, referred to as Jeffery's orbits, is due to a constant of motion in the non-linear equation. After providing a theoretical introduction to microswimmer hydrodynamics and a derivation of the Jeffery equations, we discuss possible extensions to more general shapes, including those with rapid deformation. In the latter part of this review, simple mathematical models of microswimmers in different types of flow fields are described, with a focus on constants of motion and their relation to periodic motions in phase space, together with a breakdown of degenerate orbits, to discuss the stable, unstable, and chaotic dynamics of the system. The discussion in this paper will provide a comprehensive theoretical foundation for Jeffery's orbits and will be useful to understand the motions of microswimmers under various flows.Comment: 26 pages, 13 figures. To appear in the Journal of the Physical Society of Japa

Perspectives on adaptive dynamical systems

Adaptivity is a dynamical feature that is omnipresent in nature, socio-economics, and technology. For example, adaptive couplings appear in various real-world systems like the power grid, social, and neural networks, and they form the backbone of closed-loop control strategies and machine learning algorithms. In this article, we provide an interdisciplinary perspective on adaptive systems. We reflect on the notion and terminology of adaptivity in different disciplines and discuss which role adaptivity plays for various fields. We highlight common open challenges, and give perspectives on future research directions, looking to inspire interdisciplinary approaches.Comment: 46 pages, 9 figure