On Improving Efficiency of Data-Intensive Applications in Geo-Distributed Environments

Distributed systems are pervasively demanded and adopted in nowadays for processing data-intensive workloads since they greatly accelerate large-scale data processing with scalable parallelism and improved data locality. Traditional distributed systems initially targeted computing clusters but have since evolved to data centers with multiple clusters. These systems are mostly built on top of homogeneous, tightly integrated resources connected in high-speed local-area networks (LANs), and typically require data to be ingested to a central data center for processing. Today, with enormous volumes of data continuously generated from geographically distributed locations, direct adoption of such systems is prohibitively inefficient due to the limited system scalability and high cost for centralizing the geo-distributed data over the wide-area networks (WANs). More commonly, it becomes a trend to build geo-distributed systems wherein data processing jobs are performed on top of geo-distributed, heterogeneous resources in proximity to the data at vastly distributed geo-locations. However, critical challenges and mechanisms for efficient execution of data-intensive applications in such geo-distributed environments are unclear by far. The goal of this dissertation is to identify such challenges and mechanisms, by extensively using the research principles and methodology of conventional distributed systems to investigate the geo-distributed environment, and by developing new techniques to tackle these challenges and run data-intensive applications with efficiency at scale. The contributions of this dissertation are threefold. Firstly, the dissertation shows that the high level of resource heterogeneity exhibited in the geo-distributed environment undermines the scalability of geo-distributed systems. Virtualization-based resource abstraction mechanisms have been introduced to abstract the hardware, network, and OS resources throughout the system, to mitigate the underlying resource heterogeneity and enhance the system scalability. Secondly, the dissertation reveals the overwhelming performance and monetary cost incurred by indulgent data sharing over the WANs in geo-distributed systems. Network optimization approaches, including linear- programming-based global optimization, greedy bin-packing heuristics, and TCP enhancement, are developed to optimize the network resource utilization and circumvent unnecessary expenses imposed on data sharing in WANs. Lastly, the dissertation highlights the importance of data locality for data-intensive applications running in the geo-distributed environment. Novel data caching and locality-aware scheduling techniques are devised to improve the data locality.Doctor of Philosoph

Gestion de la mobilité dans les réseaux denses de cinquième génération (5G)

Les réseaux de communications mobiles ont connu de profondes avancées technologiques au cours des deux dernières décénnies. La croissance du nombre d’abonnés mobiles ainsi que l’accès à des forfaits de données illimitées, souvent à des tarifs préférentiels, ont engendré une demande de bande passante, de vidéo et de données en forte croissance. Ces progrès significatifs ont favorisé le déploiement de nouveaux services et de nouveaux cas d’utilisation tels que l’Internet-des-objets (IoT), la réalité augmentée et virtuelle, les réseaux de villes intelligentes, les véhicules autonomes et l’automatisation industrielle. Aux technologies existantes, s’ajouteront de nouveaux modes de communication dans le but de répondre à plusieurs cas d’utilisation des réseaux mobiles qui sont encore difficiles à satisfaire à ce jour. Le résultat à long terme de cette nouvelle tournure dans le monde de la réseautique mobile est désigné sous le vocable de réseaux de cinquième génération (5G). Au-delà du déploiement d’applications avancées, les réseaux 5G offriront de nouvelles opportunités de revenus aux fournisseurs de services lorsqu’ils seront combinés aux fonctionnalités avancées telles que l’analyse de données, l’apprentissage automatique et à l’intelligence artificielle. Dans ce contexte, un large consensus est aujourd’hui établit sur la nécessité d’accroître la capacité du réseau par un déploiement massif de cellules de petite taille (Small Cell, SCs), d’un rayon de couverture réduit et à faible puissance. On parle alors d’une ultra-densification du réseau dont le but essentiel est de favoriser la proximité des points d’accès des utilisateurs finaux. Cependant, la densification du réseau implique des relèves fréquentes des usagers mobiles (MNs) entre les SCs et les zones de service. En effet, le rayon de couverture réduit des SCs rend plus complèxe la phase de sélection des relèves en plus d’accroître la fréquence de celles-ci. Ces relèves entraînent des dégradations, des perturbations et des déconnexions qui peuvent entraver l’objectif d’un accès transparent aux services du réseau. En outre, la fréquence des relèves engendre une latence et une charge de signalisation élevées dans le reseau. De plus, l’omniprésence d’applications temps réel exige une latence faible du réseau. Dans ce contexte, la gestion de la mobilité demeure encore un enjeux et il s’avère donc indispensable de concevoir de nouveaux protocoles de gestion de la mobilité capables répondre aux exigences de performances strictes des réseaux 5G.----------ABSTRACT : Mobile communications networks have experienced tremendous technological advances in the last two decades. The growth of the number of mobile subscribers and access to unlimited data plans, often at very affordable prices, have led to an increased demand for bandwidth, video and high-growth data. These significant advances have facilitated the deployment of new services and use cases such as Internet-of-things (IoT), augmented and virtual reality, smart city networks, autonomous vehicles, and industrial automation. On top of the existing technologies, new communication modes will arise to respond to several uses cases of mobile systems that are still difficult to meet today. The long-term result of this new trend in the world of mobile networking gives birth to a new paradigm called the fifth generation networks (5G). Beyond deploying advanced applications, 5G networks will offer new revenue opportunities to service providers, when combined with advanced features such as data analytics, machine learning, and artificial intelligence. In this context, a broad consensus is now established on the need to increase the network capacity through a massive deployment of small cells (Small Cell, SCs), with reduced coverage and low power. This requirement led to the ultra-densification of the network whose primary purpose is to promote the proximity of access points to the end-users. However, the densification of the cellular networks involves many mobile nodes (MNs) going through several handovers between the SCs and the service areas. The shorter SC’s radius makes the handover selection phase more complex while increasing its frequency. These handovers lead to service disruptions and disconnections that may hinder the provision of seamless mobility of network services. Moreover, the frequency of the handovers generates a high latency and signaling load in the network. Besides, the ubiquity of real-time applications requires low network latency. In this context, mobility management is still an issue, and it is, therefore, essential to design new mobility management protocols that can meet the stringent performance requirements of 5G networks