Data transfer scheduling with advance reservation and provisioning

Over the years, scientific applications have become more complex and more data intensive. Although through the use of distributed resources the institutions and organizations gain access to the resources needed for their large-scale applications, complex middleware is required to orchestrate the use of these storage and network resources between collaborating parties, and to manage the end-to-end processing of data. We present a new data scheduling paradigm with advance reservation and provisioning. Our methodology provides a basis for provisioning end-to-end high performance data transfers which require integration between system, storage and network resources, and coordination between reservation managers and data transfer nodes. This allows researchers/users and higher level meta-schedulers to use data placement as a service where they can plan ahead and reserve time and resources for their data movement operations. We present a novel approach for evaluating time-dependent structures with bandwidth guaranteed paths. We present a practical online scheduling model using advance reservation in dynamic network with time constraints. In addition, we report a new polynomial algorithm presenting possible reservation options and alternatives for earliest completion and shortest transfer duration. We enhance the advance network reservation system by extending the underlying mechanism to provide a new service in which users submit their constraints and the system suggests possible reservation requests satisfying users\u27 requirements. We have studied scheduling data transfer operation with resource and time conflicts. We have developed a new scheduling methodology considering resource allocation in client sites and bandwidth allocation on network link connecting resources. Some other major contributions of our study include enhanced reliability, adaptability, and performance optimization of distributed data placement tasks. While designing this new data scheduling architecture, we also developed other important methodologies such as early error detection, failure awareness, job aggregation, and dynamic adaptation of distributed data placement tasks. The adaptive tuning includes dynamically setting data transfer parameters and controlling utilization of available network capacity. Our research aims to provide a middleware to improve the data bottleneck in high performance computing systems

Datacenter Traffic Control: Understanding Techniques and Trade-offs

Datacenters provide cost-effective and flexible access to scalable compute and storage resources necessary for today's cloud computing needs. A typical datacenter is made up of thousands of servers connected with a large network and usually managed by one operator. To provide quality access to the variety of applications and services hosted on datacenters and maximize performance, it deems necessary to use datacenter networks effectively and efficiently. Datacenter traffic is often a mix of several classes with different priorities and requirements. This includes user-generated interactive traffic, traffic with deadlines, and long-running traffic. To this end, custom transport protocols and traffic management techniques have been developed to improve datacenter network performance. In this tutorial paper, we review the general architecture of datacenter networks, various topologies proposed for them, their traffic properties, general traffic control challenges in datacenters and general traffic control objectives. The purpose of this paper is to bring out the important characteristics of traffic control in datacenters and not to survey all existing solutions (as it is virtually impossible due to massive body of existing research). We hope to provide readers with a wide range of options and factors while considering a variety of traffic control mechanisms. We discuss various characteristics of datacenter traffic control including management schemes, transmission control, traffic shaping, prioritization, load balancing, multipathing, and traffic scheduling. Next, we point to several open challenges as well as new and interesting networking paradigms. At the end of this paper, we briefly review inter-datacenter networks that connect geographically dispersed datacenters which have been receiving increasing attention recently and pose interesting and novel research problems.Comment: Accepted for Publication in IEEE Communications Surveys and Tutorial

Failure-awareness and dynamic adaptation in data scheduling

Over the years, scientific applications have become more complex and more data intensive. Especially large scale simulations and scientific experiments in areas such as physics, biology, astronomy and earth sciences demand highly distributed resources to satisfy excessive computational requirements. Increasing data requirements and the distributed nature of the resources made I/O the major bottleneck for end-to-end application performance. Existing systems fail to address issues such as reliability, scalability, and efficiency in dealing with wide area data access, retrieval and processing. In this study, we explore data-intensive distributed computing and study challenges in data placement in distributed environments. After analyzing different application scenarios, we develop new data scheduling methodologies and the key attributes for reliability, adaptability and performance optimization of distributed data placement tasks. Inspired by techniques used in microprocessor and operating system architectures, we extend and adapt some of the known low-level data handling and optimization techniques to distributed computing. Two major contributions of this work include (i) a failure-aware data placement paradigm for increased fault-tolerance, and (ii) adaptive scheduling of data placement tasks for improved end-to-end performance. The failure-aware data placement includes early error detection, error classification, and use of this information in scheduling decisions for the prevention of and recovery from possible future errors. The adaptive scheduling approach includes dynamically tuning data transfer parameters over wide area networks for efficient utilization of available network capacity and optimized end-to-end data transfer performance

Optimization of 5G Second Phase Heterogeneous Radio Access Networks with Small Cells

Due to the exponential increase in high data-demanding applications and their services per coverage area, it is becoming challenging for the existing cellular network to handle the massive sum of users with their demands. It is conceded to network operators that the current wireless network may not be capable to shelter future traffic demands. To overcome the challenges the operators are taking interest in efficiently deploying the heterogeneous network. Currently, 5G is in the commercialization phase. Network evolution with addition of small cells will develop the existing wireless network with its enriched capabilities and innovative features. Presently, the 5G global standardization has introduced the 5G New Radio (NR) under the 3rd Generation Partnership Project (3GPP). It can support a wide range of frequency bands (<6 GHz to 100 GHz). For different trends and verticals, 5G NR encounters, functional splitting and its cost evaluation are well-thought-out. The aspects of network slicing to the assessment of the business opportunities and allied standardization endeavours are illustrated. The study explores the carrier aggregation (Pico cellular) technique for 4G to bring high spectral efficiency with the support of small cell massification while benefiting from statistical multiplexing gain. One has been able to obtain values for the goodput considering CA in LTE-Sim (4G), of 40 Mbps for a cell radius of 500 m and of 29 Mbps for a cell radius of 50 m, which is 3 times higher than without CA scenario (2.6 GHz plus 3.5 GHz frequency bands). Heterogeneous networks have been under investigation for many years. Heterogeneous network can improve users service quality and resource utilization compared to homogeneous networks. Quality of service can be enhanced by putting the small cells (Femtocells or Picocells) inside the Microcells or Macrocells coverage area. Deploying indoor Femtocells for 5G inside the Macro cellular network can reduce the network cost. Some service providers have started their solutions for indoor users but there are still many challenges to be addressed. The 5G air-simulator is updated to deploy indoor Femto-cell with proposed assumptions with uniform distribution. For all the possible combinations of apartments side length and transmitter power, the maximum number of supported numbers surpassed the number of users by more than two times compared to papers mentioned in the literature. Within outdoor environments, this study also proposed small cells optimization by putting the Pico cells within a Macro cell to obtain low latency and high data rate with the statistical multiplexing gain of the associated users. Results are presented 5G NR functional split six and split seven, for three frequency bands (2.6 GHz, 3.5GHz and 5.62 GHz). Based on the analysis for shorter radius values, the best is to select the 2.6 GHz to achieve lower PLR and to support a higher number of users, with better goodput, and higher profit (for cell radius u to 400 m). In 4G, with CA, from the analysis of the economic trade-off with Picocell, the Enhanced multi-band scheduler EMBS provide higher revenue, compared to those without CA. It is clearly shown that the profit of CA is more than 4 times than in the without CA scenario. This means that the slight increase in the cost of CA gives back more than 4-time profit relatively to the ”without” CA scenario.Devido ao aumento exponencial de aplicações/serviços de elevado débito por unidade de área, torna-se bastante exigente, para a rede celular existente, lidar com a enormes quantidades de utilizadores e seus requisitos. É reconhecido que as redes móveis e sem fios atuais podem não conseguir suportar a procura de tráfego junto dos operadores. Para responder a estes desafios, os operadores estão-se a interessar pelo desenvolvimento de redes heterogéneas eficientes. Atualmente, a 5G está na fase de comercialização. A evolução destas redes concretizar-se-á com a introdução de pequenas células com aptidões melhoradas e características inovadoras. No presente, os organismos de normalização da 5G globais introduziram os Novos Rádios (NR) 5G no contexto do 3rd Generation Partnership Project (3GPP). A 5G pode suportar uma gama alargada de bandas de frequência (<6 a 100 GHz). Abordam-se as divisões funcionais e avaliam-se os seus custos para as diferentes tendências e verticais dos NR 5G. Ilustram-se desde os aspetos de particionamento funcional da rede à avaliação das oportunidades de negócio, aliadas aos esforços de normalização. Exploram-se as técnicas de agregação de espetro (do inglês, CA) para pico células, em 4G, a disponibilização de eficiência espetral, com o suporte da massificação de pequenas células, e o ganho de multiplexagem estatística associado. Obtiveram-se valores do débito binário útil, considerando CA no LTE-Sim (4G), de 40 e 29 Mb/s para células de raios 500 e 50 m, respetivamente, três vezes superiores em relação ao caso sem CA (bandas de 2.6 mais 3.5 GHz). Nas redes heterogéneas, alvo de investigação há vários anos, a qualidade de serviço e a utilização de recursos podem ser melhoradas colocando pequenas células (femto- ou pico-células) dentro da área de cobertura de micro- ou macro-células). O desenvolvimento de pequenas células 5G dentro da rede com macro-células pode reduzir os custos da rede. Alguns prestadores de serviços iniciaram as suas soluções para ambientes de interior, mas ainda existem muitos desafios a ser ultrapassados. Atualizou-se o 5G air simulator para representar a implantação de femto-células de interior com os pressupostos propostos e distribuição espacial uniforme. Para todas as combinações possíveis do comprimento lado do apartamento, o número máximo de utilizadores suportado ultrapassou o número de utilizadores suportado (na literatura) em mais de duas vezes. Em ambientes de exterior, propuseram-se pico-células no interior de macro-células, de forma a obter atraso extremo-a-extremo reduzido e taxa de transmissão dados elevada, resultante do ganho de multiplexagem estatística associado. Apresentam-se resultados para as divisões funcionais seis e sete dos NR 5G, para 2.6 GHz, 3.5GHz e 5.62 GHz. Para raios das células curtos, a melhor solução será selecionar a banda dos 2.6 GHz para alcançar PLR (do inglês, PLR) reduzido e suportar um maior número de utilizadores, com débito binário útil e lucro mais elevados (para raios das células até 400 m). Em 4G, com CA, da análise do equilíbrio custos-proveitos com pico-células, o escalonamento multi-banda EMBS (do inglês, Enhanced Multi-band Scheduler) disponibiliza proveitos superiores em comparação com o caso sem CA. Mostra-se claramente que lucro com CA é mais de quatro vezes superior do que no cenário sem CA, o que significa que um aumento ligeiro no custo com CA resulta num aumento de 4-vezes no lucro relativamente ao cenário sem CA

Software Defined Applications in Cellular and Optical Networks

abstract: Small wireless cells have the potential to overcome bottlenecks in wireless access through the sharing of spectrum resources. A novel access backhaul network architecture based on a Smart Gateway (Sm-GW) between the small cell base stations, e.g., LTE eNBs, and the conventional backhaul gateways, e.g., LTE Servicing/Packet Gateways (S/P-GWs) has been introduced to address the bottleneck. The Sm-GW flexibly schedules uplink transmissions for the eNBs. Based on software defined networking (SDN) a management mechanism that allows multiple operator to flexibly inter-operate via multiple Sm-GWs with a multitude of small cells has been proposed. This dissertation also comprehensively survey the studies that examine the SDN paradigm in optical networks. Along with the PHY functional split improvements, the performance of Distributed Converged Cable Access Platform (DCCAP) in the cable architectures especially for the Remote-PHY and Remote-MACPHY nodes has been evaluated. In the PHY functional split, in addition to the re-use of infrastructure with a common FFT module for multiple technologies, a novel cross functional split interaction to cache the repetitive QAM symbols across time at the remote node to reduce the transmission rate requirement of the fronthaul link has been proposed.Dissertation/ThesisDoctoral Dissertation Electrical Engineering 201