Move Fast and Meet Deadlines: Fine-grained Real-time Stream Processing with Cameo

Resource provisioning in multi-tenant stream processing systems faces the dual challenges of keeping resource utilization high (without over-provisioning), and ensuring performance isolation. In our common production use cases, where streaming workloads have to meet latency targets and avoid breaching service-level agreements, existing solutions are incapable of handling the wide variability of user needs. Our framework called Cameo uses fine-grained stream processing (inspired by actor computation models), and is able to provide high resource utilization while meeting latency targets. Cameo dynamically calculates and propagates priorities of events based on user latency targets and query semantics. Experiments on Microsoft Azure show that compared to state-of-the-art, the Cameo framework: i) reduces query latency by 2.7X in single tenant settings, ii) reduces query latency by 4.6X in multi-tenant scenarios, and iii) weathers transient spikes of workload

Intelligent Management and Efficient Operation of Big Data

This chapter details how Big Data can be used and implemented in networking and computing infrastructures. Specifically, it addresses three main aspects: the timely extraction of relevant knowledge from heterogeneous, and very often unstructured large data sources, the enhancement on the performance of processing and networking (cloud) infrastructures that are the most important foundational pillars of Big Data applications or services, and novel ways to efficiently manage network infrastructures with high-level composed policies for supporting the transmission of large amounts of data with distinct requisites (video vs. non-video). A case study involving an intelligent management solution to route data traffic with diverse requirements in a wide area Internet Exchange Point is presented, discussed in the context of Big Data, and evaluated.Comment: In book Handbook of Research on Trends and Future Directions in Big Data and Web Intelligence, IGI Global, 201

Modeling Tool of TRILL Protocol

This paper proposes the first modeling tool of TRILL protocol. The application implements TRILL features, which include the most fundamental concepts of IS-IS protocol. Simplified STP protocol is also implemented. The application can be used to didactically and interactively present the advantages of the TRILL protocol over STP. Alternatively, the tool can be utilized as a decision tool for TRILL deployment in enterprise networks. The functionality of the TRILL protocol is discussed with emphasis on the main differences between STP. The application was tested on five scenarios of TRILL deployment in a typical enterprise network. Data traffic of these different deployments was compared using the application. Finally, results are further discussed

Panopticon: Reaping the Benefits of Partial SDN Deployment in Enterprise Networks

The operational challenges posed in enterprise networks, present an appealing opportunity for the software-defined orchestration of the network (SDN). However, the primary challenge to realizing solutions built on SDN in the enterprise is the deployment problem. Unlike in the data-center, network upgrades in the enterprise start with the existing deployment and are budget and resource-constrained. In this work, we investigate the prospect for partial Software Defined Network (SDN) deployment. We present Panopticon, an architecture and methodology for planning and operating networks that combine legacy and upgraded SDN switches. Panopticon exposes an abstraction of a logical SDN in a partially upgraded legacy network, where the SDN benefits extend potentially over the entire network. We evaluate the feasibility of our approach through simulation on real enterprise campus network topologies entailing over 1500 switches and routers. Our results suggest that with only a handful of upgraded switches, it becomes possible to operate most of an enterprise network as a single SDN while meeting key resource constraints

Review and analysis of networking challenges in cloud computing

Cloud Computing offers virtualized computing, storage, and networking resources, over the Internet, to organizations and individual users in a completely dynamic way. These cloud resources are cheaper, easier to manage, and more elastic than sets of local, physical, ones. This encourages customers to outsource their applications and services to the cloud. The migration of both data and applications outside the administrative domain of customers into a shared environment imposes transversal, functional problems across distinct platforms and technologies. This article provides a contemporary discussion of the most relevant functional problems associated with the current evolution of Cloud Computing, mainly from the network perspective. The paper also gives a concise description of Cloud Computing concepts and technologies. It starts with a brief history about cloud computing, tracing its roots. Then, architectural models of cloud services are described, and the most relevant products for Cloud Computing are briefly discussed along with a comprehensive literature review. The paper highlights and analyzes the most pertinent and practical network issues of relevance to the provision of high-assurance cloud services through the Internet, including security. Finally, trends and future research directions are also presented

Designing Scalable Networks for Future Large Datacenters

Modern datacenters require a network with high cross-section bandwidth, fine-grained security, support for virtualization, and simple management that can scale to hundreds of thousands of hosts at low cost. This thesis first presents the firmware for Rain Man, a novel datacenter network architecture that meets these requirements, and then performs a general scalability study of the design space. The firmware for Rain Man, a scalable Software-Defined Networking architecture, employs novel algorithms and uses previously unused forwarding hardware. This allows Rain Man to scale at high performance to networks of forty thousand hosts on arbitrary network topologies. In the general scalability study of the design space of SDN architectures, this thesis identifies three different architectural dimensions common among the networks: source versus hop-by-hop routing, the granularity at which flows are routed, and arbitrary versus restrictive routing and finds that a source-routed, host-pair granularity network with arbitrary routes is the most scalable

Creating a Worldwide Network For the Global Environment for Network Innovations (GENI) and Related Experimental Environments

Many important societal activities are global in scope, and as these activities continually expand world-wide, they are increasingly based on a foundation of advanced communication services and underlying innovative network architecture, technology, and core infrastructure. To continue progress in these areas, research activities cannot be limited to campus labs and small local testbeds or even to national testbeds. Researchers must be able to explore concepts at scale—to conduct experiments on world-wide testbeds that approximate the attributes of the real world. Today, it is possible to take advantage of several macro information technology trends, especially virtualization and capabilities for programming technology resources at a highly granulated level, to design, implement and operate network research environments at a global scale. GENI is developing such an environment, as are research communities in a number of other countries. Recently, these communities have not only been investigating techniques for federating these research environments across multiple domains, but they have also been demonstration prototypes of such federations. This chapter provides an overview of key topics and experimental activities related to GENI international networking and to related projects throughout the world

Дослідження архітектури побудови центрів обробки даних на базі технології SDN

Текстова частина дипломної роботи: 121 с., 37 рис., 23 джерела, 4 додатки. Актуальність роботи. Потрібність в динамічному розподілі ресурсів, постійне збільшення трафіку, викликане зростанням мобільного доступу, попитом на відеосервіси, використанням хмарних обчислень призводить до того, що сучасні дата-центри повинні комплексно міняти свою ресурсну базу. Дата-центри повинні мати можливість обробляти нестаціонарний обсяг трафіку і здійснювати великий обсяг паралельних транзакцій, вміти глибоко аналізувати дані, бути легко переконфігурованими під мінливі запити. За умови обмеженості ресурсів, ЦОД не можуть нескінченно розростатися. Рішення більшості цих проблем було знайдено в віртуалізації, а саме у використанні технології програмно-визначених мереж (SDN - Software-Defined Networking), що була розроблена спільно університетами Берклі і Стенфорда у 2005 році. Мета роботи провести дослідження архітектури побудови центрів обробки даних на базі технології програмно-визначених мереж (SDN), проаналізувати референсну архітектуру SDN та референсну архітектуру сучасних центрів обробки даних, яку пропонують різні компанії-виробники, запропонувати програмну реалізацію переадресації (маршрутизації) трафіку в мережі на базі технології SDN. Об’єкт дослідження програмно-визначені мережі. Предмет дослідження використання програмно-визначених мереж в центрах обробки даних. Методи дослідження. Для досягнення поставлених в дисертаційній роботі задач використано методи теорії інформації, теорії оптимального управління, системного аналізу, елементи теорії ієрархічних багаторівневих систем. Наукова новизна отриманих результатів. Проведено дослідження архітектури побудови центрів обробки даних на базі технології програмно-визначених мереж (SDN) та проаналізовано референсну архітектуру SDN та референсну архітектуру сучасних центрів обробки даних, яку пропонують різні компанії-виробники. Крім цього представлена програмна реалізація переадресації трафіку на базі технології SDN. Публікації. Роговий В.П. «Загрози безпеці у SDN середовищах», Науково-технічна конференція "Проблеми телекомунікацій": Матеріали конференції. К.: НТУУ "КПІ", 2018. – с. 146-148;The work contains 121 pages, 37 figures, 4 applications. 23 sources have been used. Work actuality. The need for a dynamic distribution of resources, a constant increase in traffic due to the growth of mobile access, demand for video services, the use of cloud computing leads to the fact that modern datacenters must complexly change its resource base. Data centers should be able to process nonstationary traffic and implement a large amount of parallel transactions, be able to deeply analyze data, be easily reconfigured for changing requests. Also data centers can not infinitely grow due to limited resources. The solution to most of these problems was found in virtualization, namely the use of Software-Defined Networking (SDN) technology, developed jointly by the Berkeley and Stanford universities in 2005. The purpose of the work is to study the architecture of the building the data centers based on the technology of software-specific networks (SDN), to analyze the reference SDN architecture and the reference architecture of modern data centers that offers by various companies, to propose the software implementation of the redirection (forwarding) of traffic in the network based on SDN technology. Object of research - software-defined networks. Subject of research - using of software-defined networks in data centers. Research methods. To achieve the goals set in the dissertation methods of information theory, theory of optimal control, system analysis, elements of the theory of hierarchical multilevel systems were used. Scientific novelty of the obtained results. The study of the architecture of the building of data centers based on the technology of software-defined networks (SDN) was conducted, reference architecture of SDN and the reference architecture of modern data centers, which are offered by various manufacturers, were analyzed. In addition, the software implementation of traffic forwarding based on SDN technology was presented. Publications: Роговий В.П. «Загрози безпеці у SDN середовищах», Науково-технічна конференція "Проблеми телекомунікацій": Матеріали конференції. К.: НТУУ "КПІ", 2018. – p. 146-148