A methodology for full-system power modeling in heterogeneous data centers

The need for energy-awareness in current data centers has encouraged the use of power modeling to estimate their power consumption. However, existing models present noticeable limitations, which make them application-dependent, platform-dependent, inaccurate, or computationally complex. In this paper, we propose a platform-and application-agnostic methodology for full-system power modeling in heterogeneous data centers that overcomes those limitations. It derives a single model per platform, which works with high accuracy for heterogeneous applications with different patterns of resource usage and energy consumption, by systematically selecting a minimum set of resource usage indicators and extracting complex relations among them that capture the impact on energy consumption of all the resources in the system. We demonstrate our methodology by generating power models for heterogeneous platforms with very different power consumption profiles. Our validation experiments with real Cloud applications show that such models provide high accuracy (around 5% of average estimation error).This work is supported by the Spanish Ministry of Economy and Competitiveness under contract TIN2015-65316-P, by the Gener- alitat de Catalunya under contract 2014-SGR-1051, and by the European Commission under FP7-SMARTCITIES-2013 contract 608679 (RenewIT) and FP7-ICT-2013-10 contracts 610874 (AS- CETiC) and 610456 (EuroServer).Peer ReviewedPostprint (author's final draft

Observing the clouds : a survey and taxonomy of cloud monitoring

This research was supported by a Royal Society Industry Fellowship and an Amazon Web Services (AWS) grant. Date of Acceptance: 10/12/2014Monitoring is an important aspect of designing and maintaining large-scale systems. Cloud computing presents a unique set of challenges to monitoring including: on-demand infrastructure, unprecedented scalability, rapid elasticity and performance uncertainty. There are a wide range of monitoring tools originating from cluster and high-performance computing, grid computing and enterprise computing, as well as a series of newer bespoke tools, which have been designed exclusively for cloud monitoring. These tools express a number of common elements and designs, which address the demands of cloud monitoring to various degrees. This paper performs an exhaustive survey of contemporary monitoring tools from which we derive a taxonomy, which examines how effectively existing tools and designs meet the challenges of cloud monitoring. We conclude by examining the socio-technical aspects of monitoring, and investigate the engineering challenges and practices behind implementing monitoring strategies for cloud computing.Publisher PDFPeer reviewe

Collaborative Traffic Measurement in Virtualized Data Center Networks

International audienceData center network monitoring can be carried out at hardware networking equipment (e.g. physical routers) and/or software networking equipment (e.g. virtual switches). While software switches offer high flexibility to deploy various monitoring tools, they have to utilize server resources, esp. CPU and memory, that can no longer be reserved fully to service users' traffic. In this paper we closely examine the costs of (i) sampling packets ; (ii) sending them to a user-space program for measurement; and (iii) forwarding them to a remote server where they will be processed in case of lack of resources locally. Starting from empirical observations, we derive an analytical model to accurately predict (R 2 = 99.5%) the three aforemen-tioned costs, as a function of the sampling rate. We next introduce a collaborative approach for traffic monitoring and sampling that maximizes the amount of collected traffic without impacting the data center's operation. We analyze, through numerical simulations, the performance of our collaborative solution. The results show that it is able to take advantage of the uneven loads on the servers to maximize the amount of traffic that can be sampled at the scale of a data center. The resulting gain reaches 200% compared to a non collaborative approach

On the Cost of Measuring Traffic in a Virtualized Environment

International audienceThe current trend in application development and deployment is to package applications and services within containers or virtual machines. This results in a blend of virtual and physical resources with complex network interconnection schemas mixing virtual and physical switches along with specific protocols to build virtual networks spanning over several servers. While the complexity of this setup is hidden by private/public cloud management solutions, e.g. OpenStack, this new environment constitutes a challenge when it comes to monitor and debug performance related issues. In this paper, we introduce the problem of measuring traffic in a virtualized environment and focus on one typical scenario, namely virtual machines interconnected with a virtual switch. For this scenario, we assess the cost of continuously measuring the network traffic activity of the machines. Specifically, we seek to estimate the competition that exists to access the physical resources (e.g., CPU) of the physical server between the measurement task and the legacy application activity

Can SDN Technology Be Transported to Software-Defined WSN/IoT?

© 2016 IEEE. Wireless sensor networks (WSNs) are essential elements of the Internet of Things ecosystem, as such, they encounter numerous IoT challenging architectural, management and application issues. These include inflexible control, manual configuration and management of sensor nodes, difficulty in an orchestration of resources, and virtualizing sensor network resources for on-demand applications and services. Addressing these issues presents a real challenge for WSNs and IoTs. By separating the network control plane from the data forwarding plane, Software-defined networking (SDN) has emerged as network technology that addresses similar problems of current switched-networks. Despite the differences between switched network and wireless sensor network domains, the SDN technology has a real potential to revolutionize WSNs/IoTs and address their challenging issues. However, very little has been attempted to bring the SDN paradigm to WSNs. This paper identifies weaknesses of existing research efforts that aims to bring the benefits of SDN to WSNs by mapping the control plane, the OpenFlow protocol, and the functionality between the two network domains. In particular, the paper investigates the difficulties and challenges in the development of software-defined wireless sensor networking (SDWSN). Finally, the paper proposes VSensor, SDIoT controller, SFlow components with specific and relevant functionality for an architecture of an SDWSN or SDIoT infrastructure

Enhanced IPFIX flow processing mechanism for overlay network monitoring

Cloud computing is an emerging technology. People are adopting cloud at a faster rate, due to this cloud network traffic is increasing at a pace which is challenging to manage. Monitoring tool is an essential aspect of cloud computing and becomes more apparent with the acquired of cloud services. Overlay network provides new path to converge network and run as an independent virtual network on top of physical network. Currently, cloud overlay network technologies in cloud infrastructure have visibility gaps, which mean cloud provider and consumers miss out the major performance issues for troubleshooting of overlay network traffic. Hence, to keep a close watch on network and catch potential problems, a network monitoring tool required, to track and report more in-depth for not only see the hidden traffic but also presents the related information of cloud overlay network technologies specifically suited to the modern cloud-scale data center. Therefore, this study proposes an enhanced IP Flow Information Export (IPFIX) mechanism for cloud overlay network monitoring by adopting flexible flow based technique. Furthermore, the solution provided in this research consist of diverse mechanisms: enhanced packet filtering mechanisms using property match filtering technique and hash-based filtering technique. Virtual Extensible Local Area Network (VXLAN) based flow classification mechanisms using 6-tuple flow pattern and adoptable flow patterns. IPFIX message template mechanisms, which is comprise set of fields for data records within the IPFIX flow processing system. The findings demonstrate that the proposed mechanism can capture multi-tenant overlay network traffic to identify, track, analyze and continuously monitor the performance of cloud overlay network services. The proposed mechanisms are resource efficient where the combination of VFMFM+6tuple+VXLAN Message consume 4.63% less CPU, while the combination of VHFM+AFCM+AFCM Message consume 11.45% less CPU than Standard IPFIX. The contributions of this study would help cloud network operators and end-users to quickly and proactively resolve any overlay network based on performance issues with end-to end visibility and actionable insights

Programmability and management of software-defined network infrastructures

In a landscape where software-based solutions are evermore central in the design, development and deployment of innovative solutions for communication networks, new challenges arise, related to how to best exploit the new solutions made available by technological advancements. The objective of this Thesis is to consolidate and improve some recent solutions for programmability, management, monitoring and provisioning in software-based infrastructures, as well as to propose new solutions for service deployment, management and monitoring over softwarized domains, along with working implementations, validating each point with punctual experimental validations and performance evaluations. The treatise starts by introducing the key concepts the research work is based upon, then the main research activities performed during the three years of PhD studies are presented. These include a high-level interface for network programmability over heterogeneous softwarized domains, an implementation of a protocol for service function chaining over non-programmable networks for multi-domain orchestration, a modular system for unified monitoring of softwarized infrastructures, a protocol for the employment of unused channels to augment the capabilities of the softwarized infrastructure, and a XaaS-aware orchestrator designed to operate over Fog computing scenarios

Enhanced IPFIX flow monitoring for VXLAN based cloud overlay networks

The demands for cloud computing services is rapidly growing due to its fast adoption and the migration of workloads from private data centers to cloud data centers. Many companies, small and large, prefer switching their data to the enterprise cloud environment rather than expanding their own data centers. As a result, the network traffic in cloud data centers is increasing rapidly. However, due to the dynamic resource provisioning and high-speed virtualized cloud networks, the traditional flow-monitoring systems is unable to provide detail visibility and information of traffic traversing the cloud overlay network environment. Hence, it does not fulfill the monitoring requirement of cloud overlay traffic. As the growth of cloud network traffic causes difficulties for the service providers and end-users to manage the traffic efficiently, an enhanced IPFIX flow monitoring mechanism for cloud overlay networks was proposed to address this problem. The monitoring mechanism provided detail visibility and information of overlay network traffic that traversed the cloud environment, which is not available in the current network monitoring systems. The experimental results showed that the proposed monitoring system able to capture overlay network traffic and segregated the tenant traffic based on virtual machines as compare to the standard monitoring system