A Server Consolidation Solution

Advances in server architecture has enabled corporations the ability to strategically redesign their data centers in order to realign the system infrastructure to business needs. The architectural design of physically and logically consolidating servers into fewer and smaller hardware platforms can reduce data center overhead costs, while adding quality of service. In order for the organization to take advantage of the architectural opportunity a server consolidation project was proposed that utilized blade technology coupled with the virtualization of servers. Physical consolidation reduced the data center facility requirements, while server virtualization reduced the number of required hardware platforms. With the constant threat of outsourcing, coupled with the explosive growth of the organization, the IT managers were challenged to provide increased system services and functionality to a larger user community, while maintaining the same head count. A means of reducing overhead costs associated with the in-house data center was to reduce the required facility and hardware resources. The reduction in the data center footprint required less real estate, electricity, fire suppression infrastructure, and HVAC utilities. In addition, since the numerous stand alone servers were consolidated onto a standard platform system administration became more agile to business opportunities.

Switching considerations in storage networks.

by Leung Yiu Tong.Thesis (M.Phil.)--Chinese University of Hong Kong, 2003.Includes bibliographical references (leaves 96-98).Abstracts in English and Chinese.Chapter 1. --- Introduction --- p.1Chapter 1.1 --- Motivation --- p.1Chapter 1.2 --- Thesis Organization --- p.3Chapter 2. --- Storage Network Fundamentals --- p.4Chapter 2.1 --- Storage Network Topology --- p.4Chapter 2.1.1 --- Direct Attached Storage (DAS) --- p.5Chapter 2.1.2 --- Network Attached Storage (NAS) --- p.7Chapter 2.1.3 --- Storage Area Network (SAN) --- p.9Chapter 2.1.3.1 --- SAN and the Fibre Channel Protocol --- p.11Chapter 2.1.4 --- Summary on Storage Network Topology --- p.12Chapter 2.2 --- Storage Protocol --- p.15Chapter 2.2.1 --- Fibre Channel --- p.15Chapter 2.2.1.1 --- Fibre Channel over IP (FCIP) --- p.17Chapter 2.2.1.2 --- Internet Fibre Channel Protocol (iFCP) --- p.19Chapter 2.2.2 --- Internet SCSI (iSCSI) --- p.20Chapter 2.2.3 --- InfiniBand --- p.22Chapter 2.2.4 --- Review on Storage Network Protocol --- p.25Chapter 2.3 --- Standard Organization --- p.27Chapter 2.4 --- Summary --- p.28Chapter 3. --- Switching Design for Storage Networks --- p.30Chapter 3.1. --- Shared Bus Design --- p.32Chapter 3.2. --- Time Division Switch --- p.36Chapter 3.3. --- Share Buffer Memory Switch --- p.37Chapter 3.3.1 --- Parallel Memory Array --- p.40Chapter 3.3.2 --- Distributive Storage --- p.43Chapter 3.4. --- Crossbar Switch --- p.45Chapter 3.4.1 --- Arbitrated Crossbar vs. Buffered Crossbar --- p.46Chapter 3.4.1.1 --- Arbitrated Crossbar Switch --- p.47Chapter 3.4.1.2 --- Buffered Crossbar Switch --- p.48Chapter 3.4.2 --- Switch Scheduling --- p.49Chapter 3.4.2.1 --- Bipartite Matching --- p.50Chapter 3.4.2.2 --- Token-based Distributive Scheduling --- p.53Chapter 3.4.2.3 --- Resource Counting using Semaphore --- p.56Chapter 3.5. --- Algebraic Switches --- p.60Chapter 3.5.1 --- Switching by Conditionally Nonblocking Properties --- p.61Chapter 3.5.2 --- Self-Routing Mechanism with Zero-Bit Buffering --- p.64Chapter 3.5.3 --- Multistage Interconnection of Self-routing Concentrators --- p.69Chapter 3.6. --- Summary --- p.73Chapter 4. --- Investigating Switching Issue in Storage Networks --- p.74Chapter 4.1 --- Choosing a Suitable Switch --- p.74Chapter 4.2 --- Quality of Service (QoS) --- p.76Chapter 4.3 --- Multicasting --- p.77Chapter 4.3.1 --- Crossbar Switch --- p.78Chapter 4.3.2 --- Shared-Buffer Memory Switches --- p.80Chapter 4.3.3 --- Algebraic Switch --- p.82Chapter 4.3.4 --- Application on Multicast Transmission --- p.86Chapter 4.4 --- Load Balancing Mechanism --- p.87Chapter 4.5 --- Optimization on Storage Utilization --- p.91Chapter 4.6 --- Summary --- p.93Chapter 5. --- Conclusion and Summary of Original Contributions --- p.9

Storage Virtualization Promises Agility in the Data Center

Data storage and protection has moved to the forefront of Information Technology solutions because the business value of data has gained in rank and importance in the world of internet commerce. Modern business models are built around instant and continuous data availability and they would not be able to function without this quality. This level of data availability requires data storage technologies to be of increased flexibility and higher performance. However the more sophisticated technologies pose a greater challenge to the architects of data storage solutions who are required to evaluate products of much higher complexity and administrators who need to manage and monitor these installations. New tool sets are required to leverage the promise of the storage virtualization technologies and extract their full potential for an agile data center. New tool sets for storage virtualization will bring the IT organizations into the position of data service provider for the business groups

Convergencia de tecnologías ópticas y Ethernet en LAN, MAN y SAN: nuevas arquitecturas, análisis de prestaciones y eficiencia energética

Mención Internacional en el título de doctorThe development of Information Technologies in the last decades, especially the last two, together with the introduction of computing devices to the mainstream consumer market, has had the logical consequence of the generalisation of the Internet access. The explosive development of the smartphone market has brought ubiquity to that generalisation, to the point that social interaction, content sharing and content production happens all the time. Social networks have all but increased that trend, maximising the diffusion of multimedia content: images, audio and video, which require high network capacities to be enjoyed quickly. This need for endless bandwidth and speed in information sharing brings challenges that affect mainly optical Metropolitan Area Networks (MANs) and Wide Area Networks (WANs). Furthermore, the wide spreading of Ethernet technologies has also brought the possibility to achieve economies of scale by either extending the reach of Ethernet Local Area Networks (LANs) to the MAN and WAN environment or even integrating them with Storage Area Networks (SANs). Finally, this generalisation of telecommunication technologies in every day life has as a consequence an important rise in energy consumption as well. Because of this, providing energy efficient strategies in networking is key to ensure the scalability of the whole Internet. In this thesis, the main technologies in all the fields mentioned above are reviewed, its core challenges identified and several contributions beyond the state of the art are suggested to improve today’s MANs andWANs. In the first contribution of this thesism, the integration between Metro Ethernet and Wavelength Division Multiplexion (WDM) optical transparent rings is explored by proposing an adaptation architecture to provide efficient broadcast and multicast. The second contribution explores the fusion between transparent WDM and OCDMA architectures to simplify medium access in a ring. Regarding SANs, the third contribution explores the challenges in SANs through the problems of Fibre Channel over Ethernet due to buffer design issues. In this contribution, analysis, design and validation with FCoE traces and simulation is provided to calculate buffer overflow probabilities in the absence of flow control mechanisms taking into account the bursty nature of SAN traffic. Finally, the fourth and last contribution addresses the problems of energy efficiency in Plastic Optical Fibres (POF), a new kind of optical fibre more suitable for transmission in vehicles and for home networking. This contribution suggests two packet coalescing strategies to further improve the energy effiency mechanisms in POFs.El desarrollo de las Tecnologías de la Información en las últimas décadas, especialmente las últimas dos, junto con la introducción de dispositivos informáticos al mercado de masas, ha tenido como consecuencia lógica la generalización del acceso a Internet. El explosivo desarrollo del mercado de teléfonos inteligentes ha añadido un factor de ubicuidad a tal generalización, al extremo de que la interacción social, la compartición y producción de contenidos sucede a cada instante. Las redes sociales no han hecho sino incrementar tal tendencia, maximizando la difusión de contenido multimedia: imágenes, audio y vídeo, los cuales requieren gran capacidad en las redes para poder obtenerse con rapidez. Esta necesidad de ancho de banda ilimitado y velocidad en la compartición de información trae consigo retos que afectan principalmente a las Redes de Área Metropolitana (Metropolitan Area Networks, MANs) y Redes de Área Extensa (Wide Area Networks, WANs). Además, la gran difusión de las tecnologías Ethernet ha traído la posibilidad de alcanzar economías de escala bien extendiendo el alcance de Ethernet más allá de las Redes de Área Local (Local Area Networks, LANs) al entorno de las MAN y las WAN o incluso integrándolas con Redes de Almacenamiento (Storage Area Networks, SANs). Finalmente, esta generalización de las tecnologías de la comunicación en la vida cotidiana tiene también como consecuencia un importante aumento en el consumo de energía. Por tanto, desarrollar estrategias de transmisión en red eficientes energéticamente es clave para asegurar la escalabilidad de Internet. En esta tesis, las principales tecnologías de todos los campos mencionados arriba serán estudiadas, sus más importantes retos identificados y se sugieren varias contribuciones más allá del actual estado del arte para mejorar las actuales MANs y WANs. En la primera contribución de esta tesis, se explora la integración entre Metro Ethernet y anillos ópticos transparentes por Multiplexión en Longitud de Onda (Wavelength Division Multiplex, WDM) mediante la proposición de una arquitectura de adaptación para permitir la difusión y multidifusión eficiente. La segunda contribución explora la fusión entre las arquitecturas transparentes WDM y arquitecturas por Accesso Dividido Múltiple por Códigos Ópticos (OCDMA) para simplificar el acceso en una red en anillo. En lo referente a las SANs, la tercera contribución explora los retos en SANs a través de los problemas de Fibre Channel sobre Ethernet debido a los problemas en el diseño de búferes. En esta contribución, se provee un análisis, diseño y validación con trazas FCoE para calcular las probabilidades de desbordamiento de buffer en ausencia de mecanismos de control de flujo teniendo en cuenta la naturaleza rafagosa del tráfico de SAN. Finalmente, la cuarta y última contribución aborda los problemas de eficiencia energética en Fibras Ópticas Plásticas (POF), una nueva variedad de fibra óptica más adecuada para la transmisión en vehículos y para entornos de red caseros. Esta contribución sugiere dos estrategias de agrupamiento de paquetes para mejorar los mecanismos de eficiencia energética en POFs.Programa Oficial de Posgrado en Ingeniería TelemáticaPresidente: Luca Valcarenghi.- Secretario: Ignacio Soto Campos.- Vocal: Bas Huiszoo

Hyperscsi : Design and development of a new protocol for storage networking

Ph.DDOCTOR OF PHILOSOPH

Developing an In-kernel File Sharing Server Solution Based on Server Message Block protocol

Multi-device and multi-service smart environments make heavy use of the Internet and intra-net, thus constantly transferring and saving large amounts of digital data leading to an exponential data growth. This has led to the development of network storage systems such as Storage Area Networks and Network Attached Storage. Network Attached Storage provides a file system level access to data from storage elements that are connected to the network. One of the most widely used protocols in network storage systems, is the Server Message Block(SMB) protocol, that interconnects users from various operating systems such as Windows, Linux and Mac OS. Samba is a popular open-source user-space server that implements the SMB protocol. There have been a multitude of discussions about moving traditional user-space applications like web servers to the kernel-space in order to improve various aspects of the server like CPU utilization, memory utilization, memory footprint, context switching, etc. In this thesis, we have designed and implemented a server in the Linux kernel space. We discuss in detail, the features and functionalities of the newly implemented server. We provide an insight into why some of the design considerations were made, in order to improve the efficiency of protocol handling by the in-kernel file sharing server. We compare the performance of the user-space Samba solution with the in-kernel file sharing solution, implemented and discussed in this thesis, against different workloads to identify the competitiveness of the developed solution. We conclude by discussing what we learned, during the implementation process, along with some ideas for further improving the feature set and performance of the in-kernel server solution

A shared-disk parallel cluster file system

Dissertação apresentada para obtenção do Grau de Doutor em Informática Pela Universidade Nova de Lisboa, Faculdade de Ciências e TecnologiaToday, clusters are the de facto cost effective platform both for high performance computing (HPC) as well as IT environments. HPC and IT are quite different environments and differences include, among others, their choices on file systems and storage: HPC favours parallel file systems geared towards maximum I/O bandwidth, but which are not fully POSIX-compliant and were devised to run on top of (fault prone) partitioned storage; conversely, IT data centres favour both external disk arrays (to provide highly available storage) and POSIX compliant file systems, (either general purpose or shared-disk cluster file systems, CFSs). These specialised file systems do perform very well in their target environments provided that applications do not require some lateral features, e.g., no file locking on parallel file systems, and no high performance writes over cluster-wide shared files on CFSs. In brief, we can say that none of the above approaches solves the problem of providing high levels of reliability and performance to both worlds. Our pCFS proposal makes a contribution to change this situation: the rationale is to take advantage on the best of both – the reliability of cluster file systems and the high performance of parallel file systems. We don’t claim to provide the absolute best of each, but we aim at full POSIX compliance, a rich feature set, and levels of reliability and performance good enough for broad usage – e.g., traditional as well as HPC applications, support of clustered DBMS engines that may run over regular files, and video streaming. pCFS’ main ideas include: · Cooperative caching, a technique that has been used in file systems for distributed disks but, as far as we know, was never used either in SAN based cluster file systems or in parallel file systems. As a result, pCFS may use all infrastructures (LAN and SAN) to move data. · Fine-grain locking, whereby processes running across distinct nodes may define nonoverlapping byte-range regions in a file (instead of the whole file) and access them in parallel, reading and writing over those regions at the infrastructure’s full speed (provided that no major metadata changes are required). A prototype was built on top of GFS (a Red Hat shared disk CFS): GFS’ kernel code was slightly modified, and two kernel modules and a user-level daemon were added. In the prototype, fine grain locking is fully implemented and a cluster-wide coherent cache is maintained through data (page fragments) movement over the LAN. Our benchmarks for non-overlapping writers over a single file shared among processes running on different nodes show that pCFS’ bandwidth is 2 times greater than NFS’ while being comparable to that of the Parallel Virtual File System (PVFS), both requiring about 10 times more CPU. And pCFS’ bandwidth also surpasses GFS’ (600 times for small record sizes, e.g., 4 KB, decreasing down to 2 times for large record sizes, e.g., 4 MB), at about the same CPU usage.Lusitania, Companhia de Seguros S.A, Programa IBM Shared University Research (SUR

Managing the Cost of Usable Data Centers

The main topic of this paper is to identify problems and present an overview of Data Center environments. To identify problems and present the overviews of business data environments and the cost of usable data center for small-midsize business organization based type of requirements on the design is one of the most important concepts of managing cost. To maximized data center efficiency administrators implement Blade Server, Virtualization, SOA, and other recent technologies. The project process will focus on most leased data centers with provided space rather than specific applications that trend the way of design, and eliminating the significant impact of multiple physical storage devices. Data Centers are complex systems with a variety of technologies that require constantly evolving skills and knowledge that range from routing and switching to load balancing and security. This project will include research, collecting sources, discussing the issues associated with network attacks Data Centers, and reviewing the other key areas related to data center development will be cover the way server availability will describes how to design a highly available infrastructure, and describes how a load balancing device can monitor the availability of applications and servers

Impact of physical layer impairments on multi-band metro networks

The traffic increase in optical metro networks will saturate the network capacity in a near future, mainly due to new cloud and 5G services, as well as to an increasing number of network users. The use of other fiber bands, than the usual C-band, is seen as a possible near term solution, for this probable capacity crunch. In this work, a metro network horseshoe topology with nodes capable of switching both C and L-band signals is studied. In particular, we have considered for switching Cband signals, reconfigurable optical add/drop multiplexer and Filterless Drop and Waste (FD&W) node architectures, whereas for switching L-band signals, we have considered only FD&W solutions, both amplified and unamplified. An analytical formalism was developed to assess the impact of the physical layer impairments due to fiber transmission and switching node in a horseshoe network performance. Moreover, the cost and power consumption of the C-band and L-band nodes are analyzed. We concluded that in a network end of life scenario, the L-band nodes cost is 3.5 times higher than the C-band nodes, being 99% of this cost attributed to the transponders. In L-band transmission, a lightpath can cross several spans with 10 or 60 km in the amplified solution. In the unamplified solution, where frequency reuse is allowed, a lightpath consists only on a single 10 km span due to the maximum optical link budget. Considering a -20 dB crosstalk level, a 1.5 dB degradation on the optical power budget is observed.O aumento de tráfego nas redes metropolitanas irá saturar a capacidade das redes num futuro próximo, principalmente devido aos novos serviços em nuvem e 5G, bem como ao aumento do número de utilizadores. O uso de novas bandas na fibra ótica, além da banda C, é visto como uma possível solução para responder a esse aumento de capacidade. Neste trabalho, é apresentada a topologia de rede ”horseshoe”, que contém nós com capacidade para comutar sinais nas bandas C e L. Na banda C, consideram-se as arquiteturas com multiplexadores de inserção/extração reconfiguráveis e arquiteturas sem filtragem, e na banda L, consideram-se apenas soluções sem filtragem, com e sem amplificação. São estudadas, analiticamente, as limitações físicas da transmissão na fibra e comutação dos sinais nos nós da rede ”horseshoe”. Além disso, é realizado um estudo do custo, consumo e capacidade dos nós nas bandas C e L. Num cenário de fim de vida da rede, o custo dos nós na banda L é 3.5 vezes superior aos da banda C. O custo dos transponders constitui 99% do custo final. Na banda L, o sinal pode atravessar várias secções de 10 ou 60 km na solução amplificada. Na solução não amplificada, onde existe reuso de frequência, só há uma secção de 10 km devido ao orçamento máximo de potência na ligação. Para um nível de diafonia de -20 dB, o orçamento de potência diminui 1.5 dB