Performance evaluation of an open distributed platform for realistic traffic generation

Network researchers have dedicated a notable part of their efforts to the area of modeling traffic and to the implementation of efficient traffic generators. We feel that there is a strong demand for traffic generators capable to reproduce realistic traffic patterns according to theoretical models and at the same time with high performance. This work presents an open distributed platform for traffic generation that we called distributed internet traffic generator (D-ITG), capable of producing traffic (network, transport and application layer) at packet level and of accurately replicating appropriate stochastic processes for both inter departure time (IDT) and packet size (PS) random variables. We implemented two different versions of our distributed generator. In the first one, a log server is in charge of recording the information transmitted by senders and receivers and these communications are based either on TCP or UDP. In the other one, senders and receivers make use of the MPI library. In this work a complete performance comparison among the centralized version and the two distributed versions of D-ITG is presented

Experimental evaluation of ZigBee and IEEE 802.15.4 for data-intensive body sensor networks

This paper presents results concerning an experimental performance evaluation of ZigBee and IEEE 802.15.4 networks applied to the transport of data-intensive traffic generated by body sensor network applications. The experimental platform is based on the Z-Stack and TIMAC software stacks and the CC2530 device, from Texas Instruments. Three quality of service metrics are evaluated: goodput, delivery ratio and delay. Results are provided for both star and tree topologies. It was observed that the overhead introduced by the stack implementation has a significant impact on the performance results. Overall, the performance of the ZigBee star topology was very good, even in the worst conditions, provided the acknowledgement mechanism was enabled. A router deadlock problem detected in other ZigBee implementations was not observed with the Z-Stack. However, we identified two different situations, triggered by periods of high traffic load, on which the ZigBee router stops relaying packets, causing a significant degradation on the network performance.Fundação para a Ciência e a Tecnologia (FCT

Network Vortex: Distributed Virtual Memory for Streaming Applications

Explosive growth of the Internet, cluster computing, and storage technology has led to generation of enormous volumes of information and the need for scalable data computing. One of the central frameworks for such analysis is MapReduce, which is a programming platform for processing streaming data in external/distributed memory. Despite a significant public effort, open-source implementations of MapReduce (e.g., Hadoop, Spark) are complicated, bulky, and inefficient. To overcome this problem, we explore employing and expanding upon a recent a C/C++ programming abstraction called Vortex that offers a simple interface to the user, zero-copy operation, low RAM consumption, and high data throughput. In particular, this research examines algorithms and techniques for enabling Vortex operation over the network, including both TCP/IP sockets and data-link RDMA (e.g., InfiniBand) interfaces. We developed a new producer-consumer memory stream abstraction presented as a Vortex stream split across two hosts, travelling through a hidden network communication layer to provide the illusion of writing a continuous stream of data directly into a window of memory on a remote machine, thereby enabling the creation of high-performance networking code and size-agnostic data transport under appropriate circumstances written as simply as an in-memory copy operation, overcoming complications normally inherent in the discrete nature of network packet transfer. While the resulting product is highly workable over standard IP-based internet networks, the design limitations of RDMA technology in interfacing with virtual memory prove to make Vortex streams a suboptimal abstraction for this programming platform, as its central appeal of zero-copy network transfers are rendered largely inaccessible. Alternative algorithms to enhance RDMA performance with Vortex are proposed for future study

ALiCE: A Java-based Grid Computing System

A computational grid is a hardware and software infrastructure that provides dependable, consistent, pervasive, and inexpensive access to high-end computational capabilities. This talk is divided into three parts. Firstly, we give an overview of the main issues in grid computing. Next, we introduce ALiCE (Adaptive and Scalable Internet-based Computing Engine), a platform independent and lightweight grid. ALiCE exploits object-level parallelism using our Object Network Transport Architecture (ONTA). Grid applications are written using ALiCE Object Programming Template that hides the complexities of the underlying grid fabric. Lastly, we present some performance results of ALiCE applications including the geo-rectification of satellite images and the progressive multiple sequence alignments problem.Singapore-MIT Alliance (SMA

Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models

Deep learning recommendation models (DLRMs) are used across many business-critical services at Facebook and are the single largest AI application in terms of infrastructure demand in its data-centers. In this paper we discuss the SW/HW co-designed solution for high-performance distributed training of large-scale DLRMs. We introduce a high-performance scalable software stack based on PyTorch and pair it with the new evolution of Zion platform, namely ZionEX. We demonstrate the capability to train very large DLRMs with up to 12 Trillion parameters and show that we can attain 40X speedup in terms of time to solution over previous systems. We achieve this by (i) designing the ZionEX platform with dedicated scale-out network, provisioned with high bandwidth, optimal topology and efficient transport (ii) implementing an optimized PyTorch-based training stack supporting both model and data parallelism (iii) developing sharding algorithms capable of hierarchical partitioning of the embedding tables along row, column dimensions and load balancing them across multiple workers; (iv) adding high-performance core operators while retaining flexibility to support optimizers with fully deterministic updates (v) leveraging reduced precision communications, multi-level memory hierarchy (HBM+DDR+SSD) and pipelining. Furthermore, we develop and briefly comment on distributed data ingestion and other supporting services that are required for the robust and efficient end-to-end training in production environments

A two-way interactive broadband satellite architecture to break the digital divide barrier

September 24-26, 2007, Turin, Ital

TV-Centric technologies to provide remote areas with two-way satellite broadband access

October 1-2, 2007, Rome, Italy TV-Centric Technologies To Provide Remote Areas With Two-Way Satellite Broadband Acces

IREEL: remote experimentation with real protocols and applications over emulated network

This paper presents a novel e-learning platform called IREEL. IREEL is a virtual laboratory allowing students to drive experiments with real Internet applications and end-to-end protocols in the context of networking courses. This platform consists in a remote network emulator offering a set of predefined applications and protocol mechanisms. Experimenters configure and control the emulation and the end-systems behavior in order to perform tests, measurements and observations on protocols or applications operating under controlled specific networking conditions. A set of end-to-end mechanisms, mainly focusing on transport and application level protocols, are currently available. IREEL is scalable and easy to use thanks to an ergonomic web interface