JTP, an energy-aware transport protocol for mobile ad hoc networks (PhD thesis)

Wireless ad-hoc networks are based on a cooperative communication model, where all nodes not only generate traffic but also help to route traffic from other nodes to its final destination. In such an environment where there is no infrastructure support the lifetime of the network is tightly coupled with the lifetime of individual nodes. Most of the devices that form such networks are battery-operated, and thus it becomes important to conserve energy so as to maximize the lifetime of a node. In this thesis, we present JTP, a new energy-aware transport protocol, whose goal is to reduce power consumption without compromising delivery requirements of applications. JTP has been implemented within the JAVeLEN system. JAVeLEN [RKM+08], is a new system architecture for ad hoc networks that has been developed to elevate energy efficiency as a first-class optimization metric at all protocol layers, from physical to transport. Thus, energy gains obtained in one layer would not be offset by incompatibilities and/or inefficiencies in other layers. To meet its goal of energy efficiency, JTP (1) contains mechanisms to balance end-toend vs. local retransmissions; (2) minimizes acknowledgment traffic using receiver regulated rate-based flow control combined with selected acknowledgments and in-network caching of packets; and (3) aggressively seeks to avoid any congestion-based packet loss. Within this ultra low-power multi-hop wireless network system, simulations and experimental results demonstrate that our transport protocol meets its goal of preserving the energy efficiency of the underlying network. JTP has been implemented on the actual JAVeLEN nodes and its benefits have been demonstrated on a real system

JTP, an energy-aware transport protocol for mobile ad hoc networks

Wireless ad-hoc networks are based on a cooperative communication model, where all nodes not only generate traffic but also help to route traffic from other nodes to its final destination. In such an environment where there is no infrastructure support the lifetime of the network is tightly coupled with the lifetime of individual nodes. Most of the devices that form such networks are battery-operated, and thus it becomes important to conserve energy so as to maximize the lifetime of a node. In this thesis, we present JTP, a new energy-aware transport protocol, whose goal is to reduce power consumption without compromising delivery requirements of applications. JTP has been implemented within the JAVeLEN system. JAVeLEN~\cite{javelen08redi}, is a new system architecture for ad hoc networks that has been developed to elevate energy efficiency as a first-class optimization metric at all protocol layers, from physical to transport. Thus, energy gains obtained in one layer would not be offset by incompatibilities and/or inefficiencies in other layers. To meet its goal of energy efficiency, JTP (1) contains mechanisms to balance end-to-end vs. local retransmissions; (2) minimizes acknowledgment traffic using receiver regulated rate-based flow control combined with selected acknowledgments and in-network caching of packets; and (3) aggressively seeks to avoid any congestion-based packet loss. Within this ultra low-power multi-hop wireless network system, simulations and experimental results demonstrate that our transport protocol meets its goal of preserving the energy efficiency of the underlying network. JTP has been implemented on the actual JAVeLEN nodes and its benefits have been demoed on a real system

A network transparent, retained mode multimedia processing framework for the Linux operating system environment

Die Arbeit präsentiert ein Multimedia-Framework für Linux, das im Unterschied zu früheren Arbeiten auf den Ideen "retained-mode processing" und "lazy evaluation" basiert: Statt Transformationen unmittelbar auszuführen, wird eine abstrakte Repräsentation aller Medienelemente aufgebaut. "renderer"-Treiber fungieren als Übersetzer, die diese Darstellung zur Laufzeit in konkrete Operationen umsetzen, wobei das Datenmodell zahlreiche Optimierungen zur Reduktion der Anzahl der Schritte oder der Minimierung von Kommunikation erlaubt. Dies erlaubt ein stark vereinfachtes Programmiermodell bei gleichzeitiger Effizienzsteigerung. "renderer"-Treiber können zur Ausführung von Transformationen den lokalen Prozessor verwenden, oder können die Operationen delegieren. In der Arbeit wird eine Erweiterung des X Window Systems um Mechanismen zur Medienverarbeitung vorgestellt, sowie ein "renderer"-Treiber, der diese zur Delegation der Verarbeitung nutzt

Improving Content Delivery and Service Discovery in Networks

Production and consumption of multimedia content on the Internet is rising, fueled by the demand for content from services such as YouTube, Netflix and Facebook video. The Internet is shifting from host-based to content-centric networking. At the same time, users are shifting away from a homogeneous desktop computing environment to using a heterogeneous mix of devices, such as smartphones, tablets and thin clients, all of which allow users to consume data on the move using wireless and cellular data networks. The popularity of these new class of devices has, in turn, increased demand for multimedia content by mobile users. The emergence of rich Internet applications and the widespread adoption and use of High Definition (HD) video has also placed higher pressure on the service providers and the core Internet backbone, forcing service providers to respond to increased bandwidth use in such networks. In my thesis, I aim to provide clarity and insight into the usage of core networking protocols and multimedia consumption on both mobile and wireless networks, as well as the network core. I also present research prototypes for potential solutions to some of the problems caused by the increased multimedia consumption on the Internet

Design and Optimization of Mobile Cloud Computing Systems with Networked Virtual Platforms

A Mobile Cloud Computing (MCC) system is a cloud-based system that is accessed by the users through their own mobile devices. MCC systems are emerging as the product of two technology trends: 1) the migration of personal computing from desktop to mobile devices and 2) the growing integration of large-scale computing environments into cloud systems. Designers are developing a variety of new mobile cloud computing systems. Each of these systems is developed with different goals and under the influence of different design constraints, such as high network latency or limited energy supply. The current MCC systems rely heavily on Computation Offloading, which however incurs new problems such as scalability of the cloud, privacy concerns due to storing personal information on the cloud, and high energy consumption on the cloud data centers. In this dissertation, I address these problems by exploring different options in the distribution of computation across different computing nodes in MCC systems. My thesis is that "the use of design and simulation tools optimized for design space exploration of the MCC systems is the key to optimize the distribution of computation in MCC." For a quantitative analysis of mobile cloud computing systems through design space exploration, I have developed netShip, the first generation of an innovative design and simulation tool, that offers large scalability and heterogeneity support. With this tool system designers and software programmers can efficiently develop, optimize, and validate large-scale, heterogeneous MCC systems. I have enhanced netShip to support the development of ever-evolving MCC applications with a variety of emerging needs including the fast simulation of new devices, e.g., Internet-of-Things devices, and accelerators, e.g., mobile GPUs. Leveraging netShip, I developed three new MCC systems where I applied three variations of a new computation distributing technique, called Reverse Offloading. By more actively leveraging the computational power on mobile devices, the MCC systems can reduce the total execution times, the burden of concentrated computations on the cloud, and the privacy concerns about storing personal information available in the cloud. This approach also creates opportunities for new services by utilizing the information available on the mobile device instead of accessing the cloud. Throughout my research I have enabled the design optimization of mobile applications and cloud-computing platforms. In particular, my design tool for MCC systems becomes a vehicle to optimize not only the performance but also the energy dissipation, an aspect of critical importance for any computing system

A shared-disk parallel cluster file system

Dissertação apresentada para obtenção do Grau de Doutor em Informática Pela Universidade Nova de Lisboa, Faculdade de Ciências e TecnologiaToday, clusters are the de facto cost effective platform both for high performance computing (HPC) as well as IT environments. HPC and IT are quite different environments and differences include, among others, their choices on file systems and storage: HPC favours parallel file systems geared towards maximum I/O bandwidth, but which are not fully POSIX-compliant and were devised to run on top of (fault prone) partitioned storage; conversely, IT data centres favour both external disk arrays (to provide highly available storage) and POSIX compliant file systems, (either general purpose or shared-disk cluster file systems, CFSs). These specialised file systems do perform very well in their target environments provided that applications do not require some lateral features, e.g., no file locking on parallel file systems, and no high performance writes over cluster-wide shared files on CFSs. In brief, we can say that none of the above approaches solves the problem of providing high levels of reliability and performance to both worlds. Our pCFS proposal makes a contribution to change this situation: the rationale is to take advantage on the best of both – the reliability of cluster file systems and the high performance of parallel file systems. We don’t claim to provide the absolute best of each, but we aim at full POSIX compliance, a rich feature set, and levels of reliability and performance good enough for broad usage – e.g., traditional as well as HPC applications, support of clustered DBMS engines that may run over regular files, and video streaming. pCFS’ main ideas include: · Cooperative caching, a technique that has been used in file systems for distributed disks but, as far as we know, was never used either in SAN based cluster file systems or in parallel file systems. As a result, pCFS may use all infrastructures (LAN and SAN) to move data. · Fine-grain locking, whereby processes running across distinct nodes may define nonoverlapping byte-range regions in a file (instead of the whole file) and access them in parallel, reading and writing over those regions at the infrastructure’s full speed (provided that no major metadata changes are required). A prototype was built on top of GFS (a Red Hat shared disk CFS): GFS’ kernel code was slightly modified, and two kernel modules and a user-level daemon were added. In the prototype, fine grain locking is fully implemented and a cluster-wide coherent cache is maintained through data (page fragments) movement over the LAN. Our benchmarks for non-overlapping writers over a single file shared among processes running on different nodes show that pCFS’ bandwidth is 2 times greater than NFS’ while being comparable to that of the Parallel Virtual File System (PVFS), both requiring about 10 times more CPU. And pCFS’ bandwidth also surpasses GFS’ (600 times for small record sizes, e.g., 4 KB, decreasing down to 2 times for large record sizes, e.g., 4 MB), at about the same CPU usage.Lusitania, Companhia de Seguros S.A, Programa IBM Shared University Research (SUR

The design and implementation of a continuous network file system

This thesis presents the design and implementation of Continuous Network File System (CNFS), a file system that provides a framework for developing efficient multimedia servers using Linux operating system. CNFS uses a modular approach for designing multimedia file systems utilizing a user-level technique and off-the-shelf commercial products. Multimedia traffic smoothing techniques and real-time issues are explored for the real implementation work. At client-side, CNFS chooses Userfs, a kernel file system skeleton, as its low-level IPC mechanism. The Userfs allows a file system be implemented in user processes. Thus, the whole development environment is more manageable for future research. At server side, KURT, a real-time extension of Linux, is used to provide QoS support for I/O operations. The technique for using KURT for multimedia system is discussed in detail. A complete prototype and functional system using CNFS is built and the process is described in the thesis. Initial performance measurements of the prototype implementation show CNFS capable of achieving real-time delivery of MPEG-1 file under reasonable system loads

A real-time networked camera system:a scheduled distributed camera system reduces the latency

This report presents the results of a Real-time Networked Camera System, com-missioned by the SAN Group in TU/e. Distributed Systems are motivated by two reasons, the first reason is the physical environment as a requirement and the second reason is to provide a better Quality of Service (QoS). This project describes the distributed system with a video processing application. The aim is to deal with the distributed system as one system thus minimizing delays while keeping the predictability in a real-time context. Time is the most crucial ingredient for the real-time systems in the sense that the tasks within the application should meet with the task deadline. With respect to the distributed system we need to consider a couple of issues. The first one is to have a distributed system and a modular application that is mapped to multiple system nodes. The second issue is to schedule the modules collectively and the third is to propose a solution when shared resource(s) (such as the network) are required by several nodes at the same time. In order to provide a distributed system, we connect 2 cameras with 1 PC via a network switch. Video processing has two parts; the first part consists of creating a frame, encoding the frame, and streaming it to the network and the second part deals with receiving the frame, decoding the frame, and displaying the frame. The first part is running on the cameras and the second part is running on the PC. In order to give real-time behavior to the system, the system components should provide the real-time behavior. The camera is installed with the µC/OS-II (Open Source Real-time Kernel). We investigated the Real-time Operating System and its installation on the PC. In order to provide resource management to the shared resources, we designed and implemented Admission control which controls access to the required con-nection to the PC. We designed and implemented a component to delay the start of any of the cameras in order to synchronize the network utilization. We also designed an enforcement component to allow the tasks to run as much as they should and monitor the frames streamed to the network. The results show that with the Admission Control, cameras only send as many frames as the network can transport. The given start delay to the system shows that overlap can be prevented, but we could not evaluate it because of the semi-tested/unreleased code which is provided by the camera providers. The source code we used is the test source code which was not mature

A storage architecture for data-intensive computing

The assimilation of computing into our daily lives is enabling the generation of data at unprecedented rates. In 2008, IDC estimated that the "digital universe" contained 486 exabytes of data [9]. The computing industry is being challenged to develop methods for the cost-effective processing of data at these large scales. The MapReduce programming model has emerged as a scalable way to perform data-intensive computations on commodity cluster computers. Hadoop is a popular open-source implementation of MapReduce. To manage storage resources across the cluster, Hadoop uses a distributed user-level filesystem. This filesystem --- HDFS --- is written in Java and designed for portability across heterogeneous hardware and software platforms. The efficiency of a Hadoop cluster depends heavily on the performance of this underlying storage system. This thesis is the first to analyze the interactions between Hadoop and storage. It describes how the user-level Hadoop filesystem, instead of efficiently capturing the full performance potential of the underlying cluster hardware, actually degrades application performance significantly. Architectural bottlenecks in the Hadoop implementation result in inefficient HDFS usage due to delays in scheduling new MapReduce tasks. Further, HDFS implicitly makes assumptions about how the underlying native platform manages storage resources, even though native filesystems and I/O schedulers vary widely in design and behavior. Methods to eliminate these bottlenecks in HDFS are proposed and evaluated both in terms of their application performance improvement and impact on the portability of the Hadoop framework. In addition to improving the performance and efficiency of the Hadoop storage system, this thesis also focuses on improving its flexibility. The goal is to allow Hadoop to coexist in cluster computers shared with a variety of other applications through the use of virtualization technology. The introduction of virtualization breaks the traditional Hadoop storage architecture, where persistent HDFS data is stored on local disks installed directly in the computation nodes. To overcome this challenge, a new flexible network-based storage architecture is proposed, along with changes to the HDFS framework. Network-based storage enables Hadoop to operate efficiently in a dynamic virtualized environment and furthers the spread of the MapReduce parallel programming model to new applications