Resurrection: Rethinking Magnetic Tapes For Cost Efficient Data Preservation

With the advent of Big Data technologies-the capacity to store and efficiently process large sets of data, doors of opportunities for developing business intelligence that was previously unknown, has opened. Each phase in the processing of this data requires specialized infrastructures. One such phase, the preservation and archiving of data, has proven its usefulness time and again. Data archives are processed using novel data mining methods to elicit vital data gathered over long periods of time and efficiently audit the growth of a business or an organization. Data preservation is also an important aspect of business processes which helps in avoiding loss of important information due to system failures, human errors and natural calamities. This thesis investigates the need, discusses possibilities and presents a novel, highly cost-effective, unified, long- term storage solution for data. Some of the common processes followed in large-scale data warehousing systems are analyzed for overlooked, inordinate shortcomings and a profitably feasible solution is conceived for them. The gap between the general needs of 'efficient' long-term storage and common, current functionalities is analyzed. An attempt to bridge this gap is made through the use of a hybrid, hierarchical media based, performance enhancing middleware and a monolithic namespace filesystem in a new storage architecture, Tape Cloud. The scope of studies carried out by us involves interpreting the effects of using heterogeneous storage media in terms of operational behavior, average latency of data transactions and power consumption. The results show the advantages of the new storage system by demonstrating the difference in operating costs, personnel costs and total cost of ownership from varied perspectives in a business model.Computer Science, Department o

Integrating Scale Out and Fault Tolerance in Stream Processing using Operator State Management

As users of big data applications expect fresh results, we witness a new breed of stream processing systems (SPS) that are designed to scale to large numbers of cloud-hosted machines. Such systems face new challenges: (i) to benefit from the pay-as-you-go model of cloud computing, they must scale out on demand, acquiring additional virtual machines (VMs) and parallelising operators when the workload increases; (ii) failures are common with deployments on hundreds of VMs - systems must be fault-tolerant with fast recovery times, yet low per-machine overheads. An open question is how to achieve these two goals when stream queries include stateful operators, which must be scaled out and recovered without affecting query results. Our key idea is to expose internal operator state explicitly to the SPS through a set of state management primitives. Based on them, we describe an integrated approach for dynamic scale out and recovery of stateful operators. Externalised operator state is checkpointed periodically by the SPS and backed up to upstream VMs. The SPS identifies individual operator bottlenecks and automatically scales them out by allocating new VMs and partitioning the check-pointed state. At any point, failed operators are recovered by restoring checkpointed state on a new VM and replaying unprocessed tuples. We evaluate this approach with the Linear Road Benchmark on the Amazon EC2 cloud platform and show that it can scale automatically to a load factor of L=350 with 50 VMs, while recovering quickly from failures. Copyright © 2013 ACM

Multi-Hop Wireless Networking with OSPF: MPR-based Routing Extensions for MANETs

Incorporating multi-hop wireless networks in the IP infrastructure is an effort to which a growing community participates. One instance of such activity is the extension of the routing protocol OSPF, for operation on MANETs. Such extension allows OSPF, the most widely deployed interior gateway routing protocol on the Internet, to work on heterogeneous networks encompassing both wired and wireless routers. The latter may self-organize as multi-hop wireless subnetworks, and may be mobile. Three solutions have been proposed for this extension, among which two based on techniques derived from multi-point relaying (MPR) techniques and OLSR. This paper analyzes these two approaches and identifies some fundamental discussion items that pertain to adapting OSPF mechanisms to multi-hop wireless networking, before concluding with a proposal for a unique, merged solution based on this analysis

Energy management in communication networks: a journey through modelling and optimization glasses

The widespread proliferation of Internet and wireless applications has produced a significant increase of ICT energy footprint. As a response, in the last five years, significant efforts have been undertaken to include energy-awareness into network management. Several green networking frameworks have been proposed by carefully managing the network routing and the power state of network devices. Even though approaches proposed differ based on network technologies and sleep modes of nodes and interfaces, they all aim at tailoring the active network resources to the varying traffic needs in order to minimize energy consumption. From a modeling point of view, this has several commonalities with classical network design and routing problems, even if with different objectives and in a dynamic context. With most researchers focused on addressing the complex and crucial technological aspects of green networking schemes, there has been so far little attention on understanding the modeling similarities and differences of proposed solutions. This paper fills the gap surveying the literature with optimization modeling glasses, following a tutorial approach that guides through the different components of the models with a unified symbolism. A detailed classification of the previous work based on the modeling issues included is also proposed

Logging and bookkeeping, Administrator's guide

Logging and Bookkeeping (LB for short) is a Grid service that keeps a short-term trace of Grid jobs as they are processed by individual Grid component

Optimal Protection Coordination of Active Distribution Networks Powered by Synchronverters

The integration of distributed generators (DGs) into distribution networks leads to the emergence of active distribution networks (ADNs). These networks have advantages, such as deferring the network upgrade, lower power losses, reduced power generation cost, and lower greenhouse gas emission, DGs are classified due to their interface with the network as inverter-interfaced or synchronous-interfaced. However, DGs integration results in bidirectional power flow, higher fault current levels, deterioration of the protection coordination of the directional overcurrent relays (DOCRs) which are used in ADNs, reduced system stability due to the inverters’ lack of damping. The stability can be enhanced by controlling the inverters to behave as synchronous generators, which are known as synchronverters. In this thesis, a two-stage optimal protection coordination (OPC) scheme is proposed to guarantee reliable protection of ADNs while protecting synchronverters from overcurrent using virtual impedance fault current limiters (VI-FCLs). VI-FCLs provide a cost-effective way to protect synchronverters from overcurrent. The first stage integrates the fault current calculations of synchronverters in the fault analysis to find the parameters of VI-FCLs used to limit the synchronverter’s fault current. In the second stage, the fault current calculations, along with the designed VI-FCLs from the first stage, are employed to determine the optimal relays’ settings to minimize the total operating times for all the DOCR. It is found that fixed VI-FCLs can limit synchronverters’ fault currents but may make the OPC problem infeasible to solve. Thus, an adaptive VI-FCL is proposed to ensure a feasible OPC under various fault conditions, i.e., locations and resistances

Sixth Goddard Conference on Mass Storage Systems and Technologies Held in Cooperation with the Fifteenth IEEE Symposium on Mass Storage Systems

This document contains copies of those technical papers received in time for publication prior to the Sixth Goddard Conference on Mass Storage Systems and Technologies which is being held in cooperation with the Fifteenth IEEE Symposium on Mass Storage Systems at the University of Maryland-University College Inn and Conference Center March 23-26, 1998. As one of an ongoing series, this Conference continues to provide a forum for discussion of issues relevant to the management of large volumes of data. The Conference encourages all interested organizations to discuss long term mass storage requirements and experiences in fielding solutions. Emphasis is on current and future practical solutions addressing issues in data management, storage systems and media, data acquisition, long term retention of data, and data distribution. This year's discussion topics include architecture, tape optimization, new technology, performance, standards, site reports, vendor solutions. Tutorials will be available on shared file systems, file system backups, data mining, and the dynamics of obsolescence