SDN Enabled Network Efficient Data Regeneration for Distributed Storage Systems

Distributed Storage Systems (DSSs) have seen increasing levels of deployment in data centers and in cloud storage networks. DSS provides efficient and cost-effective ways to store large amount of data. To ensure reliability and resilience to failures, DSS employ mirroring and coding schemes at the block and file level. While mirroring techniques provide an efficient way to recover lost data, they do not utilize disk space efficiently, resulting in large overheads in terms of data storage. Coding techniques on the other hand provide a better way to recover data as they reduce the amount of storage space required for data recovery purposes. However, the current recovery process for coded data is not efficient due to the need to transfer large amounts of data to regenerate the data lost as a result of a failure. This results in significant delays and excessive network traffic resulting in a major performance bottleneck. In this thesis, we propose a new architecture for efficient data regeneration in distribution storage systems. A key idea of our architecture is to enable network switches to perform network coding operations, i.e., combine packets they receive over incoming links and forward the resulting packet towards the destination and do this in a principled manner. Another key element of our framework is a transport-layer reverse multicast protocol that takes advantage of network coding to minimize the rebuild time required to transmit the data by allowing more efficient utilization of network bandwidth. The new architecture is supported using the principles of Software Defined Networking (SDN) and making extensions where required in a principled manner. To enable the switches to perform network coding operations, we propose an extension of packet processing pipeline in the dataplane of a software switch. Our testbed experiments show that the proposed architecture results in modest performance gains

On Utilization of Contributory Storage in Desktop Grids

The availability of desktop grids and shared computing platforms has popularized the use of contributory resources, such as desktops, as computing substrates for a variety of applications. However, addressing the exponentially growing storage demands of applications, especially in a contributory environment, remains a challenging research problem. In this report, we propose a transparent distributed storage system that harnesses the storage contributed by grid participants arranged in a peer-to-peer network to yield a scalable, robust, and self-organizing system. The novelty of our work lies in (i) design simplicity to facilitate actual use; (ii) support for easy integration with grid platforms; (iii) ingenious use of striping and error coding techniques to support very large data files; and (iv) the use of multicast techniques for data replication. Experimental results through simulations and an actual implementation show that our system can provide reliable and efficient storage with large file support for desktop grid applications

Optimization of the composition of crop collections for ex situ conservation

Many crop genetic resources collections have been established without a clearly defined conservation goal or mandate, which has resulted in collections of considerable size, unbalanced composition and high levels of duplication. Attempts to improve the composition of collections are hampered by the fact that conceptual views to optimize collection composition are very rare. An optimization strategy is proposed herein, which largely builds on the concepts of core collection and core selection. The proposed strategy relies on hierarchically structuring the crop gene pool and assigning a relative importance to each of its different components. Comparison of the resulting optimized distribution of the number of accessions with the actual distribution allows identification of under- and over-representation within a collection. Application of this strategy is illustrated by an example using potato. The proposed optimization strategy is applicable not only to individual genebanks, but also to consortia of cooperating genebanks, which makes it relevant for ongoing activities within projects that aim at sharing responsibilities among institutions on the basis of rational conservation, such as a European genebank integrated system and the global cacao genetic resources network CacaoNet