A Running Time Improvement for Two Thresholds Two Divisors Algorithm

Chunking algorithms play an important role in data de-duplication systems. The Basic Sliding Window (BSW) algorithm is the first prototype of the content-based chunking algorithm which can handle most types of data. The Two Thresholds Two Divisors (TTTD) algorithm was proposed to improve the BSW algorithm in terms of controlling the variations of the chunk-size. In this project, we investigate and compare the BSW algorithm and TTTD algorithm from different factors by a series of systematic experiments. Up to now, no paper conducts these experimental evaluations for these two algorithms. This is the first value of this paper. According to our analyses and the results of experiments, we provide a running time improvement for the TTTD algorithm. Our new solution reduces about 7 % of the total running time and also reduces about 50 % of the large-sized chunks while comparing with the original TTTD algorithm and make average chunk-size closer to the expected chunk-size. These significant results are the second important value of this project

A survey and classification of storage deduplication systems

The automatic elimination of duplicate data in a storage system commonly known as deduplication is increasingly accepted as an effective technique to reduce storage costs. Thus, it has been applied to different storage types, including archives and backups, primary storage, within solid state disks, and even to random access memory. Although the general approach to deduplication is shared by all storage types, each poses specific challenges and leads to different trade-offs and solutions. This diversity is often misunderstood, thus underestimating the relevance of new research and development. The first contribution of this paper is a classification of deduplication systems according to six criteria that correspond to key design decisions: granularity, locality, timing, indexing, technique, and scope. This classification identifies and describes the different approaches used for each of them. As a second contribution, we describe which combinations of these design decisions have been proposed and found more useful for challenges in each storage type. Finally, outstanding research challenges and unexplored design points are identified and discussed.This work is funded by the European Regional Development Fund (EDRF) through the COMPETE Programme (operational programme for competitiveness) and by National Funds through the Fundacao para a Ciencia e a Tecnologia (FCT; Portuguese Foundation for Science and Technology) within project RED FCOMP-01-0124-FEDER-010156 and the FCT by PhD scholarship SFRH-BD-71372-2010

Deduplicação segura em ambientes móveis

A utilização crescente de dispositivos móveis como smart phones, PDAs e tablets com capacidades de armazenamento e processamento a crescer rapidamente, permite aos utilizadores transportar um crescente volume de informação. O valor desta informação tem de ser protegido de perdas, roubos ou qualquer outro tipo de acidente que possa ocorrer com os dispositivos, tornando o processo de cópias de segurança um fator chave. Nos dispositivos móveis a transmissão de dados tem um elevado custo, em preço e energia, e estes dispositivos dependem de baterias, otimizar a utilização de largura de banda é crucial. A deduplicação permite uma redução substancial do volume de dados a serem transmitidos sobre a rede em sessões de cópias de segurança, identificando blocos de dados semelhantes, transferindo-os e armazenando-os apenas uma vez. Por outro lado, habitualmente os dispositivos móveis usam redes móveis, e o armazenamento na nuvem é uma solução viável para guardar dados de cópias de segurança. Por consequência é essencial a utilização de processos criptográficos para proteger a informação durante a transmissão, mas também no armazenamento. Neste trabalho avalia-se a viabilidade da utilização de dispositivos móveis no processamento de cópias de segurança, empregando a deduplicação e criptografia simultaneamente.An increased use of mobile devices such as smart phones, PDAs and tablet computers with storing and processing capacities is rapidly increasing ,allowing users to carry a bigger volume of data along with them. The value of this information must be protected from loss, theft or any kind of accident, which may occur with the devices, making the backup process a key factor. In what concerns mobile devices, the data transmission has a high cost, not only in price, but also in terms of energy. On the other hand these devices depend on batteries, so the optimization of the bandwidth use is crucial. Deduplication allows substantial reductions in the data volume to be transmitted over network on backup sessions, by identifying common amounts of data, transferring and storing them only once. On the other hand, mobile devices usually use mobile networks and cloud storage is a viable solution for data backup. Thus the use of cryptographic processes to protect data is essential, not only during the transmission, but also in what concerns storage. This work evaluates the viability of the use of mobile devices, in what deals with the processing of backup, by using deduplication and encryption, simultaneously

Cooperative Data Backup for Mobile Devices

Les dispositifs informatiques mobiles tels que les ordinateurs portables, assistants personnels et téléphones portables sont de plus en plus utilisés. Cependant, bien qu'ils soient utilisés dans des contextes où ils sont sujets à des endommagements, à la perte, voire au vol, peu de mécanismes permettent d'éviter la perte des données qui y sont stockées. Dans cette thèse, nous proposons un service de sauvegarde de données coopératif pour répondre à ce problème. Cette approche tire parti de communications spontanées entre de tels dispositifs, chaque dispositif stockant une partie des données des dispositifs rencontrés. Une étude analytique des gains de cette approche en termes de sûreté de fonctionnement est proposée. Nous étudions également des mécanismes de stockage réparti adaptés. Les problèmes de coopération entre individus mutuellement suspicieux sont également abordés. Enfin, nous décrivons notre mise en oeuvre du service de sauvegarde coopérative. ABSTRACT : Mobile devices such as laptops, PDAs and cell phones are increasingly relied on but are used in contexts that put them at risk of physical damage, loss or theft. However, few mechanisms are available to reduce the risk of losing the data stored on these devices. In this dissertation, we try to address this concern by designing a cooperative backup service for mobile devices. The service leverages encounters and spontaneous interactions among participating devices, such that each device stores data on behalf of other devices. We first provide an analytical evaluation of the dependability gains of the proposed service. Distributed storage mechanisms are explored and evaluated. Security concerns arising from thecooperation among mutually suspicious principals are identified, and core mechanisms are proposed to allow them to be addressed. Finally, we present our prototype implementation of the cooperative backup servic

Evaluation of Efficient Archival Storage Techniques

The ever-increasing volume of archival data that need to be retained for long periods of time has motivated the design of low-cost, high-efficiency storage systems. Inter-file compression has been proposed as a technique to improve storage efficiency by exploiting the high degree of similarity among archival data. We evaluate the two main inter-file compression techniques, data chunking and delta encoding, and compare them with traditional intra-file compression. We report on experimental results from a range of representative archival data sets