333 research outputs found
Secure and Private Cloud Storage Systems with Random Linear Fountain Codes
An information theoretic approach to security and privacy called Secure And
Private Information Retrieval (SAPIR) is introduced. SAPIR is applied to
distributed data storage systems. In this approach, random combinations of all
contents are stored across the network. Our coding approach is based on Random
Linear Fountain (RLF) codes. To retrieve a content, a group of servers
collaborate with each other to form a Reconstruction Group (RG). SAPIR achieves
asymptotic perfect secrecy if at least one of the servers within an RG is not
compromised. Further, a Private Information Retrieval (PIR) scheme based on
random queries is proposed. The PIR approach ensures the users privately
download their desired contents without the servers knowing about the requested
contents indices. The proposed scheme is adaptive and can provide privacy
against a significant number of colluding servers.Comment: 8 pages, 2 figure
Babel Storage: Uncoordinated Content Delivery from Multiple Coded Storage Systems
In future content-centric networks, content is identified independently of
its location. From an end-user's perspective, individual storage systems
dissolve into a seemingly omnipresent structureless `storage fog'. Content
should be delivered oblivious of the network topology, using multiple storage
systems simultaneously, and at minimal coordination overhead. Prior works have
addressed the advantages of error correction coding for distributed storage and
content delivery separately. This work takes a comprehensive approach to
highlighting the tradeoff between storage overhead and transmission overhead in
uncoordinated content delivery from multiple coded storage systems.
Our contribution is twofold. First, we characterize the tradeoff between
storage and transmission overhead when all participating storage systems employ
the same code. Second, we show that the resulting stark inefficiencies can be
avoided when storage systems use diverse codes. What is more, such code
diversity is not just technically desirable, but presumably will be the reality
in the increasingly heterogeneous networks of the future. To this end, we show
that a mix of Reed-Solomon, low-density parity-check and random linear network
codes achieves close-to-optimal performance at minimal coordination and
operational overhead
What broke where for distributed and parallel applications — a whodunit story
Detection, diagnosis and mitigation of performance problems in today\u27s large-scale distributed and parallel systems is a difficult task. These large distributed and parallel systems are composed of various complex software and hardware components. When the system experiences some performance or correctness problem, developers struggle to understand the root cause of the problem and fix in a timely manner. In my thesis, I address these three components of the performance problems in computer systems. First, we focus on diagnosing performance problems in large-scale parallel applications running on supercomputers. We developed techniques to localize the performance problem for root-cause analysis. Parallel applications, most of which are complex scientific simulations running in supercomputers, can create up to millions of parallel tasks that run on different machines and communicate using the message passing paradigm. We developed a highly scalable and accurate automated debugging tool called PRODOMETER, which uses sophisticated algorithms to first, create a logical progress dependency graph of the tasks to highlight how the problem spread through the system manifesting as a system-wide performance issue. Second, uses this logical progress dependence graph to identify the task where the problem originated. Finally, PRODOMETER pinpoints the code region corresponding to the origin of the bug. Second, we developed a tool-chain that can detect performance anomaly using machine-learning techniques and can achieve very low false positive rate. Our input-aware performance anomaly detection system consists of a scalable data collection framework to collect performance related metrics from different granularity of code regions, an offline model creation and prediction-error characterization technique, and a threshold based anomaly-detection-engine for production runs. Our system requires few training runs and can handle unknown inputs and parameter combinations by dynamically calibrating the anomaly detection threshold according to the characteristics of the input data and the characteristics of the prediction-error of the models. Third, we developed performance problem mitigation scheme for erasure-coded distributed storage systems. Repair operations of the failed blocks in erasure-coded distributed storage system take really long time in networked constrained data-centers. The reason being, during the repair operation for erasure-coded distributed storage, a lot of data from multiple nodes are gathered into a single node and then a mathematical operation is performed to reconstruct the missing part. This process severely congests the links toward the destination where newly recreated data is to be hosted. We proposed a novel distributed repair technique, called Partial-Parallel-Repair (PPR) that performs this reconstruction in parallel on multiple nodes and eliminates network bottlenecks, and as a result, greatly speeds up the repair process. Fourth, we study how for a class of applications, performance can be improved (or performance problems can be mitigated) by selectively approximating some of the computations. For many applications, the main computation happens inside a loop that can be logically divided into a few temporal segments, we call phases. We found that while approximating the initial phases might severely degrade the quality of the results, approximating the computation for the later phases have very small impact on the final quality of the result. Based on this observation, we developed an optimization framework that for a given budget of quality-loss, would find the best approximation settings for each phase in the execution
- …