Asymptotically MDS Array BP-XOR Codes

Belief propagation or message passing on binary erasure channels (BEC) is a low complexity decoding algorithm that allows the recovery of message symbols based on bipartite graph prunning process. Recently, array XOR codes have attracted attention for storage systems due to their burst error recovery performance and easy arithmetic based on Exclusive OR (XOR)-only logic operations. Array BP-XOR codes are a subclass of array XOR codes that can be decoded using BP under BEC. Requiring the capability of BP-decodability in addition to Maximum Distance Separability (MDS) constraint on the code construction process is observed to put an upper bound on the maximum achievable code block length, which leads to the code construction process to become a harder problem. In this study, we introduce asymptotically MDS array BP-XOR codes that are alternative to exact MDS array BP-XOR codes to pave the way for easier code constructions while keeping the decoding complexity low with an asymptotically vanishing coding overhead. We finally provide and analyze a simple code construction method that is based on discrete geometry to fulfill the requirements of the class of asymptotically MDS array BP-XOR codes.Comment: 8 pages, 4 figures, to be submitte

Asymptotically MDS Array BP-XOR Codes

Belief propagation or message passing on binary erasure channels (BEC) is a low complexity decoding algorithm that allows the recovery of message symbols based on bipartite graph prunning process. Recently, array XOR codes have attracted attention for storage systems due to their burst error recovery performance and easy arithmetic based on Exclusive OR (XOR)-only logic operations. Array BP-XOR codes are a subclass of array XOR codes that can be decoded using BP under BEC. Requiring the capability of BP-decodability in addition to Maximum Distance Separability (MDS) constraint on the code construction process is observed to put an upper bound on the maximum achievable code block length, which leads to the code construction process to become a harder problem. In this study, we introduce asymptotically MDS array BP-XOR codes that are alternative to exact MDS array BP-XOR codes to pave the way for easier code constructions while keeping the decoding complexity low with an asymptotically vanishing coding overhead. We finally provide and analyze a simple code construction method that is based on discrete geometry to fulfill the requirements of the class of asymptotically MDS array BP-XOR codes.Comment: 8 pages, 4 figures, to be submitte

On encoding symbol degrees of array BP-XOR codes

Low density parity check (LDPC) codes, LT codes and digital fountain techniques have received significant attention from both academics and industry in the past few years. By employing the underlying ideas of efficient Belief Propagation (BP) decoding process (also called iterative message passing decoding process) on binary erasure channels (BEC) in LDPC codes, Wang has recently introduced the concept of array BP-XOR codes and showed the necessary and sufficient conditions for MDS [k + 2,k] and [n,2] array BP-XOR codes. In this paper, we analyze the encoding symbol degree requirements for array BP-XOR codes and present new necessary conditions for array BP-XOR codes. These new necessary conditions are used as a guideline for constructing several array BP-XOR codes and for presenting a complete characterization (necessary and sufficient conditions) of degree two array BP-XOR codes and for designing new edge-colored graphs. Meanwhile, these new necessary conditions are used to show that the codes by Feng, Deng, Bao, and Shen in IEEE Transactions on Computers are incorrect

A reliability model for dependent and distributed MDS disk array units

Archiving and systematic backup of large digital data generates a quick demand for multi-petabyte scale storage systems. As drive capacities continue to grow beyond the few terabytes range to address the demands of today’s cloud, the likelihood of having multiple/simultaneous disk failures became a reality. Among the main factors causing catastrophic system failures, correlated disk failures and the network bandwidth are reported to be the two common source of performance degradation. The emerging trend is to use efficient/sophisticated erasure codes (EC) equipped with multiple parities and efficient repairs in order to meet the reliability/bandwidth requirements. It is known that mean time to failure and repair rates reported by the disk manufacturers cannot capture life-cycle patterns of distributed storage systems. In this study, we develop failure models based on generalized Markov chains that can accurately capture correlated performance degradations with multiparity protection schemes based on modern maximum distance separable EC. Furthermore, we use the proposed model in a distributed storage scenario to quantify two example use cases: Primarily, the common sense that adding more parity disks are only meaningful if we have a decent decorrelation between the failure domains of storage systems and the reliability of generic multiple single-dimensional EC protected storage systems.WOS:000460728600008Scopus - Affiliation ID: 60105072Science Citation Index ExpandedQ1 - Q2ArticleUluslararası işbirliği ile yapılmayan - HAYIRMart2019YÖK - 2018-1

Almost BPXOR Coding Technique for Tolerating Three Disk Failures in RAID7 Architectures

Redundant Array of Independent Disks (RAID) storage architectures provide protection of digital infrastructure against potential disks failures. For example, RAID-5 and RAID-6 architectures provide protection against one and two disk failures, respectively. Recently, the data generation has significantly increased due to the emergence of new technologies. Thus, the size of storage systems is also growing rapidly to accommodate such large data sizes, which increases the probability for disks failures. This necessitates a new RAID architecture that can tolerate up to three disk failures. RAID architectures implement coding techniques. The code specifies how data is stored among multiple disks and how lost data can be recovered from surviving disks. This abstract introduces a novel coding scheme for new RAID-7 architectures that can tolerate up to three disks failures. The code is an improved version of the existing BP-XOR code and is called "Almost BP-XOR".There are multiple codes that can be used for RAID-7 architectures. However, [5,2] BP-XOR codes have significantly lower encoding and decoding complexities than most common codes [1]. Regardless of this fact, this code does not achieve the fastest data decoding and reconstruction speeds due to its relatively low efficiency of 0.4. Furthermore, the existence of MDS [6,3] bx6 BP-XOR codes, b>2 (which achieves efficiency of 0.5) is still an open research question. This work proposes [6,3] 2 x 6 Almost BP-XOR codes. These codes largely utilize the simple and fast BP-XOR decoder while achieving an efficiency of 0.5, leading to the fastest recovery from disk failures among other state-of-the-art codes. An algorithm to generate a [6, 3] 2 x 6 Almost BP-XOR code has been developed and an example code is provided in Table 1. The [6, 3] 2 x 6 Almost BP-XOR codes are constructed in a way that any three-column-erasure pattern will result in one of the following two main scenarios. First: At least one of the surviving degree-three encoding symbols contains two known information symbols. This scenario occurs in 70% of three-column erasure cases (i.e. 14 out of the 20 possible cases). The recovery process in such scenario is identical to that of the BP-XOR codes; Knowing any two information symbols in a degree-three encoding symbol is sufficient to know the third information symbol through performing a simple XOR operation. Second: None of the surviving degree-three encoding symbols contains two known information symbols. This scenario occurs in the remaining 30% of three-column erasure cases (i.e., 6 out if the possible 20). The BP-XOR decoder fails in such a scenario. However, due to the construction of the codes, at least one surviving degree-three encoding symbol contains a known information symbol. Thus, knowing one of the reaming two information symbols in such a degree-three encoding symbol will initiate the BP-XOR decoder again.Table 2 shows these erasure patterns along with an expression for one of the missing information symbols. these expressions can be stored in buffers and used whenever the corresponding erasure pattern occurs. Solutions in Table 2 are derived from the inverse of a 6x6 submatrix that results from a generator matrix G by deleting columns from G corresponding to erased code columns. The read complexity of almost BP-XOR codes is 1. On the other hand. The decoding of almost BP-XOR codes require just 6 XOR operations when for a given three-column-erasure pattern BP-XOR decoding succeeds. However, when the BP- XOR decoder fails, it will require up to 15 XOR operations in total. The normalized repairing complexity is 15/6 = 2.5.Experimentally, Fig.1 shows that the proposed Almost BP-XOR codes require the least amount of time to decode and reconstruct erased columns. Thus, it is concluded that the [6, 3] 2x6 almost BP-XOR codes are best suited for RAID-7 system that requires storage efficiency of 0.5.qscienc

Hyfs: design and implementation of a reliable file system

Building reliable data storage systems is crucial to any commercial or scientific applications. Modern storage systems are complicated, and they are comprised of many components, from hardware to software. Problems may occur to any component of storage systems and cause data loss. When this kind of failures happens, storage systems cannot continue their data services, which may result in large revenue loss or even catastrophe to enterprises. Therefore, it is critically important to build reliable storage systems to ensure data reliability. In this dissertation, we propose to employ general erasure codes to build a reliable file system, called HyFS. HyFS is a cluster system, which can aggregate distributed storage servers to provide reliable data service. On client side, HyFS is implemented as a native file system so that applications can transparently run on top of HyFS. On server side, HyFS utilizes multiple distributed storage servers to provide highly reliable data service by employing erasure codes. HyFS is able to offer high throughput for either random or sequential file access, which makes HyFS an attractive choice for primary or backup storage systems. This dissertation studies five relevant topics of HyFS. Firstly, it presents several algorithms that can perform encoding operation efficiently for XOR-based erasure codes. Secondly, it discusses an efficient decoding algorithm for RAID-6 erasure codes. This algorithm can recover various types of disk failures. Thirdly, it describes an efficient algorithm to detect and correct errors for the STAR code, which further improves a storage system\u27s reliability. Fourthly, it describes efficient implementations for the arithmetic operations of large finite fields. This is to improve a storage system\u27s security. Lastly and most importantly, it presents the design and implementation of HyFS and evaluates the performance of HyFS

Data storage security and privacy in cloud computing: A comprehensive survey

Cloud Computing is a form of distributed computing wherein resources and application platforms are distributed over the Internet through on demand and pay on utilization basis. Data Storage is main feature that cloud data centres are provided to the companies/organizations to preserve huge data. But still few organizations are not ready to use cloud technology due to lack of security. This paper describes the different techniques along with few security challenges, advantages and also disadvantages. It also provides the analysis of data security issues and privacy protection affairs related to cloud computing by preventing data access from unauthorized users, managing sensitive data, providing accuracy and consistency of data store

Achieving reliable and enhanced communication in vehicular ad hoc networks (VANETs)

A thesis submitted to the University of Bedfordshire in partial fulfilment of the requirement for the degree of Doctor of PhilosophyWith the envisioned age of Internet of Things (IoTs), different aspects of Intelligent Transportation System (ITS) will be linked so as to advance road transportation safety, ease congestion of road traffic, lessen air pollution, improve passenger transportation comfort and significantly reduce road accidents. In vehicular networks, regular exchange of current position, direction, speed, etc., enable mobile vehicle to foresee an imminent vehicle accident and notify the driver early enough in order to take appropriate action(s) or the vehicle on its own may take adequate preventive measures to avert the looming accident. Actualizing this concept requires use of shared media access protocol that is capable of guaranteeing reliable and timely broadcast of safety messages. This dissertation investigates the use of Network Coding (NC) techniques to enrich the content of each transmission and ensure improved high reliability of the broadcasted safety messages with less number of retransmissions. A Code Aided Retransmission-based Error Recovery (CARER) protocol is proposed. In order to avoid broadcast storm problem, a rebroadcasting vehicle selection metric η, is developed, which is used to select a vehicle that will rebroadcast the received encoded message. Although the proposed CARER protocol demonstrates an impressive performance, the level of incurred overhead is fairly high due to the use of complex rebroadcasting vehicle selection metric. To resolve this issue, a Random Network Coding (RNC) and vehicle clustering based vehicular communication scheme with low algorithmic complexity, named Reliable and Enhanced Cooperative Cross-layer MAC (RECMAC) scheme, is proposed. The use of this clustering technique enables RECMAC to subdivide the vehicular network into small manageable, coordinated clusters which further improve transmission reliability and minimise negative impact of network overhead. Similarly, a Cluster Head (CH) selection metric ℱ(\u1d457) is designed, which is used to determine and select the most suitably qualified candidate to become the CH of a particular cluster. Finally, in order to investigate the impact of available radio spectral resource, an in-depth study of the required amount of spectrum sufficient to support high transmission reliability and minimum latency requirements of critical road safety messages in vehicular networks was carried out. The performance of the proposed schemes was clearly shown with detailed theoretical analysis and was further validated with simulation experiments