Founsure 1.0: An erasure code library with efficient repair and update features

Founsure is an open-source software library that implements a multi-dimensional graph-based erasure coding entirely based on fast exclusive OR (XOR) logic. Its implementation utilizes compiler optimizations and multi-threading to generate the right assembly code for the given multi-core CPU architecture with vector processing capabilities. Founsure possesses important features that shall find various applications in modern data storage, communication, and networked computer systems, in which the data needs protection against device, hardware, and node failures. As data size reached unprecedented levels, these systems have become hungry for network bandwidth, computational resources, and average consumed power. To address that, the proposed library provides a three-dimensional design space that trades off the computational complexity, coding overhead, and data/node repair bandwidth to meet different requirements of modern distributed data storage and processing systems. Founsure library enables efficient encoding, decoding, repairs/rebuilds, and updates while all the required data storage and computations are distributed across the network nodes.Turkiye Bilimsel ve Teknolojik Arastirma Kurumu (TUBITAK) Grant Number : 115C111 - 119E235WOS:000656825700019Scopus - Affiliation ID: 60105072Science Citation Index ExpandedQ3ArticleUluslararası işbirliği ile yapılmayan - HAYIRJanuary2021YÖK - 2020-2

Deterministic Rateless Codes for BSC

A rateless code encodes a finite length information word into an infinitely long codeword such that longer prefixes of the codeword can tolerate a larger fraction of errors. A rateless code achieves capacity for a family of channels if, for every channel in the family, reliable communication is obtained by a prefix of the code whose rate is arbitrarily close to the channel's capacity. As a result, a universal encoder can communicate over all channels in the family while simultaneously achieving optimal communication overhead. In this paper, we construct the first \emph{deterministic} rateless code for the binary symmetric channel. Our code can be encoded and decoded in $O(\beta)$ time per bit and in almost logarithmic parallel time of $O(\beta \log n)$, where $\beta$ is any (arbitrarily slow) super-constant function. Furthermore, the error probability of our code is almost exponentially small $\exp(-\Omega(n/\beta))$. Previous rateless codes are probabilistic (i.e., based on code ensembles), require polynomial time per bit for decoding, and have inferior asymptotic error probabilities. Our main technical contribution is a constructive proof for the existence of an infinite generating matrix that each of its prefixes induce a weight distribution that approximates the expected weight distribution of a random linear code

Joint source-channel-network coding in wireless mesh networks with temporal reuse

Technological innovation that empowers tiny low-cost transceivers to operate with a high degree of utilisation efficiency in multihop wireless mesh networks is contributed in this dissertation. Transmission scheduling and joint source-channel-network coding are two of the main aspects that are addressed. This work focuses on integrating recent enhancements such as wireless network coding and temporal reuse into a cross-layer optimisation framework, and to design a joint coding scheme that allows for space-optimal transceiver implementations. Link-assigned transmission schedules with timeslot reuse by multiple links in both the space and time domains are investigated for quasi-stationary multihop wireless mesh networks with both rate and power adaptivity. Specifically, predefined cross-layer optimised schedules with proportionally fair end-to-end flow rates and network coding capability are constructed for networks operating under the physical interference model with single-path minimum hop routing. Extending transmission rights in a link-assigned schedule allows for network coding and temporal reuse, which increases timeslot usage efficiency when a scheduled link experiences packet depletion. The schedules that suffer from packet depletion are characterised and a generic temporal reuse-aware achievable rate region is derived. Extensive computational experiments show improved schedule capacity, quality of service, power efficiency and benefit from opportunistic bidirectional network coding accrued with schedules optimised in the proposed temporal reuse-aware convex capacity region. The application of joint source-channel coding, based on fountain codes, in the broadcast timeslot of wireless two-way network coding is also investigated. A computationally efficient subroutine is contributed to the implementation of the fountain compressor, and an error analysis is done. Motivated to develop a true joint source-channel-network code that compresses, adds robustness against channel noise and network codes two packets on a single bipartite graph and iteratively decodes the intended packet on the same Tanner graph, an adaptation of the fountain compressor is presented. The proposed code is shown to outperform a separated joint source-channel and network code in high source entropy and high channel noise regions, in anticipated support of dense networks that employ intelligent signalling. AFRIKAANS : Tegnologiese innovasie wat klein lae-koste kommunikasie toestelle bemagtig om met ’n hoë mate van benuttings doeltreffendheid te werk word bygedra in hierdie proefskrif. Transmissie-skedulering en gesamentlike bron-kanaal-netwerk kodering is twee van die belangrike aspekte wat aangespreek word. Hierdie werk fokus op die integrasie van onlangse verbeteringe soos draadlose netwerk kodering en temporêre herwinning in ’n tussen-laag optimaliserings raamwerk, en om ’n gesamentlike kodering skema te ontwerp wat voorsiening maak vir spasie-optimale toestel implementerings. Skakel-toegekende transmissie skedules met tydgleuf herwinning deur veelvuldige skakels in beide die ruimte en tyd domeine word ondersoek vir kwasi-stilstaande, veelvuldige-sprong draadlose rooster netwerke met beide transmissie-spoed en krag aanpassings. Om spesifiek te wees, word vooraf bepaalde tussen-laag geoptimiseerde skedules met verhoudings-regverdige punt-tot-punt vloei tempo’s en netwerk kodering vermoë saamgestel vir netwerke wat bedryf word onder die fisiese inmengings-model met enkel-pad minimale sprong roetering. Die uitbreiding van transmissie-regte in ’n skakel-toegekende skedule maak voorsiening vir netwerk kodering en temporêre herwinning, wat tydgleuf gebruiks-doeltreffendheid verhoog wanneer ’n geskeduleerde skakel pakkie-uitputting ervaar. Die skedules wat ly aan pakkie-uitputting word gekenmerk en ’n generiese temporêre herwinnings-bewuste haalbare transmissie-spoed gebied word afgelei. Omvattende berekenings-eksperimente toon verbeterde skedulerings kapasiteit, diensgehalte, krag doeltreffendheid asook verbeterde voordeel wat getrek word uit opportunistiese tweerigting netwerk kodering met die skedules wat geoptimiseer word in die temporêre herwinnings-bewuste konvekse transmissie-spoed gebied. Die toepassing van gesamentlike bron-kanaal kodering, gebaseer op fontein kodes, in die uitsaai-tydgleuf van draadlose tweerigting netwerk kodering word ook ondersoek. ’n Berekenings-effektiewe subroetine word bygedra in die implementering van die fontein kompressor, en ’n foutanalise word gedoen. Gemotiveer om ’n ware gesamentlike bron-kanaal-netwerk kode te ontwikkel, wat robuustheid byvoeg teen kanaal geraas en twee pakkies netwerk kodeer op ’n enkele bipartiete grafiek en die beoogde pakkie iteratief dekodeer op dieselfde Tanner grafiek, word ’n aanpassing van die fontein kompressor aangebied. Dit word getoon dat die voorgestelde kode ’n geskeide gesamentlike bron-kanaal en netwerk kode in hoë bron-entropie en ho¨e kanaal-geraas gebiede oortref in verwagte ondersteuning van digte netwerke wat van intelligente sein-metodes gebruik maak.Dissertation (MEng)--University of Pretoria, 2011.Electrical, Electronic and Computer Engineeringunrestricte

Towards reliable communication in LTE-A connected heterogeneous machine to machine network

Machine to machine (M2M) communication is an emerging technology that enables heterogeneous devices to communicate with each other without human intervention and thus forming so-called Internet of Things (IoTs). Wireless cellular networks (WCNs) play a significant role in the successful deployment of M2M communication. Specially the ongoing massive deployment of long term evolution advanced (LTE-A) makes it possible to establish machine type communication (MTC) in most urban and remote areas, and by using LTE-A backhaul network, a seamless network communication is being established between MTC-devices and-applications. However, the extensive network coverage does not ensure a successful implementation of M2M communication in the LTE-A, and therefore there are still some challenges. Energy efficient reliable transmission is perhaps the most compelling demand for various M2M applications. Among the factors affecting reliability of M2M communication are the high endto-end delay and high bit error rate. The objective of the thesis is to provide reliable M2M communication in LTE-A network. In this aim, to alleviate the signalling congestion on air interface and efficient data aggregation we consider a cluster based architecture where the MTC devices are grouped into number of clusters and traffics are forwarded through some special nodes called cluster heads (CHs) to the base station (BS) using single or multi-hop transmissions. In many deployment scenarios, some machines are allowed to move and change their location in the deployment area with very low mobility. In practice, the performance of data transmission often degrades with the increase of distance between neighboring CHs. CH needs to be reselected in such cases. However, frequent re-selection of CHs results in counter effect on routing and reconfiguration of resource allocation associated with CH-dependent protocols. In addition, the link quality between a CH-CH and CH-BS are very often affected by various dynamic environmental factors such as heat and humidity, obstacles and RF interferences. Since CH aggregates the traffic from all cluster members, failure of the CH means that the full cluster will fail. Many solutions have been proposed to combat with error prone wireless channel such as automatic repeat request (ARQ) and multipath routing. Though the above mentioned techniques improve the communication reliability but intervene the communication efficiency. In the former scheme, the transmitter retransmits the whole packet even though the part of the packet has been received correctly and in the later one, the receiver may receive the same information from multiple paths; thus both techniques are bandwidth and energy inefficient. In addition, with retransmission, overall end to end delay may exceed the maximum allowable delay budget. Based on the aforementioned observations, we identify CH-to-CH channel is one of the bottlenecks to provide reliable communication in cluster based multihop M2M network and present a full solution to support fountain coded cooperative communications. Our solution covers many aspects from relay selection to cooperative formation to meet the user’s QoS requirements. In the first part of the thesis, we first design a rateless-coded-incremental-relay selection (RCIRS) algorithm based on greedy techniques to guarantee the required data rate with a minimum cost. After that, we develop fountain coded cooperative communication protocols to facilitate the data transmission between two neighbor CHs. In the second part, we propose joint network and fountain coding schemes for reliable communication. Through coupling channel coding and network coding simultaneously in the physical layer, joint network and fountain coding schemes efficiently exploit the redundancy of both codes and effectively combat the detrimental effect of fading conditions in wireless channels. In the proposed scheme, after correctly decoding the information from different sources, a relay node applies network and fountain coding on the received signals and then transmits to the destination in a single transmission. Therefore, the proposed schemes exploit the diversity and coding gain to improve the system performance. In the third part, we focus on the reliable uplink transmission between CHs and BS where CHs transmit to BS directly or with the help of the LTE-A relay nodes (RN). We investigate both type-I and type-II enhanced LTE-A networks and propose a set of joint network and fountain coding schemes to enhance the link robustness. Finally, the proposed solutions are evaluated through extensive numerical simulations and the numerical results are presented to provide a comparison with the related works found in the literature

Random Linear Network Coding for 5G Mobile Video Delivery

An exponential increase in mobile video delivery will continue with the demand for higher resolution, multi-view and large-scale multicast video services. Novel fifth generation (5G) 3GPP New Radio (NR) standard will bring a number of new opportunities for optimizing video delivery across both 5G core and radio access networks. One of the promising approaches for video quality adaptation, throughput enhancement and erasure protection is the use of packet-level random linear network coding (RLNC). In this review paper, we discuss the integration of RLNC into the 5G NR standard, building upon the ideas and opportunities identified in 4G LTE. We explicitly identify and discuss in detail novel 5G NR features that provide support for RLNC-based video delivery in 5G, thus pointing out to the promising avenues for future research.Comment: Invited paper for Special Issue "Network and Rateless Coding for Video Streaming" - MDPI Informatio

GRACE: Loss-Resilient Real-Time Video through Neural Codecs

In real-time video communication, retransmitting lost packets over high-latency networks is not viable due to strict latency requirements. To counter packet losses without retransmission, two primary strategies are employed -- encoder-based forward error correction (FEC) and decoder-based error concealment. The former encodes data with redundancy before transmission, yet determining the optimal redundancy level in advance proves challenging. The latter reconstructs video from partially received frames, but dividing a frame into independently coded partitions inherently compromises compression efficiency, and the lost information cannot be effectively recovered by the decoder without adapting the encoder. We present a loss-resilient real-time video system called GRACE, which preserves the user's quality of experience (QoE) across a wide range of packet losses through a new neural video codec. Central to GRACE's enhanced loss resilience is its joint training of the neural encoder and decoder under a spectrum of simulated packet losses. In lossless scenarios, GRACE achieves video quality on par with conventional codecs (e.g., H.265). As the loss rate escalates, GRACE exhibits a more graceful, less pronounced decline in quality, consistently outperforming other loss-resilient schemes. Through extensive evaluation on various videos and real network traces, we demonstrate that GRACE reduces undecodable frames by 95% and stall duration by 90% compared with FEC, while markedly boosting video quality over error concealment methods. In a user study with 240 crowdsourced participants and 960 subjective ratings, GRACE registers a 38% higher mean opinion score (MOS) than other baselines

Variable Rate Transmission Over Noisy Channels

Hybrid automatic repeat request transmission (hybrid ARQ) schemes aim to provide system reliability for transmissions over noisy channels while still maintaining a reasonably high throughput efficiency by combining retransmissions of automatic repeat requests with forward error correction (FEC) coding methods. In type-II hybrid ARQ schemes, the additional parity information required by channel codes to achieve forward error correction is provided only when errors have been detected. Hence, the available bits are partitioned into segments, some of which are sent to the receiver immediately, others are held back and only transmitted upon the detection of errors. This scheme raises two questions. Firstly, how should the available bits be ordered for optimal partitioning into consecutive segments? Secondly, how large should the individual segments be? This thesis aims to provide an answer to both of these questions for the transmission of convolutional and Turbo Codes over additive white Gaussian noise (AWGN), inter-symbol interference (ISI) and Rayleigh channels. Firstly, the ordering of bits is investigated by simulating the transmission of packets split into segments with a size of 1 bit and finding the critical number of bits, i.e. the number of bits where the output of the decoder is error-free. This approach provides a maximum, practical performance limit over a range of signal-to-noise levels. With these practical performance limits, the attention is turned to the size of the individual segments, since packets of 1 bit cause an intolerable overhead and delay. An adaptive, hybrid ARQ system is investigated, in which the transmitter uses the number of bits sent to the receiver and the receiver decoding results to adjust the size of the first, initial, packet and subsequent segments to the conditions of a stationary channel