Video Streaming Framework, Journal of Telecommunications and Information Technology, 2011, nr 3

The framework for testing video streaming techniques is presented in this paper. Short review of error resilience and concealments tools available for the H.264/AVC standard is given. The video streaming protocols and the H.264 payload format are also described. The experimental results obtained with the framework are presented in this paper too

Enhanced quality reconstruction of erroneous video streams using packet filtering based on non-desynchronizing bits and UDP checksum-filtered list decoding

The latest video coding standards, such as H.264 and H.265, are extremely vulnerable in error-prone networks. Due to their sophisticated spatial and temporal prediction tools, the effect of an error is not limited to the erroneous area but it can easily propagate spatially to the neighboring blocks and temporally to the following frames. Thus, reconstructed video packets at the decoder side may exhibit significant visual quality degradation. Error concealment and error corrections are two mechanisms that have been developed to improve the quality of reconstructed frames in the presence of errors. In most existing error concealment approaches, the corrupted packets are ignored and only the correctly received information of the surrounding areas (spatially and/or temporally) is used to recover the erroneous area. This is due to the fact that there is no perfect error detection mechanism to identify correctly received blocks within a corrupted packet, and moreover because of the desynchronization problem caused by the transmission errors on the variable-length code (VLC). But, as many studies have shown, the corrupted packets may contain valuable information that can be used to reconstruct adequately of the lost area (e.g. when the error is located at the end of a slice). On the other hand, error correction approaches, such as list decoding, exploit the corrupted packets to generate several candidate transmitted packets from the corrupted received packet. They then select, among these candidates, the one with the highest likelihood of being the transmitted packet based on the available soft information (e.g. log-likelihood ratio (LLR) of each bit). However, list decoding approaches suffer from a large solution space of candidate transmitted packets. This is worsened when the soft information is not available at the application layer; a more realistic scenario in practice. Indeed, since it is unknown which bits have higher probabilities of having been modified during transmission, the candidate received packets cannot be ranked by likelihood. In this thesis, we propose various strategies to improve the quality of reconstructed packets which have been lightly damaged during transmission (e.g. at most a single error per packet). We first propose a simple but efficient mechanism to filter damaged packets in order to retain those likely to lead to a very good reconstruction and discard the others. This method can be used as a complement to most existing concealment approaches to enhance their performance. The method is based on the novel concept of non-desynchronizing bits (NDBs) defined, in the context of an H.264 context-adaptive variable-length coding (CAVLC) coded sequence, as a bit whose inversion does not cause desynchronization at the bitstream level nor changes the number of decoded macroblocks. We establish that, on typical coded bitstreams, the NDBs constitute about a one-third (about 30%) of a bitstream, and that the effect on visual quality of flipping one of them in a packet is mostly insignificant. In most cases (90%), the quality of the reconstructed packet when modifying an individual NDB is almost the same as the intact one. We thus demonstrate that keeping, under certain conditions, a corrupted packet as a candidate for the lost area can provide better visual quality compared to the concealment approaches. We finally propose a non-desync-based decoding framework, which retains a corrupted packet, under the condition of not causing desynchronization and not altering the number of expected macroblocks. The framework can be combined with most current concealment approaches. The proposed approach is compared to the frame copy (FC) concealment of Joint Model (JM) software (JM-FC) and a state-of-the-art concealment approach using the spatiotemporal boundary matching algorithm (STBMA) mechanism, in the case of one bit in error, and on average, respectively, provides 3.5 dB and 1.42 dB gain over them. We then propose a novel list decoding approach called checksum-filtered list decoding (CFLD) which can correct a packet at the bit stream level by exploiting the receiver side user datagram protocol (UDP) checksum value. The proposed approach is able to identify the possible locations of errors by analyzing the pattern of the calculated UDP checksum on the corrupted packet. This makes it possible to considerably reduce the number of candidate transmitted packets in comparison to conventional list decoding approaches, especially when no soft information is available. When a packet composed of N bits contains a single bit in error, instead of considering N candidate packets, as is the case in conventional list decoding approaches, the proposed approach considers approximately N/32 candidate packets, leading to a 97% reduction in the number of candidates. This reduction can increase to 99.6% in the case of a two-bit error. The method’s performance is evaluated using H.264 and high efficiency video coding (HEVC) test model software. We show that, in the case H.264 coded sequence, on average, the CFLD approach is able to correct the packet 66% of the time. It also offers a 2.74 dB gain over JM-FC and 1.14 dB and 1.42 dB gains over STBMA and hard output maximum likelihood decoding (HO-MLD), respectively. Additionally, in the case of HEVC, the CFLD approach corrects the corrupted packet 91% of the time, and offers 2.35 dB and 4.97 dB gains over our implementation of FC concealment in HEVC test model software (HM-FC) in class B (1920×1080) and C (832×480) sequences, respectively

Journal of Telecommunications and Information Technology, 2011, nr 3

kwartalni

Error resilience and concealment techniques for packet video transmission

Since the quality of compressed video is vulnerable to errors, video transmission over unreliable Internet is very challenging today. In this thesis, two important issues about robust packet video transmission are investigated. The first issue is error resilient (ER) video compression. Motivated by two-hypothesis motion-compensated prediction (THMCP), we first propose an error resilient video coding technique, where two-hypothesis and one-hypothesis predictions are alternately used to encode the video stream. We use some schemes to determine which kind of prediction should be used, so that in some cases of frame losses, the propagated error can be first decreased to some extent before it spreads to the subsequent frames. Then we extend the previous work for THMCP to a more generalized one, i.e. the reference frames can be some distance from the current frame, instead of only the immediately preceding ones. Three types of prediction patterns are proposed and implemented by the generalized B pictures in H.264/AVC. In the case of a single frame loss during the transmission, the induced error propagation is analyzed and the closed-form expression for error energy is derived. The second issue is error concealment (EC) for packet losses in video transmission. We first propose an error concealment algorithm to reconstruct a lost INTER-frame in the odd/even temporal sub-sampling MDC. The neighboring frames in the error-free stream are used to temporally interpolate the lost frame, based the preserved motion vector in this correct stream. To further improve the reconstructed video quality after the lost one, we also propose a multi-hypothesis error concealment (MHC) algorithm. In addition to error-concealing the lost frame, MHC also applies temporal interpolation to some additional frames after the frame loss so as to reduce propagated error quickly. After discussing the EC algorithms for INTER-frame losses, we propose two algorithms to error-conceal a lost INTRA-frame. The novelty is that not only the INTRA-frame but also the subsequent INTER-frames are error concealed based on the strong correlation between adjacent pixel values. In addition, motion compensation is used to reconstruct the INTER-pixel which has an INTRA-pixel in its motion trajectory. Finally, we propose an edge-directed error concealment algorithm to recover lost slices in FMO-encoded video. The strong edges in a corrupted frame are estimated first, which are then used to direct the recovery of the structure regions and the remaining erroneous regions