Exact Asymptotics for the Random Coding Error Probability

Error probabilities of random codes for memoryless channels are considered in this paper. In the area of communication systems, admissible error probability is very small and it is sometimes more important to discuss the relative gap between the achievable error probability and its bound than to discuss the absolute gap. Scarlett et al. derived a good upper bound of a random coding union bound based on the technique of saddlepoint approximation but it is not proved that the relative gap of their bound converges to zero. This paper derives a new bound on the achievable error probability in this viewpoint for a class of memoryless channels. The derived bound is strictly smaller than that by Scarlett et al. and its relative gap with the random coding error probability (not a union bound) vanishes as the block length increases for a fixed coding rate.Comment: Full version of the paper in ISIT2015 with some corrections and refinement

Refinement of the random coding bound

An improved pre-factor for the random coding bound is proved. Specifically, for channels with critical rate not equal to capacity, if a regularity condition is satisfied (resp. not satisfied), then for any $\epsilon >0$ a pre-factor of $O(N^{-\frac{1}{2}\left( 1 - \epsilon + \bar{\rho}^\ast_R \right)})$ (resp. $O(N^{-\frac{1}{2}})$) is achievable for rates above the critical rate, where $N$ and $R$ is the blocklength and rate, respectively. The extra term $\bar{\rho}^\ast_R$ is related to the slope of the random coding exponent. Further, the relation of these bounds with the authors' recent refinement of the sphere-packing bound, as well as the pre-factor for the random coding bound below the critical rate, is discussed.Comment: Submitted to IEEE Trans. Inform. Theor

Hypergraph-based Source Codes for Function Computation Under Maximal Distortion

This work investigates functional source coding problems with maximal distortion, motivated by approximate function computation in many modern applications. The maximal distortion treats imprecise reconstruction of a function value as good as perfect computation if it deviates less than a tolerance level, while treating reconstruction that differs by more than that level as a failure. Using a geometric understanding of the maximal distortion, we propose a hypergraph-based source coding scheme for function computation that is constructive in the sense that it gives an explicit procedure for defining auxiliary random variables. Moreover, we find that the hypergraph-based coding scheme achieves the optimal rate-distortion function in the setting of coding for computing with side information and the Berger-Tung sum-rate inner bound in the setting of distributed source coding for computing. It also achieves the El Gamal-Cover inner bound for multiple description coding for computing and is optimal for successive refinement and cascade multiple description problems for computing. Lastly, the benefit of complexity reduction of finding a forward test channel is shown for a class of Markov sources

Multiuser Successive Refinement and Multiple Description Coding

We consider the multiuser successive refinement (MSR) problem, where the users are connected to a central server via links with different noiseless capacities, and each user wishes to reconstruct in a successive-refinement fashion. An achievable region is given for the two-user two-layer case and it provides the complete rate-distortion region for the Gaussian source under the MSE distortion measure. The key observation is that this problem includes the multiple description (MD) problem (with two descriptions) as a subsystem, and the techniques useful in the MD problem can be extended to this case. We show that the coding scheme based on the universality of random binning is sub-optimal, because multiple Gaussian side informations only at the decoders do incur performance loss, in contrast to the case of single side information at the decoder. We further show that unlike the single user case, when there are multiple users, the loss of performance by a multistage coding approach can be unbounded for the Gaussian source. The result suggests that in such a setting, the benefit of using successive refinement is not likely to justify the accompanying performance loss. The MSR problem is also related to the source coding problem where each decoder has its individual side information, while the encoder has the complete set of the side informations. The MSR problem further includes several variations of the MD problem, for which the specialization of the general result is investigated and the implication is discussed.Comment: 10 pages, 5 figures. To appear in IEEE Transaction on Information Theory. References updated and typos correcte

Improved bounds for the rate loss of multiresolution source codes

We present new bounds for the rate loss of multiresolution source codes (MRSCs). Considering an M-resolution code, the rate loss at the ith resolution with distortion D/sub i/ is defined as L/sub i/=R/sub i/-R(D/sub i/), where R/sub i/ is the rate achievable by the MRSC at stage i. This rate loss describes the performance degradation of the MRSC compared to the best single-resolution code with the same distortion. For two-resolution source codes, there are three scenarios of particular interest: (i) when both resolutions are equally important; (ii) when the rate loss at the first resolution is 0 (L/sub 1/=0); (iii) when the rate loss at the second resolution is 0 (L/sub 2/=0). The work of Lastras and Berger (see ibid., vol.47, p.918-26, Mar. 2001) gives constant upper bounds for the rate loss of an arbitrary memoryless source in scenarios (i) and (ii) and an asymptotic bound for scenario (iii) as D/sub 2/ approaches 0. We focus on the squared error distortion measure and (a) prove that for scenario (iii) L/sub 1/<1.1610 for all D/sub 2/<0.7250; (c) tighten the Lastras-Berger bound for scenario (i) from L/sub i//spl les/1/2 to L/sub i/<0.3802, i/spl isin/{1,2}; and (d) generalize the bounds for scenarios (ii) and (iii) to M-resolution codes with M/spl ges/2. We also present upper bounds for the rate losses of additive MRSCs (AMRSCs). An AMRSC is a special MRSC where each resolution describes an incremental reproduction and the kth-resolution reconstruction equals the sum of the first k incremental reproductions. We obtain two bounds on the rate loss of AMRSCs: one primarily good for low-rate coding and another which depends on the source entropy

MAC with Action-Dependent State Information at One Encoder

Problems dealing with the ability to take an action that affects the states of state-dependent communication channels are of timely interest and importance. Therefore, we extend the study of action-dependent channels, which until now focused on point-to-point models, to multiple-access channels (MAC). In this paper, we consider a two-user, state-dependent MAC, in which one of the encoders, called the informed encoder, is allowed to take an action that affects the formation of the channel states. Two independent messages are to be sent through the channel: a common message known to both encoders and a private message known only to the informed encoder. In addition, the informed encoder has access to the sequence of channel states in a non-causal manner. Our framework generalizes previously evaluated settings of state dependent point-to-point channels with actions and MACs with common messages. We derive a single letter characterization of the capacity region for this setting. Using this general result, we obtain and compute the capacity region for the Gaussian action-dependent MAC. The unique methods used in solving the Gaussian case are then applied to obtain the capacity of the Gaussian action-dependent point-to-point channel; a problem was left open until this work. Finally, we establish some dualities between action-dependent channel coding and source coding problems. Specifically, we obtain a duality between the considered MAC setting and the rate distortion model known as "Successive Refinement with Actions". This is done by developing a set of simple duality principles that enable us to successfully evaluate the outcome of one problem given the other.Comment: 1. Parts of this paper appeared in the IEEE International Symposium on Information Theory (ISIT 2012),Cambridge, MA, US, July 2012 and at the IEEE 27th Convention of Electrical and Electronics Engineers in Israel (IEEEI 2012), Nov. 2012. 2. This work has been supported by the CORNET Consortium Israel Ministry for Industry and Commerc