Information Spectrum Approach to the Source Channel Separation Theorem

A source-channel separation theorem for a general channel has recently been shown by Aggrawal et. al. This theorem states that if there exist a coding scheme that achieves a maximum distortion level d_{max} over a general channel W, then reliable communication can be accomplished over this channel at rates less then R(d_{max}), where R(.) is the rate distortion function of the source. The source, however, is essentially constrained to be discrete and memoryless (DMS). In this work we prove a stronger claim where the source is general, satisfying only a "sphere packing optimality" feature, and the channel is completely general. Furthermore, we show that if the channel satisfies the strong converse property as define by Han & verdu, then the same statement can be made with d_{avg}, the average distortion level, replacing d_{max}. Unlike the proofs there, we use information spectrum methods to prove the statements and the results can be quite easily extended to other situations

Source Broadcasting to the Masses: Separation has a Bounded Loss

This work discusses the source broadcasting problem, i.e. transmitting a source to many receivers via a broadcast channel. The optimal rate-distortion region for this problem is unknown. The separation approach divides the problem into two complementary problems: source successive refinement and broadcast channel transmission. We provide bounds on the loss incorporated by applying time-sharing and separation in source broadcasting. If the broadcast channel is degraded, it turns out that separation-based time-sharing achieves at least a factor of the joint source-channel optimal rate, and this factor has a positive limit even if the number of receivers increases to infinity. For the AWGN broadcast channel a better bound is introduced, implying that all achievable joint source-channel schemes have a rate within one bit of the separation-based achievable rate region for two receivers, or within $\log_2 T$ bits for $T$ receivers

Linear-Codes-Based Lossless Joint Source-Channel Coding for Multiple-Access Channels

A general lossless joint source-channel coding (JSCC) scheme based on linear codes and random interleavers for multiple-access channels (MACs) is presented and then analyzed in this paper. By the information-spectrum approach and the code-spectrum approach, it is shown that a linear code with a good joint spectrum can be used to establish limit-approaching lossless JSCC schemes for correlated general sources and general MACs, where the joint spectrum is a generalization of the input-output weight distribution. Some properties of linear codes with good joint spectra are investigated. A formula on the "distance" property of linear codes with good joint spectra is derived, based on which, it is further proved that, the rate of any systematic codes with good joint spectra cannot be larger than the reciprocal of the corresponding alphabet cardinality, and any sparse generator matrices cannot yield linear codes with good joint spectra. The problem of designing arbitrary rate coding schemes is also discussed. A novel idea called "generalized puncturing" is proposed, which makes it possible that one good low-rate linear code is enough for the design of coding schemes with multiple rates. Finally, various coding problems of MACs are reviewed in a unified framework established by the code-spectrum approach, under which, criteria and candidates of good linear codes in terms of spectrum requirements for such problems are clearly presented.Comment: 18 pages, 3 figure

Asymptotic Estimates in Information Theory with Non-Vanishing Error Probabilities

This monograph presents a unified treatment of single- and multi-user problems in Shannon's information theory where we depart from the requirement that the error probability decays asymptotically in the blocklength. Instead, the error probabilities for various problems are bounded above by a non-vanishing constant and the spotlight is shone on achievable coding rates as functions of the growing blocklengths. This represents the study of asymptotic estimates with non-vanishing error probabilities. In Part I, after reviewing the fundamentals of information theory, we discuss Strassen's seminal result for binary hypothesis testing where the type-I error probability is non-vanishing and the rate of decay of the type-II error probability with growing number of independent observations is characterized. In Part II, we use this basic hypothesis testing result to develop second- and sometimes, even third-order asymptotic expansions for point-to-point communication. Finally in Part III, we consider network information theory problems for which the second-order asymptotics are known. These problems include some classes of channels with random state, the multiple-encoder distributed lossless source coding (Slepian-Wolf) problem and special cases of the Gaussian interference and multiple-access channels. Finally, we discuss avenues for further research.Comment: Further comments welcom

The effect of point sources on satellite observations of the cosmic microwave background

We study the effect of extragalactic point sources on satellite observations of the cosmic microwave background (CMB). In order to separate the contributions due to different foreground components, a maximum-entropy method is applied to simulated observations by the Planck Surveyor satellite. In addition to point sources, the simulations include emission from the CMB and the kinetic and thermal Sunyaev-Zel'dovich (SZ) effects from galaxy clusters, as well as Galactic dust, free-free and synchrotron emission. We find that the main input components are faithfully recovered and, in particular, that the quality of the CMB reconstruction is only slightly reduced by the presence of point sources. In addition, we find that it is possible to recover accurate point source catalogues at each of the Planck Surveyor observing frequencies.Comment: 12 pages, 9 figures, submitted to MNRA

Complex Random Vectors and ICA Models: Identifiability, Uniqueness and Separability

In this paper the conditions for identifiability, separability and uniqueness of linear complex valued independent component analysis (ICA) models are established. These results extend the well-known conditions for solving real-valued ICA problems to complex-valued models. Relevant properties of complex random vectors are described in order to extend the Darmois-Skitovich theorem for complex-valued models. This theorem is used to construct a proof of a theorem for each of the above ICA model concepts. Both circular and noncircular complex random vectors are covered. Examples clarifying the above concepts are presented.Comment: To appear in IEEE TR-IT March 200

Eigen-Inference for Energy Estimation of Multiple Sources

In this paper, a new method is introduced to blindly estimate the transmit power of multiple signal sources in multi-antenna fading channels, when the number of sensing devices and the number of available samples are sufficiently large compared to the number of sources. Recent advances in the field of large dimensional random matrix theory are used that result in a simple and computationally efficient consistent estimator of the power of each source. A criterion to determine the minimum number of sensors and the minimum number of samples required to achieve source separation is then introduced. Simulations are performed that corroborate the theoretical claims and show that the proposed power estimator largely outperforms alternative power inference techniques.Comment: to appear in IEEE Trans. on Information Theory, 17 pages, 13 figure