Source Coding in Networks with Covariance Distortion Constraints

We consider a source coding problem with a network scenario in mind, and formulate it as a remote vector Gaussian Wyner-Ziv problem under covariance matrix distortions. We define a notion of minimum for two positive-definite matrices based on which we derive an explicit formula for the rate-distortion function (RDF). We then study the special cases and applications of this result. We show that two well-studied source coding problems, i.e. remote vector Gaussian Wyner-Ziv problems with mean-squared error and mutual information constraints are in fact special cases of our results. Finally, we apply our results to a joint source coding and denoising problem. We consider a network with a centralized topology and a given weighted sum-rate constraint, where the received signals at the center are to be fused to maximize the output SNR while enforcing no linear distortion. We show that one can design the distortion matrices at the nodes in order to maximize the output SNR at the fusion center. We thereby bridge between denoising and source coding within this setup

Rate-Exponent Region for a Class of Distributed Hypothesis Testing Against Conditional Independence Problems

We study a class of $K$-encoder hypothesis testing against conditional independence problems. Under the criterion that stipulates minimization of the Type II error subject to a (constant) upper bound $\epsilon$ on the Type I error, we characterize the set of encoding rates and exponent for both discrete memoryless and memoryless vector Gaussian settings. For the DM setting, we provide a converse proof and show that it is achieved using the Quantize-Bin-Test scheme of Rahman and Wagner. For the memoryless vector Gaussian setting, we develop a tight outer bound by means of a technique that relies on the de Bruijn identity and the properties of Fisher information. In particular, the result shows that for memoryless vector Gaussian sources the rate-exponent region is exhausted using the Quantize-Bin-Test scheme with \textit{Gaussian} test channels; and there is \textit{no} loss in performance caused by restricting the sensors' encoders not to employ time sharing. Furthermore, we also study a variant of the problem in which the source, not necessarily Gaussian, has finite differential entropy and the sensors' observations noises under the null hypothesis are Gaussian. For this model, our main result is an upper bound on the exponent-rate function. The bound is shown to mirror a corresponding explicit lower bound, except that the lower bound involves the source power (variance) whereas the upper bound has the source entropy power. Part of the utility of the established bound is for investigating asymptotic exponent/rates and losses incurred by distributed detection as function of the number of sensors.Comment: Submitted for publication to the IEEE Transactions of Information Theory. arXiv admin note: substantial text overlap with arXiv:1904.03028, arXiv:1811.0393

Bounds on the Sum-Rate of MIMO Causal Source Coding Systems with Memory under Spatio-Temporal Distortion Constraints

In this paper, we derive lower and upper bounds on the OPTA of a two-user multi-input multi-output (MIMO) causal encoding and causal decoding problem. Each user’s source model is described by a multidimensional Markov source driven by additive i.i.d. noise process subject to three classes of spatio-temporal distortion constraints. To characterize the lower bounds, we use state augmentation techniques and a data processing theorem, which recovers a variant of rate distortion function as an information measure known in the literature as nonanticipatory ϵ-entropy, sequential or nonanticipative RDF. We derive lower bound characterizations for a system driven by an i.i.d. Gaussian noise process, which we solve using the SDP algorithm for all three classes of distortion constraints. We obtain closed form solutions when the system’s noise is possibly non-Gaussian for both users and when only one of the users is described by a source model driven by a Gaussian noise process. To obtain the upper bounds, we use the best linear forward test channel realization that corresponds to the optimal test channel realization when the system is driven by a Gaussian noise process and apply a sequential causal DPCM-based scheme with a feedback loop followed by a scaled ECDQ scheme that leads to upper bounds with certain performance guarantees. Then, we use the linear forward test channel as a benchmark to obtain upper bounds on the OPTA, when the system is driven by an additive i.i.d. non-Gaussian noise process. We support our framework with various simulation studies

Information-theoretic Secrecy in Multi-user Channels

Inherent openness of the wireless medium imposes stronger challenges on the security of wireless communications. Information-theoretic security addresses these challenges at the physical layer by using tools from wireless communication theory, signal processing and information theory. In information-theoretic security, physical layer communication is intelligently designed to exploit the characteristics of the wireless medium, such as fading, interference, cooperation, and multi-dimensional signaling, in order to provide or improve security. In this dissertation, we study the security of several fundamental wireless network configurations from an information-theoretic perspective. First, we study the Gaussian multiple-input multiple-output (MIMO) wiretap channel. In this channel, the transmitter sends a common message to both the legitimate user and the eavesdropper. In addition to the common message, a private message is sent only to the legitimate user, which needs to be kept hidden as much as possible from the eavesdropper. We obtain the entire capacity-equivocation region for this channel model. In particular, we show the sufficiency of jointly Gaussian auxiliary random variables and channel input to evaluate the existing single-letter description of the capacity-equivocation region due to Csiszar-Korner. Next, we study the secure broadcasting problem, where a transmitter wants to have secure communication with multiple legitimate users in the presence of an external eavesdropper. We study several special cases of the secure broadcasting problem. First, we consider the degraded multi-receiver wiretap channel, and establish its secrecy capacity region. Second, we consider the parallel less noisy multi-receiver wiretap channel, and obtain its common message secrecy capacity and sum secrecy capacity. Third, we consider the parallel degraded multi-receiver wiretap channel for the two-user and two-sub-channel case, and obtain its entire secrecy capacity region. Finally, we consider a parallel channel model with two sub-channels, where the transmitter can use only one of the subchannels at any time, and characterize its secrecy capacity region. Then, we study the two-user Gaussian MIMO broadcast channel with common and confidential messages. In this channel model, the transmitter sends a common message to both users, and a confidential message to each user which needs to be kept perfectly secret from the other user. We obtain the entire capacity region of this channel. We also explore the connections between this channel model and its non-confidential counterpart, i.e., the Gaussian MIMO broadcast channel with common and private message. Next, we consider the Gaussian MIMO multi-receiver wiretap channel and obtain its secrecy capacity region for the most general case. We first show that even for the single-input single-output (SISO) case, existing converse techniques fall short of proving the secrecy capacity region, to emphasize the need for a new proof technique, which we develop by using the relationships between the Fisher information and the differential entropy. Using this new proof technique, we obtain the secrecy capacity region of the degraded MIMO channel. We then establish the secrecy capacity region of the general MIMO channel by using the channel enhancement technique in conjunction with the capacity result we obtained for the degraded MIMO channel. For the general MIMO channel, we show that dirty-paper coding (DPC) combined with stochastic encoding attains the entire secrecy capacity region. Then, we study the multi-receiver wiretap channel for a more general scenario, where, in addition to confidential messages, the transmitter sends public messages to the legitimate users, on which there are no secrecy constraints. First, we consider the degraded discrete memoryless channel, and obtain inner and outer bounds for the capacity region. These inner and outer bounds match for certain cases, providing the capacity region. Second, we obtain an inner bound for the general discrete memoryless channel by using Marton's inner bound. Third, we consider the degraded Gaussian MIMO channel, and show that jointly Gaussian auxiliary random variables and channel input are sufficient to exhaust the inner and outer bounds. Finally, we provide an inner bound for the capacity region of the general Gaussian MIMO channel. Next, we focus on the multiple access wiretap (MAC-WT) channel whose capacity region is unknown. We consider a special class of MAC-WT channels which we call the weak eavesdropper class, where each user's link to the legitimate receiver is stronger than its link to the eavesdropper. For this class of channels, we develop an outer bound for the secrecy capacity region, which partially matches the achievable region in an n-letter form. We evaluate a looser version of our outer bound for the Gaussian case, and show that our outer bound is within 0.5 bits/channel use of the achievable rates along the individual secrecy rates for all weak eavesdropper Gaussian MAC-WT. Then, we investigate the effects of user cooperation on the secrecy of broadcast channels by considering the cooperative relay broadcast channel (CRBC). We propose an achievable scheme that combines Marton's coding scheme for broadcast channels and Cover and El Gamal's compress-and-forward (CAF) scheme for relay channels. For the Gaussian CRBC, we show that both users can have positive secrecy rates, which is not possible for scalar Gaussian broadcast channels without cooperation. We further investigate the effects of user cooperation on secrecy by considering the multiple access channel with generalized feedback (MAC-GF), which can be viewed as the MAC-dual of the CRBC. We propose a CAF-based achievable secrecy rate region for the MAC-GF. Specializing our results to a Gaussian MAC-GF, we present numerical results which demonstrate that cooperation can improve secrecy for the MAC-GF. Next, we study the two-user one-eavesdropper discrete memoryless compound wiretap channel, and provide the best known lower bound for the secrecy capacity of this compound channel. We evaluate this achievable secrecy rate for the Gaussian MIMO case by using DPC. We show that this achievable secrecy rate achieves at least half of the secrecy capacity of this Gaussian MIMO compound wiretap channel, and also attains the secrecy capacity when the eavesdropper is degraded with respect to one of the two users. Then, we study the degraded compound multi-receiver wiretap channel (DCMRWC), which, in addition to a group of eavesdroppers, has two groups of users, namely the stronger group and the weaker group. We study two different communication scenarios for this channel. In the first scenario, there is only one eavesdropper, and the transmitter sends a confidential message to each group of legitimate users while keeping both messages secret from the eavesdropper. In the second scenario, we study the DCMRWC with layered messages without any restriction on the number of eavesdroppers. For both scenarios, we obtain the secrecy capacity region for the discrete memoryless channel, the parallel channel, and the Gaussian parallel channel. For the Gaussian MIMO channel, we obtain the secrecy capacity region when there is only one user in the second group. Next, we study the two-user fading broadcast channel and obtain its ergodic secrecy capacity region. We show that, thanks to fading, both users can have simultaneous secure communication with the transmitter, although this is not possible in the scalar non-fading Gaussian broadcast channel where only one user can have secure communication. This simultaneous secrecy of both users is achieved by an opportunistic communication scheme, in which, at each time instant, the transmitter communicates with the user having a better channel gain. Then, we study the secure lossy transmission of a vector Gaussian source to a legitimate user in the presence of an eavesdropper, where both the legitimate user and the eavesdropper have vector Gaussian side information. We obtain an outer bound for the rate, equivocation and distortion region. Moreover, we obtain the maximum equivocation at the eavesdropper when there is no constraint on the transmission rate. By using this maximum equivocation result, we show two facts. First, for this problem, in general, Wyner-Ziv scheme is suboptimal, although, it is optimal in the absence of an eavesdropper. And, second, even when there is no transmission rate constraint, an uncoded transmission scheme is suboptimal; the presence of an eavesdropper necessitates the use of a coded scheme to attain the maximum equivocation. Finally, we revisit the secure lossy source coding problem. In all works on this problem, either the equivocation of the source at the eavesdropper or the equivocation of the legitimate user's reconstruction of the source at the eavesdropper is used to measure secrecy. We first propose the relative equivocation of the source at the eavesdropper with respect to the legitimate user as a new secrecy measure. We argue that this new secrecy measure is the one that corresponds to the natural generalization of the equivocation in a wiretap channel to the context of secure lossy source coding. Under this new secrecy measure, we provide a single-letter description of the rate, relative equivocation and distortion region, as well as its specializations to degraded and reversely degraded cases. We investigate the relationships between the optimal scheme that attains this region and the Wyner-Ziv scheme