The MIMOME Channel

The MIMOME channel is a Gaussian wiretap channel in which the sender, receiver, and eavesdropper all have multiple antennas. We characterize the secrecy capacity as the saddle-value of a minimax problem. Among other implications, our result establishes that a Gaussian distribution maximizes the secrecy capacity characterization of Csisz{\'a}r and K{\"o}rner when applied to the MIMOME channel. We also determine a necessary and sufficient condition for the secrecy capacity to be zero. Large antenna array analysis of this condition reveals several useful insights into the conditions under which secure communication is possible.Comment: In Proceedings of the 45th Annual Allerton Conference on Communication, Control, and Computing, October 2007, 8 page

Secure Transmission with Multiple Antennas II: The MIMOME Wiretap Channel

The capacity of the Gaussian wiretap channel model is analyzed when there are multiple antennas at the sender, intended receiver and eavesdropper. The associated channel matrices are fixed and known to all the terminals. A computable characterization of the secrecy capacity is established as the saddle point solution to a minimax problem. The converse is based on a Sato-type argument used in other broadcast settings, and the coding theorem is based on Gaussian wiretap codebooks. At high signal-to-noise ratio (SNR), the secrecy capacity is shown to be attained by simultaneously diagonalizing the channel matrices via the generalized singular value decomposition, and independently coding across the resulting parallel channels. The associated capacity is expressed in terms of the corresponding generalized singular values. It is shown that a semi-blind "masked" multi-input multi-output (MIMO) transmission strategy that sends information along directions in which there is gain to the intended receiver, and synthetic noise along directions in which there is not, can be arbitrarily far from capacity in this regime. Necessary and sufficient conditions for the secrecy capacity to be zero are provided, which simplify in the limit of many antennas when the entries of the channel matrices are independent and identically distributed. The resulting scaling laws establish that to prevent secure communication, the eavesdropper needs 3 times as many antennas as the sender and intended receiver have jointly, and that the optimimum division of antennas between sender and intended receiver is in the ratio of 2:1.Comment: To Appear, IEEE Trans. Information Theor

Source Coding with Fixed Lag Side Information

We consider source coding with fixed lag side information at the decoder. We focus on the special case of perfect side information with unit lag corresponding to source coding with feedforward (the dual of channel coding with feedback) introduced by Pradhan. We use this duality to develop a linear complexity algorithm which achieves the rate-distortion bound for any memoryless finite alphabet source and distortion measure.Comment: 10 pages, 3 figure

On the Universality of the Logistic Loss Function

A loss function measures the discrepancy between the true values (observations) and their estimated fits, for a given instance of data. A loss function is said to be proper (unbiased, Fisher consistent) if the fits are defined over a unit simplex, and the minimizer of the expected loss is the true underlying probability of the data. Typical examples are the zero-one loss, the quadratic loss and the Bernoulli log-likelihood loss (log-loss). In this work we show that for binary classification problems, the divergence associated with smooth, proper and convex loss functions is bounded from above by the Kullback-Leibler (KL) divergence, up to a multiplicative normalization constant. It implies that by minimizing the log-loss (associated with the KL divergence), we minimize an upper bound to any choice of loss functions from this set. This property justifies the broad use of log-loss in regression, decision trees, deep neural networks and many other applications. In addition, we show that the KL divergence bounds from above any separable Bregman divergence that is convex in its second argument (up to a multiplicative normalization constant). This result introduces a new set of divergence inequalities, similar to the well-known Pinsker inequality

A refined analysis of the Poisson channel in the high-photon-efficiency regime

We study the discrete-time Poisson channel under the constraint that its average input power (in photons per channel use) must not exceed some constant E. We consider the wideband, high-photon-efficiency extreme where E approaches zero, and where the channel's "dark current" approaches zero proportionally with E. Improving over a previously obtained first-order capacity approximation, we derive a refined approximation, which includes the exact characterization of the second-order term, as well as an asymptotic characterization of the third-order term with respect to the dark current. We also show that pulse-position modulation is nearly optimal in this regime.Comment: Revised version to appear in IEEE Transactions on Information Theor

Secret-key Agreement with Channel State Information at the Transmitter

We study the capacity of secret-key agreement over a wiretap channel with state parameters. The transmitter communicates to the legitimate receiver and the eavesdropper over a discrete memoryless wiretap channel with a memoryless state sequence. The transmitter and the legitimate receiver generate a shared secret key, that remains secret from the eavesdropper. No public discussion channel is available. The state sequence is known noncausally to the transmitter. We derive lower and upper bounds on the secret-key capacity. The lower bound involves constructing a common state reconstruction sequence at the legitimate terminals and binning the set of reconstruction sequences to obtain the secret-key. For the special case of Gaussian channels with additive interference (secret-keys from dirty paper channel) our bounds differ by 0.5 bit/symbol and coincide in the high signal-to-noise-ratio and high interference-to-noise-ratio regimes. For the case when the legitimate receiver is also revealed the state sequence, we establish that our lower bound achieves the the secret-key capacity. In addition, for this special case, we also propose another scheme that attains the capacity and requires only causal side information at the transmitter and the receiver.Comment: 10 Pages, Submitted to IEEE Transactions on Information Forensics and Security, Special Issue on Using the Physical Layer for Securing the Next Generation of Communication System