On mutual information, likelihood-ratios and estimation error for the additive Gaussian channel

This paper considers the model of an arbitrary distributed signal x observed through an added independent white Gaussian noise w, y=x+w. New relations between the minimal mean square error of the non-causal estimator and the likelihood ratio between y and \omega are derived. This is followed by an extended version of a recently derived relation between the mutual information I(x;y) and the minimal mean square error. These results are applied to derive infinite dimensional versions of the Fisher information and the de Bruijn identity. The derivation of the results is based on the Malliavin calculus.Comment: 21 pages, to appear in the IEEE Transactions on Information Theor

Mutual Information and Minimum Mean-square Error in Gaussian Channels

This paper deals with arbitrarily distributed finite-power input signals observed through an additive Gaussian noise channel. It shows a new formula that connects the input-output mutual information and the minimum mean-square error (MMSE) achievable by optimal estimation of the input given the output. That is, the derivative of the mutual information (nats) with respect to the signal-to-noise ratio (SNR) is equal to half the MMSE, regardless of the input statistics. This relationship holds for both scalar and vector signals, as well as for discrete-time and continuous-time noncausal MMSE estimation. This fundamental information-theoretic result has an unexpected consequence in continuous-time nonlinear estimation: For any input signal with finite power, the causal filtering MMSE achieved at SNR is equal to the average value of the noncausal smoothing MMSE achieved with a channel whose signal-to-noise ratio is chosen uniformly distributed between 0 and SNR

Power allocation and signal labelling on physical layer security

PhD ThesisSecure communications between legitimate users have received considerable attention recently. Transmission cryptography, which introduces secrecy on the network layer, is heavily relied on conventionally to secure communications. However, it is theoretically possible to break the encryption if unlimited computational resource is provided. As a result, physical layer security becomes a hot topic as it provides perfect secrecy from an information theory perspective. The study of physical layer security on real communication system model is challenging and important, as the previous researches are mainly focusing on the Gaussian input model which is not practically implementable. In this thesis, the physical layer security of wireless networks employing finite-alphabet input schemes are studied. In particular, firstly, the secrecy capacity of the single-input single-output (SISO) wiretap channel model with coded modulation (CM) and bit-interleaved coded modulation (BICM) is derived in closed-form, while a fast, sub-optimal power control policy (PCP) is presented to maximize the secrecy capacity performance. Since finite-alphabet input schemes achieve maximum secrecy capacity at medium SNR range, the maximum amount of energy that the destination can harvest from the transmission while satisfying the secrecy rate constraint is computed. Secondly, the effects of mapping techniques on secrecy capacity of BICM scheme are investigated, the secrecy capacity performances of various known mappings are compared on 8PSK, 16QAM and (1,5,10) constellations, showing that Gray mapping obtains lowest secrecy capacity value at high SNRs. We propose a new mapping algorithm, called maximum error event (MEE), to optimize the secrecy capacity over a wide range of SNRs. At low SNR, MEE mapping achieves a lower secrecy rate than other well-known mappings, but at medium-to-high SNRs MEE mapping achieves a significantly higher secrecy rate over a wide range of SNRs. Finally, the secrecy capacity and power allocation algorithm (PA) of finite-alphabet input wiretap channels with decode-and-forward (DF) relays are proposed, the simulation results are compared with the equal power allocation algorithm