Sequential Necessary and Sufficient Conditions for Capacity Achieving Distributions of Channels with Memory and Feedback

We derive sequential necessary and sufficient conditions for any channel input conditional distribution ${\cal P}_{0,n}\triangleq\{P_{X_t|X^{t-1},Y^{t-1}}:~t=0,\ldots,n\}$ to maximize the finite-time horizon directed information defined by $$C^{FB}_{X^n \rightarrow Y^n} \triangleq \sup_{{\cal P}_{0,n}} I(X^n\rightarrow{Y^n}),~~~ I(X^n \rightarrow Y^n) =\sum_{t=0}^n{I}(X^t;Y_t|Y^{t-1})$$ for channel distributions $\{P_{Y_t|Y^{t-1},X_t}:~t=0,\ldots,n\}$ and $\{P_{Y_t|Y_{t-M}^{t-1},X_t}:~t=0,\ldots,n\}$, where $Y^t\triangleq\{Y_0,\ldots,Y_t\}$ and $X^t\triangleq\{X_0,\ldots,X_t\}$ are the channel input and output random processes, and $M$ is a finite nonnegative integer. \noi We apply the necessary and sufficient conditions to application examples of time-varying channels with memory and we derive recursive closed form expressions of the optimal distributions, which maximize the finite-time horizon directed information. Further, we derive the feedback capacity from the asymptotic properties of the optimal distributions by investigating the limit $$C_{X^\infty \rightarrow Y^\infty}^{FB} \triangleq \lim_{n \longrightarrow \infty} \frac{1}{n+1} C_{X^n \rightarrow Y^n}^{FB}$$ without any \'a priori assumptions, such as, stationarity, ergodicity or irreducibility of the channel distribution. The necessary and sufficient conditions can be easily extended to a variety of channels with memory, beyond the ones considered in this paper.Comment: 57 pages, 9 figures, part of the paper was accepted for publication in the proceedings of the IEEE International Symposium on Information Theory (ISIT), Barcelona, Spain 10-15 July, 2016 (Date of submission of the conference paper: 25/1/2016

Information Nonanticipative Rate Distortion Function and Its Applications

This paper investigates applications of nonanticipative Rate Distortion Function (RDF) in a) zero-delay Joint Source-Channel Coding (JSCC) design based on average and excess distortion probability, b) in bounding the Optimal Performance Theoretically Attainable (OPTA) by noncausal and causal codes, and computing the Rate Loss (RL) of zero-delay and causal codes with respect to noncausal codes. These applications are described using two running examples, the Binary Symmetric Markov Source with parameter p, (BSMS(p)) and the multidimensional partially observed Gaussian-Markov source. For the multidimensional Gaussian-Markov source with square error distortion, the solution of the nonanticipative RDF is derived, its operational meaning using JSCC design via a noisy coding theorem is shown by providing the optimal encoding-decoding scheme over a vector Gaussian channel, and the RL of causal and zero-delay codes with respect to noncausal codes is computed. For the BSMS(p) with Hamming distortion, the solution of the nonanticipative RDF is derived, the RL of causal codes with respect to noncausal codes is computed, and an uncoded noisy coding theorem based on excess distortion probability is shown. The information nonanticipative RDF is shown to be equivalent to the nonanticipatory epsilon-entropy, which corresponds to the classical RDF with an additional causality or nonanticipative condition imposed on the optimal reproduction conditional distribution.Comment: 34 pages, 12 figures, part of this paper was accepted for publication in IEEE International Symposium on Information Theory (ISIT), 2014 and in book Coordination Control of Distributed Systems of series Lecture Notes in Control and Information Sciences, 201

Applications of Information Nonanticipative Rate Distortion Function

The objective of this paper is to further investigate various applications of information Nonanticipative Rate Distortion Function (NRDF) by discussing two working examples, the Binary Symmetric Markov Source with parameter $p$ (BSMS($p$)) with Hamming distance distortion, and the multidimensional partially observed Gaussian-Markov source. For the BSMS($p$), we give the solution to the NRDF, and we use it to compute the Rate Loss (RL) of causal codes with respect to noncausal codes. For the multidimensional Gaussian-Markov source, we give the solution to the NRDF, we show its operational meaning via joint source-channel matching over a vector of parallel Gaussian channels, and we compute the RL of causal and zero-delay codes with respect to noncausal codes.Comment: 5 pages, 3 figures, accepted for publication in IEEE International Symposium on Information Theory (ISIT) proceedings, 201

Causal Rate Distortion Function on Abstract Alphabets: Optimal Reconstruction and Properties

A causal rate distortion function with a general fidelity criterion is formulated on abstract alphabets and a coding theorem is derived. Existence of the minimizing kernel is shown using the topology of weak convergence of probability measures. The optimal reconstruction kernel is derived, which is causal, and certain properties of the causal rate distortion function are presented.Comment: 5 pages, Submitted to Internation Symposium on Information Theory(ISIT) 201

Second-line antiretroviral therapy in a workplace and community-based treatment programme in South Africa: determinants of virological outcome.

: Background: As antiretroviral treatment (ART) programmes in resource-limited settings mature, more patients are experiencing virological failure. Without resistance testing, deciding who should switch to second-line ART can be difficult. The consequences for second-line outcomes are unclear. In a workplace- and community-based multi-site programme, with 6-monthly virological monitoring, we describe outcomes and predictors of viral suppression on second-line, protease inhibitor-based ART.Methods: We used prospectively collected clinic data from patients commencing first-line ART between 1/1/03 and 31/12/08 to construct a study cohort of patients switched to second-line ART in the presence of a viral load (VL) ?400 copies/ml. Predictors of VL<400 copies/ml within 15 months of switch were assessed using modified Poisson regression to estimate risk ratios.Results: 205 workplace patients (91.7% male; median age 43 yrs) and 212 community patients (38.7% male; median age 36 yrs) switched regimens. At switch compared to community patients, workplace patients had a longer duration of viraemia, higher VL, lower CD4 count, and higher reported non-adherence on first-line ART. Non-adherence was the reported reason for switching in a higher proportion of workplace patients. Following switch, 48.3% (workplace) and 72.0% (community) achieved VL<400, with non-adherence (17.9% vs. 1.4%) and virological rebound (35.6% vs. 13.2% with available measures) reported more commonly in the workplace programme. In adjusted analysis of the workplace programme, lower switch VL and younger age were associated with VL<400. In the community programme, shorter duration of viraemia, higher CD4 count and transfers into programme on ART were associated with VL<400.Conclusion: High levels of viral suppression on second-line ART can be, but are not always, achieved in multi-site treatment programmes with both individual- and programme-level factors influencing outcomes. Strategies to support both healthcare workers and patients during this switch period need to be evaluated; sub-optimal adherence, particularly in the workplace programme must be addressed

Capacity of Binary State Symmetric Channel with and without Feedback and Transmission Cost

We consider a unit memory channel, called Binary State Symmetric Channel (BSSC), in which the channel state is the modulo2 addition of the current channel input and the previous channel output. We derive closed form expressions for the capacity and corresponding channel input distribution, of this BSSC with and without feedback and transmission cost. We also show that the capacity of the BSSC is not increased by feedback, and it is achieved by a first order symmetric Markov process