Information without rolling dice

The deterministic notions of capacity and entropy are studied in the context of communication and storage of information using square-integrable, bandlimited signals subject to perturbation. The $(\epsilon,\delta)$-capacity, that extends the Kolmogorov $\epsilon$-capacity to packing sets of overlap at most $\delta$, is introduced and compared to the Shannon capacity. The functional form of the results indicates that in both Kolmogorov and Shannon's settings, capacity and entropy grow linearly with the number of degrees of freedom, but only logarithmically with the signal to noise ratio. This basic insight transcends the details of the stochastic or deterministic description of the information-theoretic model. For $\delta=0$, the analysis leads to new bounds on the Kolmogorov $\epsilon$-capacity, and to a tight asymptotic expression of the Kolmogorov $\epsilon$-entropy of bandlimited signals. A deterministic notion of error exponent is introduced. Applications of the theory are briefly discussed

Time-Limited Toeplitz Operators on Abelian Groups: Applications in Information Theory and Subspace Approximation

Toeplitz operators are fundamental and ubiquitous in signal processing and information theory as models for linear, time-invariant (LTI) systems. Due to the fact that any practical system can access only signals of finite duration, time-limited restrictions of Toeplitz operators are naturally of interest. To provide a unifying treatment of such systems working on different signal domains, we consider time-limited Toeplitz operators on locally compact abelian groups with the aid of the Fourier transform on these groups. In particular, we survey existing results concerning the relationship between the spectrum of a time-limited Toeplitz operator and the spectrum of the corresponding non-time-limited Toeplitz operator. We also develop new results specifically concerning the eigenvalues of time-frequency limiting operators on locally compact abelian groups. Applications of our unifying treatment are discussed in relation to channel capacity and in relation to representation and approximation of signals

Noiseless Privacy

In this paper, we define noiseless privacy, as a non-stochastic rival to differential privacy, requiring that the outputs of a mechanism (i.e., function composition of a privacy-preserving mapping and a query) can attain only a few values while varying the data of an individual (the logarithm of the number of the distinct values is bounded by the privacy budget). Therefore, the output of the mechanism is not fully informative of the data of the individuals in the dataset. We prove several guarantees for noiselessly-private mechanisms. The information content of the output about the data of an individual, even if an adversary knows all the other entries of the private dataset, is bounded by the privacy budget. The zero-error capacity of memory-less channels using noiselessly private mechanisms for transmission is upper bounded by the privacy budget. The performance of a non-stochastic hypothesis-testing adversary is bounded again by the privacy budget. Finally, assuming that an adversary has access to a stochastic prior on the dataset, we prove that the estimation error of the adversary for individual entries of the dataset is lower bounded by a decreasing function of the privacy budget. In this case, we also show that the maximal information leakage is bounded by the privacy budget. In addition to privacy guarantees, we prove that noiselessly-private mechanisms admit composition theorem and post-processing does not weaken their privacy guarantees. We prove that quantization operators can ensure noiseless privacy if the number of quantization levels is appropriately selected based on the sensitivity of the query and the privacy budget. Finally, we illustrate the privacy merits of noiseless privacy using multiple datasets in energy and transport