Empirical processes, typical sequences and coordinated actions in standard Borel spaces

This paper proposes a new notion of typical sequences on a wide class of abstract alphabets (so-called standard Borel spaces), which is based on approximations of memoryless sources by empirical distributions uniformly over a class of measurable "test functions." In the finite-alphabet case, we can take all uniformly bounded functions and recover the usual notion of strong typicality (or typicality under the total variation distance). For a general alphabet, however, this function class turns out to be too large, and must be restricted. With this in mind, we define typicality with respect to any Glivenko-Cantelli function class (i.e., a function class that admits a Uniform Law of Large Numbers) and demonstrate its power by giving simple derivations of the fundamental limits on the achievable rates in several source coding scenarios, in which the relevant operational criteria pertain to reproducing empirical averages of a general-alphabet stationary memoryless source with respect to a suitable function class.Comment: 14 pages, 3 pdf figures; accepted to IEEE Transactions on Information Theor

A Generalized Typicality for Abstract Alphabets

A new notion of typicality for arbitrary probability measures on standard Borel spaces is proposed, which encompasses the classical notions of weak and strong typicality as special cases. Useful lemmas about strong typical sets, including conditional typicality lemma, joint typicality lemma, and packing and covering lemmas, which are fundamental tools for deriving many inner bounds of various multi-terminal coding problems, are obtained in terms of the proposed notion. This enables us to directly generalize lots of results on finite alphabet problems to general problems involving abstract alphabets, without any complicated additional arguments. For instance, quantization procedure is no longer necessary to achieve such generalizations. Another fundamental lemma, Markov lemma, is also obtained but its scope of application is quite limited compared to others. Yet, an alternative theory of typical sets for Gaussian measures, free from this limitation, is also developed. Some remarks on a possibility to generalize the proposed notion for sources with memory are also given.Comment: 44 pages; submitted to IEEE Transactions on Information Theor

Joint Empirical Coordination of Source and Channel

In a decentralized and self-configuring network, the communication devices are considered as autonomous decision-makers that sense their environment and that implement optimal transmission schemes. It is essential that these autonomous devices cooperate and coordinate their actions, to ensure the reliability of the transmissions and the stability of the network. We study a point-to-point scenario in which the encoder and the decoder implement decentralized policies that are coordinated. The coordination is measured in terms of empirical frequency of symbols of source and channel. The encoder and the decoder perform a coding scheme such that the empirical distribution of the symbols is close to a target joint probability distribution. We characterize the set of achievable target probability distributions for a point-to-point source-channel model, in which the encoder is non-causal and the decoder is strictly causal i.e., it returns an action based on the observation of the past channel outputs. The objectives of the encoder and of the decoder, are captured by some utility function, evaluated with respect to the set of achievable target probability distributions. In this article, we investigate the maximization problem of a utility function that is common to both encoder and decoder. We show that the compression and the transmission of information are particular cases of the empirical coordination.Comment: accepted to IEEE Trans. on I

Designing a strategy-proof spot market mechanism with many traders : twenty-two steps to Walrasian equilibrium

To prove their Walrasian equilibrium existence theorem, Arrow and Debreu (1954) devised an abstract economy that Shapley and Shubik (1977) cricitized as a market game because, especially with untrustworthy traders, it fails to determine a credible outcome away from equilibrium. All this earlier work also postulated a Walrasian auctioneer with complete information about traders’ preferences and endowments. To ensure credible outcomes, even in disequilibrium, warehousing is introduced into a multi-stage market game. To achieve Walrasian outcomes in a large economy with incomplete information, even about traders’ endowments, a strategy-proof demand revelation mechanism is considered, and then extended to include warehousing

A Formula for the Capacity of the General Gel'fand-Pinsker Channel

We consider the Gel'fand-Pinsker problem in which the channel and state are general, i.e., possibly non-stationary, non-memoryless and non-ergodic. Using the information spectrum method and a non-trivial modification of the piggyback coding lemma by Wyner, we prove that the capacity can be expressed as an optimization over the difference of a spectral inf- and a spectral sup-mutual information rate. We consider various specializations including the case where the channel and state are memoryless but not necessarily stationary.Comment: Accepted to the IEEE Transactions on Communication

On a Generalised Typicality and Its Applications in Information Theory

Typicality lemmas have been successfully applied in many information theoretical problems. The conventional strong typicality is only defined for finite alphabets. Conditional typicality and Markov lemmas can be obtained for strong typicality. Weak typicality can be defined based on a measurable space without additional constraints, and can be easily defined based on a general stochastic process. However, to the best of our knowledge, no conditional typicality or strong Markov lemmas have been obtained for weak typicality in classic works. As a result, some important coding theorems can only be proved by strong typicality lemmas and using the discretisation-and-approximation-technique. In order to solve the aforementioned problems, we will show that the conditional typicality lemma can be obtained for a generic typicality. We will then define a multivariate typicality for general alphabets and general probability measures on product spaces, based on the relative entropy, which can be a measure of the relevance between multiple sources. We will provide a series of multivariate typicality lemmas, including conditional and joint typicality lemmas, packing and covering lemmas, as well as the strong Markov lemma for our proposed generalised typicality. These typicality lemmas can be used to solve source and channel coding problems in a unified way for finite, continuous, or more general alphabets. We will present some coding theorems with general settings using the generalised multivariate typicality lemmas without using the discretisation-and-approximation technique. Generally, the proofs of the coding theorems in general settings are simpler by using the generalised typicality, than using strong typicality with the discretisation-and-approximation technique

Machine Learning Applications in Spacecraft State and Environment Estimation

There are some problems in spacecraft systems engineering with highly non-linear characteristics and noise where traditional nonlinear estimation techniques fail to yield accurate results. In this thesis, we consider approaching two such problems using kernel methods in machine learning. First, we present a novel formulation and solution to orbit determination of spacecraft and spacecraft groups which can be applied with very weakly observable and highly noisy scenarios. We present a ground station network architecture that can perform orbit determination using Doppler-only observations over the network. Second, we present a machine learning solution to the spacecraft magnetic field interference cancellation problem using distributed magnetometers paving the way for space magnetometry with boom-less CubeSats. We present an approach to orbit determination under very broad conditions that are satisfied for n-body problems. We show that domain generalization and distribution regression techniques can learn to estimate orbits of a group of satellites and identify individual satellites especially with prior understanding of correlations between orbits and provide asymptotic convergence conditions. The approach presented requires only observability of the dynamical system and visibility of the spacecraft and is particularly useful for autonomous spacecraft operations using low-cost ground stations or sensors. With the absence of linear region constraints in the proposed method, we are able to identify orbits that are 800 km apart and reduce orbit uncertainty by 92.5% to under 60 km with noisy Doppler-only measurements. We present an architecture for collaborative orbit determination using networked ground stations. We focus on clusters of satellites deployed in low Earth orbit and measurements of their Doppler-shifted transmissions made by low-gain antenna systems in a software-defined federated ground station network. We develop a network architecture enabling scheduling and tracking with uncertain orbit information. For the proposed network, we also present scheduling and coordinated tracking algorithms for tracking with the purpose of generating measurements for orbit determination. We validate our algorithms and architecture with its application to high fidelity simulations of different networked orbit determination scenarios. We demonstrate how these low-cost ground stations can be used to provide accurate and timely orbital tracking information for large satellite deployments, which is something that remains a challenge for current tracking systems. Last, we present a novel approach and algorithm to the problem of magnetic field interference cancellation of time-varying interference using distributed magnetometers and spacecraft telemetry with particular emphasis on the computational and power requirements of CubeSats. The spacecraft magnetic field interference cancellation problem involves estimation of noise when the number of interfering sources far exceed the number of sensors required to decouple the noise from the signal. The proposed approach models this as a contextual bandit learning problem and the proposed algorithm learns to identify the optimal low-noise combination of distributed magnetometers based on indirect information gained on spacecraft currents through telemetry. Experimental results based on on-orbit spacecraft telemetry shows a 50% reduction in interference compared to the best magnetometer.PHDElectrical Engineering: SystemsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/147688/1/srinag_1.pd