Common Information Components Analysis

We give an information-theoretic interpretation of Canonical Correlation Analysis (CCA) via (relaxed) Wyner's common information. CCA permits to extract from two high-dimensional data sets low-dimensional descriptions (features) that capture the commonalities between the data sets, using a framework of correlations and linear transforms. Our interpretation first extracts the common information up to a pre-selected resolution level, and then projects this back onto each of the data sets. In the case of Gaussian statistics, this procedure precisely reduces to CCA, where the resolution level specifies the number of CCA components that are extracted. This also suggests a novel algorithm, Common Information Components Analysis (CICA), with several desirable features, including a natural extension to beyond just two data sets.Comment: 5 pages, 1 figure. Presented at the 2020 Information Theory and Applications (ITA) Workshop, San Diego, CA, USA, February 2-7, 202

Function Computation over Networks:Efficient Information Processing for Cache and Sensor Applications

This thesis looks at efficient information processing for two network applications: content delivery with caching and collecting summary statistics in wireless sensor networks. Both applications are studied under the same paradigm: function computation over networks, where distributed source nodes cooperatively communicate some functions of individual observations to one or multiple destinations. One approach that always works is to convey all observations and then let the destinations compute the desired functions by themselves. However, if the available communication resources are limited, then revealing less unwanted information becomes critical. Centered on this goal, this thesis develops new coding schemes using information-theoretic tools. The first part of this thesis focuses on content delivery with caching. Caching is a technique that facilitates reallocation of communication resources in order to avoid network congestion during peak-traffic times. An information-theoretic model, termed sequential coding for computing, is proposed to analyze the potential gains offered by the caching technique. For the single-user case, the proposed framework succeeds in verifying the optimality of some simple caching strategies and in providing guidance towards optimal caching strategies. For the two-user case, five representative subproblems are considered, which draw connections with classic source coding problems including the Gray-Wyner system, successive refinement, and the Kaspi/Heegard-Berger problem. Afterwards, the problem of distributed computing with successive refinement is considered. It is shown that if full data recovery is required in the second stage of successive refinement, then any information acquired in the first stage will be useful later in the second stage. The second part of this thesis looks at the collection of summary statistics in wireless sensor networks. Summary statistics include arithmetic mean, median, standard deviation, etc, and they belong to the class of symmetric functions. This thesis develops arithmetic computation coding in order to efficiently perform in-network computation for weighted arithmetic sums and symmetric functions. The developed arithmetic computation coding increases the achievable computation rate from $\Theta((\log L)/L)$ to $\Theta(1/\log L)$, where $L$ is the number of sensors. Finally, this thesis demonstrates that interaction among sensors is beneficial for computation of type-threshold functions, e.g., the maximum and the indicator function, and that a non-vanishing computation rate is achievable

Physical layer security performance study for Two-Hop wireless networks with Buffer-Aided relay selection

指導教員：姜　暁

Joint Spatial and Spectrum Cooperation in Wireless Network.

PhDThe sky-rocketing growth of multimedia infotainment applications and broadband-hungry mobile devices exacerbate the stringent demand for ultra high data rate and more spectrum resources. Along with it, the unbalanced temporal and geographical variations of spectrum usage further inspires those spectral-efficient networks, namely, cognitive radio and heterogeneous cellular networks (HCNs). This thesis focuses on the system design and performance enhancement of cognitive radio (CR) and HCNs. Three different aspects of performance improvement are considered, including link reliability of cognitive radio networks (CNs), security enhancement of CNs, and energy efficiency improvement of CNs and HCNs. First, generalized selection combining (GSC) is proposed as an effective receiver design for interference reduction and reliability improvement of CNs with outdated CSI. A uni- ed way for deriving the distribution of received signal-to-noise ratio (SNR) is developed in underlay spectrum sharing networks subject to interference from the primary trans- mitter (PU-Tx) to the secondary receiver (SU-Rx), maximum transmit power constraint at the secondary transmitter (SU-Tx), and peak interference power constraint at the PU receiver (PU-Rx), is developed. Second, transmit antenna selection with receive generalized selection combining (TAS/GSC) in multi-antenna relay-aided communica- tion is introduced in CNs under Rayleigh fading and Nakagami-m fading. Based on newly derived complex statistical properties of channel power gain of TAS/GSC, exact ergodic capacity and high SNR ergodic capacity are derived over Nakagami-m fading. Third, beamforming and arti cial noise generation (BF&AN) is introduced as a robust scheme to enhance the secure transmission of large-scale spectrum sharing networks with multiple randomly located eavesdroppers (Eves) modeled as homogeneous Poisson Point Process (PPP). Stochastic geometry is applied to model and analyze the impact of i BF&AN on this complex network. Optimal power allocation factor for BF&AN which maximizes the average secrecy rate is further studied under the outage probability con- straint of primary network. Fourth, a new wireless energy harvesting protocol is proposed for underlay cognitive relay networks with the energy-constrained SU-Txs. Exact and asymptotic outage probability, delay-sensitive throughput, and delay-tolerant through- put are derived to explore the tradeoff between the energy harvested from the PU-Txs and the interference caused by the PU-Txs. Fifth, a harvest-then-transmit protocol is proposed in K-tier HCNs with randomly located multiple-antenna base stations (BSs) and single antenna mobile terminals (MTs) modeled as homogeneous PPP. The average received power at MT, the uplink (UL) outage probability, and the UL average ergodic rate are derived to demonstrate the intrinsic relationship between the energy harvested from BSs in the downlink (DL) and the MT performance in the UL. Throughout the thesis, it is shown that link reliability, secrecy performance, and energy efficiency of CNs and HCNs can be signi cantly leveraged by taking advantage of multiple antennas, relays, and wireless energy harvesting

Inferring noncompensatory choice heuristics

Thesis (Ph. D.)--Massachusetts Institute of Technology, Sloan School of Management, Operations Research Center, 2006.Includes bibliographical references (p. 121-128).Human decision making is a topic of great interest to marketers, psychologists, economists, and others. People are often modeled as rational utility maximizers with unlimited mental resources. However, due to the structure of the environment as well as cognitive limitations, people frequently use simplifying heuristics for making quick yet accurate decisions. In this research, we apply discrete optimization to infer from observed data if a person is behaving in way consistent with a choice heuristic (e.g., a noncompensatory lexicographic decision rule). We analyze the computational complexity of several inference related problems, showing that while some are easy due to possessing a greedoid language structure, many are hard and likely do not have polynomial time solutions. For the hard problems we develop an exact dynamic programming algorithm that is robust and scalable in practice, as well as analyze several local search heuristics. We conduct an empirical study of SmartPhone preferences and find that the behavior of many respondents can be explained by lexicographic strategies.(cont.) Furthermore, we find that lexicographic decision rules predict better on holdout data than some standard compensatory models. Finally, we look at a more general form of noncompensatory decision process in the context of consideration set formation. Specifically, we analyze the computational complexity of rule-based consideration set formation, develop solution techniques for inferring rules given observed consideration data, and apply the techniques to a real dataset.by Michael J. Yee.Ph.D

Relaxed Wyner's Common Information

In the problem of coded caching for media delivery, two separate coding opportunities have been identified. The first opportunity is a multi-user advantage and crucially hinges on a public broadcast link in the delivery phase. This has been explored in a plethora of works. The second opportunity has received far less attention and concerns similarities between files in the database. Here, the paradigm is to cache "the similarity" between the files. Upon the request, the encoder refines this by providing the specific details for the requested files. Extending Gray and Wyner's work (1974), it follows that the right measure of file similarity is Wyner's Common Information and its generalizations. The present paper surveys and extends the role of Wyner's Common Information in caching. As a novel result, explicit solutions are found for the Gaussian case under mean-squared error, both for the caching problem as well as for the network considered by Gray and Wyner. Our solution leverages and extends the recent technique of factorization of convex envelopes

Feedback and Common Information: Bounds and Capacity for Gaussian Networks

Network information theory studies the communication of information in a network and considers its fundamental limits. Motivating from the extensive presence of the networks in the daily life, the thesis studies the fundamental limits of particular networks including channel coding such as Gaussian multiple access channel with feedback and source coding such as lossy Gaussian Gray-Wyner network. On one part, we establish the sum-Capacity of the Gaussian multiple-access channel with feedback. The converse bounds that are derived from the dependence-balance argument of Hekstra and Willems meet the achievable scheme introduced by Kramer. Even though the problem is not convex, the factorization of lower convex envelope method that is introduced by Geng and Nair, combined with a Gaussian property are invoked to compute the sum-Capacity. Additionally, we characterize the rate region of lossy Gaussian Gray-Wyner network for symmetric distortion. The problem is not convex, thus the method of factorization of lower convex envelope is used to show the Gaussian optimality of the auxiliaries. Both of the networks, are a long-standing open problem. On the other part, we consider the common information that is introduced by Wyner and the natural relaxation of Wyner's common information. Wyner's common information is a measure that quantifies and assesses the commonality between two random variables. The operational significance of the newly introduced quantity is in Gray-Wyner network. Thus, computing the relaxed Wyner's common information is directly connected with computing the rate region in Gray-Wyner network. We derive a lower bound to Wyner's common information for any given source. The bound meets the exact Wyner's common information for sources that are expressed as sum of a common random variable and Gaussian noises. Moreover, we derive an upper bound on an extended variant of information bottleneck. Finally, we use Wyner's common information and its relaxation as a tool to extract common information between datasets. Thus, we introduce a novel procedure to construct features from data, referred to as Common Information Components Analysis (CICA). We establish that in the case of Gaussian statistics, CICA precisely reduces to Canonical Correlation Analysis (CCA), where the relaxing parameter determines the number of CCA components that are extracted. In this sense, we establish a novel rigorous connection between information measures and CCA, and CICA is a strict generalization of the latter. Moreover, we show that CICA has several desirable features, including a natural extension to beyond just two data sets

Common Information Components Analysis

We give an information-theoretic interpretation of Canonical Correlation Analysis (CCA) via (relaxed) Wyner's common information. CCA permits to extract from two high-dimensional data sets low-dimensional descriptions (features) that capture the commonalities between the data sets, using a framework of correlations and linear transforms. Our interpretation first extracts the common information up to a pre-selected resolution level, and then projects this back onto each of the data sets. In the case of Gaussian statistics, this procedure precisely reduces to CCA, where the resolution level specifies the number of CCA components that are extracted. This also suggests a novel algorithm, Common Information Components Analysis (CICA), with several desirable features, including a natural extension to beyond just two data sets