Unsupervised Learning via Total Correlation Explanation

Learning by children and animals occurs effortlessly and largely without obvious supervision. Successes in automating supervised learning have not translated to the more ambiguous realm of unsupervised learning where goals and labels are not provided. Barlow (1961) suggested that the signal that brains leverage for unsupervised learning is dependence, or redundancy, in the sensory environment. Dependence can be characterized using the information-theoretic multivariate mutual information measure called total correlation. The principle of Total Cor-relation Ex-planation (CorEx) is to learn representations of data that "explain" as much dependence in the data as possible. We review some manifestations of this principle along with successes in unsupervised learning problems across diverse domains including human behavior, biology, and language.Comment: Invited contribution for IJCAI 2017 Early Career Spotlight. 5 pages, 1 figur

An exploration of two infinite families of snarks

Thesis (M.S.) University of Alaska Fairbanks, 2019In this paper, we generalize a single example of a snark that admits a drawing with even rotational symmetry into two infinite families using a voltage graph construction techniques derived from cyclic Pseudo-Loupekine snarks. We expose an enforced chirality in coloring the underlying 5-pole that generated the known example, and use this fact to show that the infinite families are in fact snarks. We explore the construction of these families in terms of the blowup construction. We show that a graph in either family with rotational symmetry of order m has automorphism group of order m2m⁺¹. The oddness of graphs in both families is determined exactly, and shown to increase linearly with the order of rotational symmetry.Chapter 1: Introduction -- 1.1 General Graph Theory -- Chapter 2: Introduction to Snarks -- 2.1 History -- 2.2 Motivation -- 2.3 Loupekine Snarks and k-poles -- 2.4 Conditions on Triviality -- Chapter 3: The Construction of Two Families of Snarks -- 3.1 Voltage Graphs and Lifts -- 3.2 The Family of Snarks, Fm -- 3.3 A Second Family of Snarks, Rm -- Chapter 4: Results -- 4.1 Proof that the graphs Fm and Rm are Snarks -- 4.2 Interpreting Fm and Rm as Blowup Graphs -- 4.3 Automorphism Group -- 4.4 Oddness -- Chapter 5: Conclusions and Open Questions -- References

Food Environment, Food Store Access, Consumer Behavior, and Diet

Food Environment, Food Deserts, Obesity, Consumer Behavior, Diet, Food Consumption/Nutrition/Food Safety, I18, R50,

Relaxed uncertainty relations and information processing

We consider a range of "theories" that violate the uncertainty relation for anti-commuting observables derived in [JMP, 49, 062105 (2008)]. We first show that Tsirelson's bound for the CHSH inequality can be derived from this uncertainty relation, and that relaxing this relation allows for non-local correlations that are stronger than what can be obtained in quantum mechanics. We continue to construct a hierarchy of related non-signaling theories, and show that on one hand they admit superstrong random access encodings and exponential savings for a particular communication problem, while on the other hand it becomes much harder in these theories to learn a state. We show that the existence of these effects stems from the absence of certain constraints on the expectation values of commuting measurements from our non-signaling theories that are present in quantum theory.Comment: 33 pages, 1 figure. v2: improved notation, to appear in QI

Efficient Estimation of Mutual Information for Strongly Dependent Variables

We demonstrate that a popular class of nonparametric mutual information (MI) estimators based on k-nearest-neighbor graphs requires number of samples that scales exponentially with the true MI. Consequently, accurate estimation of MI between two strongly dependent variables is possible only for prohibitively large sample size. This important yet overlooked shortcoming of the existing estimators is due to their implicit reliance on local uniformity of the underlying joint distribution. We introduce a new estimator that is robust to local non-uniformity, works well with limited data, and is able to capture relationship strengths over many orders of magnitude. We demonstrate the superior performance of the proposed estimator on both synthetic and real-world data.Comment: 13 pages, to appear in International Conference on Artificial Intelligence and Statistics (AISTATS) 201