Keeping Watch

Undergraduate 2-

Entropy Concentration and the Empirical Coding Game

We give a characterization of Maximum Entropy/Minimum Relative Entropy inference by providing two `strong entropy concentration' theorems. These theorems unify and generalize Jaynes' `concentration phenomenon' and Van Campenhout and Cover's `conditional limit theorem'. The theorems characterize exactly in what sense a prior distribution Q conditioned on a given constraint, and the distribution P, minimizing the relative entropy D(P ||Q) over all distributions satisfying the constraint, are `close' to each other. We then apply our theorems to establish the relationship between entropy concentration and a game-theoretic characterization of Maximum Entropy Inference due to Topsoe and others.Comment: A somewhat modified version of this paper was published in Statistica Neerlandica 62(3), pages 374-392, 200

From Physics to Economics: An Econometric Example Using Maximum Relative Entropy

Econophysics, is based on the premise that some ideas and methods from physics can be applied to economic situations. We intend to show in this paper how a physics concept such as entropy can be applied to an economic problem. In so doing, we demonstrate how information in the form of observable data and moment constraints are introduced into the method of Maximum relative Entropy (MrE). A general example of updating with data and moments is shown. Two specific econometric examples are solved in detail which can then be used as templates for real world problems. A numerical example is compared to a large deviation solution which illustrates some of the advantages of the MrE method.Comment: This paper has been accepted in Physica A. 19 Pages, 3 Figure

An integrated remote sensing approach for identifying ecological range sites

A model approach for identifying ecological range sites was applied to high elevation sagebrush-dominated rangelands on Parker Mountain, in south-central Utah. The approach utilizes map information derived from both high altitude color infrared photography and LANDSAT digital data, integrated with soils, geological, and precipitation maps. Identification of the ecological range site for a given area requires an evaluation of all relevant environmental factors which combine to give that site the potential to produce characteristic types and amounts of vegetation. A table is presented which allows the user to determine ecological range site based upon an integrated use of the maps which were prepared. The advantages of identifying ecological range sites through an integrated photo interpretation/LANDSAT analysis are discussed

Effective use of remote sensing products in litigation

A boiled-down version of major legal principles affecting the admissibility of data and products from remote sensing devices is presented. It is suggested that enhancements or classifications of digital data (from scanning devices or from digitized aerial photography) be proffered as evidence in a fashion similar to the manner in which maps from photogrammetric techniques are introduced as evidence. Every effort should be made to illucidate the processes by which digital data are analytically treated or manipulated. Remote sensing expert witnesses should be practiced in providing concise and clear explanations of both data and methods. Special emphasis should be placed on being prepared to provide a detailed accounting of steps taken to calibrate and verify spectral characteristics with ground truth

A short account of a connection of Power Laws to the Information Entropy

We use the formalism of 'Maximum Principle of Shannon's Entropy' to derive the general power law distribution function, using what seems to be a reasonable physical assumption, namely, the demand of a constant mean "internal order" (Boltzmann Entropy) of a complex, self interacting, self organized system. Since the Shannon entropy is equivalent to the Boltzmann's entropy under equilibrium, non interacting conditions, we interpret this result as the complex system making use of its intra-interactions and its non equilibrium in order to keep the equilibrium Boltzmann's entropy constant on the average, thus enabling it an advantage at surviving over less ordered systems, i.e. hinting towards an "Evolution of Structure". We then demonstrate the formalism using a toy model to explain the power laws observed in Cities' populations and show how Zipf's law comes out as a natural special point of the model. We also suggest further directions of theory.Comment: 7 pages, no figures, accepted for publication in "Physica A

A simple derivation and classification of common probability distributions based on information symmetry and measurement scale

Commonly observed patterns typically follow a few distinct families of probability distributions. Over one hundred years ago, Karl Pearson provided a systematic derivation and classification of the common continuous distributions. His approach was phenomenological: a differential equation that generated common distributions without any underlying conceptual basis for why common distributions have particular forms and what explains the familial relations. Pearson's system and its descendants remain the most popular systematic classification of probability distributions. Here, we unify the disparate forms of common distributions into a single system based on two meaningful and justifiable propositions. First, distributions follow maximum entropy subject to constraints, where maximum entropy is equivalent to minimum information. Second, different problems associate magnitude to information in different ways, an association we describe in terms of the relation between information invariance and measurement scale. Our framework relates the different continuous probability distributions through the variations in measurement scale that change each family of maximum entropy distributions into a distinct family.Comment: 17 pages, 0 figure

Minimum aberration designs for discrete choice experiments

A discrete choice experiment (DCE) is a survey method that givesinsight into individual preferences for particular attributes.Traditionally, methods for constructing DCEs focus on identifyingthe individual effect of each attribute (a main effect). However, aninteraction effect between two attributes (a two-factor interaction)better represents real-life trade-offs, and provides us a better understandingof subjects’ competing preferences. In practice it is oftenunknown which two-factor interactions are significant. To address theuncertainty, we propose the use of minimum aberration blockeddesigns to construct DCEs. Such designs maximize the number ofmodels with estimable two-factor interactions in a DCE with two-levelattributes. We further extend the minimum aberration criteria toDCEs with mixed-level attributes and develop some general theoreticalresults