Entropy of Overcomplete Kernel Dictionaries

In signal analysis and synthesis, linear approximation theory considers a linear decomposition of any given signal in a set of atoms, collected into a so-called dictionary. Relevant sparse representations are obtained by relaxing the orthogonality condition of the atoms, yielding overcomplete dictionaries with an extended number of atoms. More generally than the linear decomposition, overcomplete kernel dictionaries provide an elegant nonlinear extension by defining the atoms through a mapping kernel function (e.g., the gaussian kernel). Models based on such kernel dictionaries are used in neural networks, gaussian processes and online learning with kernels. The quality of an overcomplete dictionary is evaluated with a diversity measure the distance, the approximation, the coherence and the Babel measures. In this paper, we develop a framework to examine overcomplete kernel dictionaries with the entropy from information theory. Indeed, a higher value of the entropy is associated to a further uniform spread of the atoms over the space. For each of the aforementioned diversity measures, we derive lower bounds on the entropy. Several definitions of the entropy are examined, with an extensive analysis in both the input space and the mapped feature space.Comment: 10 page

Advancing Shannon entropy for measuring diversity in systems

From economic inequality and species diversity to power laws and the analysis of multiple trends and trajectories, diversity within systems is a major issue for science. Part of the challenge is measuring it. Shannon entropy H has been used to re-think diversity within probability distributions, based on the notion of information. However, there are two major limitations to Shannon's approach. First, it cannot be used to compare diversity distributions that have different levels of scale. Second, it cannot be used to compare parts of diversity distributions to the whole. To address these limitations, we introduce a re-normalization of probability distributions based on the notion of case-based entropy Cc as a function of the cumulative probability c. Given a probability density p(x), Cc measures the diversity of the distribution up to a cumulative probability of c, by computing the length or support of an equivalent uniform distribution that has the same Shannon information as the conditional distribution of ^pc(x) up to cumulative probability c. We illustrate the utility of our approach by re-normalizing and comparing three well-known energy distributions in physics, namely, the Maxwell-Boltzmann, Bose-Einstein and Fermi-Dirac distributions for energy of sub-atomic particles. The comparison shows that Cc is a vast improvement over H as it provides a scale-free comparison of these diversity distributions and also allows for a comparison between parts of these diversity distributions

Qualitative evaluation of European Rural Development Policy: Evidence from Comparative Case Studies

Complexity and variable uptake of CAP (Pillar 2) measures and rural diversity of the EU provide significant challenges for evaluation. The rationale of indepth case study analysis as an essential complement to formal evaluation techniques is illustrated with comparative studies of employment impacts of Pillar Two policies in 6 rural areas in different EU member states. Recommendations arising include accelerated shifts from commodity support to measures strengthening non-farm sectors of the rural economy, whilst retaining support for farming adaptation; use of clear structural indicators and local expertise to determine priorities; and integration of Pillar Two policies with other measures in consistent, spatially nested Action Plans for Rural Development which set targets for improvement in economic and demographic performance.rural policies, evaluation, Community/Rural/Urban Development,

Concentration Curves and Have-Statistics for Ecological Analysis of Diversity: Part III: Comparisons of Measures of Diversity

Given the central importance of diversity in ecology and the life sciences more generally, it is not surprising that a variety of methods and measures have been developed to describe and summarize diversity. In the two previous parts of this series of papers, comparisons were drawn between concentration curves and frequency distributions, the most widely used graphical display of variation, and between concentration curves and dominance-diversity curves. This final part of the three paper series compares various statistics that might be used to summarize diversity, with a focus on the usefulness of have-statistics as a supplement to more traditional measures. The first section of our discussion lays out some reasonable criteria and principles that good measures of diversity should satisfy: some traditional measures violate at least one of the criteria; the have-statistics pass the hurdles and have some desirable properties in addition. We then illustrate the use of different measures by way of examples drawn from Howard's studies of bullfrogs (discussed in Part I), the study of species diversity among diatoms (discussed in Part II), an analysis of mating systems of various birds, and a survey of human fertility in 41 countries

Upward Mobility in the American Mountain West

Upward economic and social mobility is an intrinsic element of American society. Data from the Equality of Opportunity Project (EOP) demonstrates that upward mobility is a critical issue for our nation’s metros. An analysis of Mountain West metros and the performances of colleges and universities in this region reveal how the differing economic, demographic, and social characteristics affect mobility. This brief explores upward mobility rates, measures of diversity, levels of domestic and foreign migration, and students’ family household income and their eventual individual incomes. The comparison of postsecondary institutions in Mountain West metros serves as a microcosm to better understand how metros and their universities can best serve our nation’s ever diversifying population

Relationships between adaptive and neutral genetic diversity and ecological structure and functioning: a meta-analysis

Understanding the effects of intraspecific genetic diversity on the structure and functioning of ecological communities is a fundamentally important part of evolutionary ecology and may also have conservation relevance in identifying the situations in which genetic diversity coincides with species‐level diversity. Early studies within this field documented positive relationships between genetic diversity and ecological structure, but recent studies have challenged these findings. Conceptual synthesis has been hampered because studies have used different measures of intraspecific variation (phenotypically adaptive vs. neutral) and have considered different measures of ecological structure in different ecological and spatial contexts. The aim of this study is to strengthen conceptual understanding by providing an empirical synthesis quantifying the relationship between genetic diversity and ecological structure. Here, I present a meta‐analysis of the relationship between genetic diversity within plant populations and the structure and functioning of associated ecological communities (including 423 effect sizes from 70 studies). I used Bayesian meta‐analyses to examine (i) the strength and direction of this relationship, (ii) the extent to which phenotypically adaptive and neutral (molecular) measures of diversity differ in their association with ecological structure and (iii) variation in outcomes among different measures of ecological structure and in different ecological contexts. Effect sizes measuring the relationship between adaptive diversity (genotypic richness) and both community‐ and ecosystem‐level ecological responses were small, but significantly positive. These associations were supported by genetic effects on species richness and productivity, respectively. There was no overall association between neutral genetic diversity and measures of ecological structure, but a positive correlation was observed under a limited set of demographic conditions. These results suggest that adaptive and neutral genetic diversity should not be treated as ecologically equivalent measures of intraspecific variation. Synthesis. This study advances the debate over whether relationships between genetic diversity and ecological structure are either simply positive or negative, by showing how the strength and direction of these relationships changes with different measures of diversity and in different ecological contexts. The results provide a solid foundation for assessing when and where an expanded synthesis between ecology and genetics will be most fruitful

Test Set Diameter: Quantifying the Diversity of Sets of Test Cases

A common and natural intuition among software testers is that test cases need to differ if a software system is to be tested properly and its quality ensured. Consequently, much research has gone into formulating distance measures for how test cases, their inputs and/or their outputs differ. However, common to these proposals is that they are data type specific and/or calculate the diversity only between pairs of test inputs, traces or outputs. We propose a new metric to measure the diversity of sets of tests: the test set diameter (TSDm). It extends our earlier, pairwise test diversity metrics based on recent advances in information theory regarding the calculation of the normalized compression distance (NCD) for multisets. An advantage is that TSDm can be applied regardless of data type and on any test-related information, not only the test inputs. A downside is the increased computational time compared to competing approaches. Our experiments on four different systems show that the test set diameter can help select test sets with higher structural and fault coverage than random selection even when only applied to test inputs. This can enable early test design and selection, prior to even having a software system to test, and complement other types of test automation and analysis. We argue that this quantification of test set diversity creates a number of opportunities to better understand software quality and provides practical ways to increase it.Comment: In submissio