Entropy-based active learning for object recognition

Most methods for learning object categories require large amounts of labeled training data. However, obtaining such data can be a difficult and time-consuming endeavor. We have developed a novel, entropy-based ldquoactive learningrdquo approach which makes significant progress towards this problem. The main idea is to sequentially acquire labeled data by presenting an oracle (the user) with unlabeled images that will be particularly informative when labeled. Active learning adaptively prioritizes the order in which the training examples are acquired, which, as shown by our experiments, can significantly reduce the overall number of training examples required to reach near-optimal performance. At first glance this may seem counter-intuitive: how can the algorithm know whether a group of unlabeled images will be informative, when, by definition, there is no label directly associated with any of the images? Our approach is based on choosing an image to label that maximizes the expected amount of information we gain about the set of unlabeled images. The technique is demonstrated in several contexts, including improving the efficiency of Web image-search queries and open-world visual learning by an autonomous agent. Experiments on a large set of 140 visual object categories taken directly from text-based Web image searches show that our technique can provide large improvements (up to 10 x reduction in the number of training examples needed) over baseline techniques

Separability conditions from entropic uncertainty relations

We derive a collection of separability conditions for bipartite systems of dimensions d X d which is based on the entropic version of the uncertainty relations. A detailed analysis of the two-qubit case is given by comparing the new separability conditions with existing criteria.Comment: 10 pages, 2 figure (Typos removed. To appear in Phys. Rev. A

Considerate Approaches to Achieving Sufficiency for ABC model selection

For nearly any challenging scientific problem evaluation of the likelihood is problematic if not impossible. Approximate Bayesian computation (ABC) allows us to employ the whole Bayesian formalism to problems where we can use simulations from a model, but cannot evaluate the likelihood directly. When summary statistics of real and simulated data are compared --- rather than the data directly --- information is lost, unless the summary statistics are sufficient. Here we employ an information-theoretical framework that can be used to construct (approximately) sufficient statistics by combining different statistics until the loss of information is minimized. Such sufficient sets of statistics are constructed for both parameter estimation and model selection problems. We apply our approach to a range of illustrative and real-world model selection problems

A classification of entanglement in three-qubit systems

We present a classification of three-qubit states based in their three-qubit and reduced two-qubit entanglements. For pure states these criteria can be easily implemented, and the different types can be related with sets of equivalence classes under Local Unitary operations. For mixed states characterization of full tripartite entanglement is not yet solved in general; some partial results will be presented here.Comment: Shortened version. Accepted in EPJ

A mean field method with correlations determined by linear response

We introduce a new mean-field approximation based on the reconciliation of maximum entropy and linear response for correlations in the cluster variation method. Within a general formalism that includes previous mean-field methods, we derive formulas improving upon, e.g., the Bethe approximation and the Sessak-Monasson result at high temperature. Applying the method to direct and inverse Ising problems, we find improvements over standard implementations.Comment: 15 pages, 8 figures, 9 appendices, significant expansion on versions v1 and v

Taxonomic evidence applying intelligent information algorithm and the principle of maximum entropy: the case of asteroids families

The Numeric Taxonomy aims to group operational taxonomic units in clusters (OTUs or taxons or taxa), using the denominated structure analysis by means of numeric methods. These clusters that constitute families are the purpose of this series of projects and they emerge of the structural analysis, of their phenotypical characteristic, exhibiting the relationships in terms of grades of similarity of the OTUs, employing tools such as i) the Euclidean distance and ii) nearest neighbor techniques. Thus taxonomic evidence is gathered so as to quantify the similarity for each pair of OTUs (pair-group method) obtained from the basic data matrix and in this way the significant concept of spectrum of the OTUs is introduced, being based the same one on the state of their characters. A new taxonomic criterion is thereby formulated and a new approach to Computational Taxonomy is presented, that has been already employed with reference to Data Mining, when apply of Machine Learning techniques, in particular to the C4.5 algorithms, created by Quinlan, the degree of efficiency achieved by the TDIDT family´s algorithms when are generating valid models of the data in classification problems with the Gain of Entropy through Maximum Entropy Principle.Fil: Perichinsky, Gregorio. Universidad de Buenos Aires. Facultad de Ingeniería; ArgentinaFil: Jiménez Rey, Elizabeth Miriam. Universidad de Buenos Aires. Facultad de Ingeniería; ArgentinaFil: Grossi, María Delia. Universidad de Buenos Aires. Facultad de Ingeniería; ArgentinaFil: Vallejos, Félix Anibal. Universidad de Buenos Aires. Facultad de Ingeniería; Argentina. Universidad Nacional de La Plata. Facultad de Ciencias Astronómicas y Geofísicas; ArgentinaFil: Servetto, Arturo Carlos. Universidad de Buenos Aires. Facultad de Ingeniería; ArgentinaFil: Orellana, Rosa Beatriz. Universidad Nacional de La Plata. Facultad de Ciencias Astronómicas y Geofísicas; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Plastino, Ángel Luis. Universidad Nacional de La Plata. Facultad de Ciencias Exactas. Departamento de Física; Argentin

Information-theoretic inference of common ancestors

A directed acyclic graph (DAG) partially represents the conditional independence structure among observations of a system if the local Markov condition holds, that is, if every variable is independent of its non-descendants given its parents. In general, there is a whole class of DAGs that represents a given set of conditional independence relations. We are interested in properties of this class that can be derived from observations of a subsystem only. To this end, we prove an information theoretic inequality that allows for the inference of common ancestors of observed parts in any DAG representing some unknown larger system. More explicitly, we show that a large amount of dependence in terms of mutual information among the observations implies the existence of a common ancestor that distributes this information. Within the causal interpretation of DAGs our result can be seen as a quantitative extension of Reichenbach's Principle of Common Cause to more than two variables. Our conclusions are valid also for non-probabilistic observations such as binary strings, since we state the proof for an axiomatized notion of mutual information that includes the stochastic as well as the algorithmic version.Comment: 18 pages, 4 figure