Information theoretic approach to interactive learning

The principles of statistical mechanics and information theory play an important role in learning and have inspired both theory and the design of numerous machine learning algorithms. The new aspect in this paper is a focus on integrating feedback from the learner. A quantitative approach to interactive learning and adaptive behavior is proposed, integrating model- and decision-making into one theoretical framework. This paper follows simple principles by requiring that the observer's world model and action policy should result in maximal predictive power at minimal complexity. Classes of optimal action policies and of optimal models are derived from an objective function that reflects this trade-off between prediction and complexity. The resulting optimal models then summarize, at different levels of abstraction, the process's causal organization in the presence of the learner's actions. A fundamental consequence of the proposed principle is that the learner's optimal action policies balance exploration and control as an emerging property. Interestingly, the explorative component is present in the absence of policy randomness, i.e. in the optimal deterministic behavior. This is a direct result of requiring maximal predictive power in the presence of feedback.Comment: 6 page

The thermodynamics of prediction

A system responding to a stochastic driving signal can be interpreted as computing, by means of its dynamics, an implicit model of the environmental variables. The system's state retains information about past environmental fluctuations, and a fraction of this information is predictive of future ones. The remaining nonpredictive information reflects model complexity that does not improve predictive power, and thus represents the ineffectiveness of the model. We expose the fundamental equivalence between this model inefficiency and thermodynamic inefficiency, measured by dissipation. Our results hold arbitrarily far from thermodynamic equilibrium and are applicable to a wide range of systems, including biomolecular machines. They highlight a profound connection between the effective use of information and efficient thermodynamic operation: any system constructed to keep memory about its environment and to operate with maximal energetic efficiency has to be predictive.Comment: 5 pages, 1 figur

Quasi-Homogeneous Thermodynamics and Black Holes

We propose a generalized thermodynamics in which quasi-homogeneity of the thermodynamic potentials plays a fundamental role. This thermodynamic formalism arises from a generalization of the approach presented in paper [1], and it is based on the requirement that quasi-homogeneity is a non-trivial symmetry for the Pfaffian form $\delta Q_{rev}$. It is shown that quasi-homogeneous thermodynamics fits the thermodynamic features of at least some self-gravitating systems. We analyze how quasi-homogeneous thermodynamics is suggested by black hole thermodynamics. Then, some existing results involving self-gravitating systems are also shortly discussed in the light of this thermodynamic framework. The consequences of the lack of extensivity are also recalled. We show that generalized Gibbs-Duhem equations arise as a consequence of quasi-homogeneity of the thermodynamic potentials. An heuristic link between this generalized thermodynamic formalism and the thermodynamic limit is also discussed.Comment: 39 pages, uses RevteX. Published version (minor changes w.r.t. the original one

Approximation theory in combinatorial optimization. Application to the generalized minimum spanning tree problem

We present an overview of the approximation theory in combinatorial optimization. As an application we consider the Generalized Minimum Spanning Tree (GMST) problem which is defined on an undirected complete graph with the nodes partitioned into clusters and non-negative costs are associated to the edges. This problem is NP-hard and it is known that a polynomial approximation algorithm cannot exist. We present an in-approximability result for the GMST problem and under special assumptions: cost function satisfying the triangle inequality and with cluster sizes bounded by $\rho$, we give an approximation algorithm with ratio $2 \rho$

The Swift-UVOT ultraviolet and visible grism calibration

We present the calibration of the Swift UVOT grisms, of which there are two, providing low-resolution field spectroscopy in the ultraviolet and optical bands respectively. The UV grism covers the range 1700-5000 Angstrom with a spectral resolution of 75 at 2600 Angstrom for source magnitudes of u=10-16 mag, while the visible grism covers the range 2850-6600 Angstrom with a spectral resolution of 100 at 4000 Angstrom for source magnitudes of b=12-17 mag. This calibration extends over all detector positions, for all modes used during operations. The wavelength accuracy (1-sigma) is 9 Angstrom in the UV grism clocked mode, 17 Angstrom in the UV grism nominal mode and 22 Angstrom in the visible grism. The range below 2740 Angstrom in the UV grism and 5200 Angstrom in the visible grism never suffers from overlapping by higher spectral orders. The flux calibration of the grisms includes a correction we developed for coincidence loss in the detector. The error in the coincidence loss correction is less than 20%. The position of the spectrum on the detector only affects the effective area (sensitivity) by a few percent in the nominal modes, but varies substantially in the clocked modes. The error in the effective area is from 9% in the UV grism clocked mode to 15% in the visible grism clocked mode .Comment: 27 pages, 31 figures; MNRAS accepted 23 February 201

Secondary-Structure Design of Proteins by a Backbone Torsion Energy

We propose a new backbone-torsion-energy term in the force field for protein systems. This torsion-energy term is represented by a double Fourier series in two variables, the backbone dihedral angles phi and psi. It gives a natural representation of the torsion energy in the Ramachandran space in the sense that any two-dimensional energy surface periodic in both phi and psi can be expanded by the double Fourier series. We can then easily control secondary-structure-forming tendencies by modifying the torsion-energy surface. For instance, we can increase/decrease the alpha-helix-forming-tendencies by lowering/raising the torsion-energy surface in the alpha-helix region and likewise increase/decrease the beta-sheet-forming tendencies by lowering/raising the surface in the beta-sheet region in the Ramachandran space. We applied our approach to AMBER parm94 and AMBER parm96 force fields and demonstrated that our modifications of the torsion-energy terms resulted in the expected changes of secondary-structure-forming-tendencies by performing folding simulations of alpha-helical and beta-hairpin peptides.Comment: 13 pages, (Revtex4), 5 figure

Network information and connected correlations

Entropy and information provide natural measures of correlation among elements in a network. We construct here the information theoretic analog of connected correlation functions: irreducible $N$--point correlation is measured by a decrease in entropy for the joint distribution of $N$ variables relative to the maximum entropy allowed by all the observed $N-1$ variable distributions. We calculate the ``connected information'' terms for several examples, and show that it also enables the decomposition of the information that is carried by a population of elements about an outside source.Comment: 4 pages, 3 figure

Highly Conducting pi-Conjugated Molecular Junctions Covalently Bonded to Gold Electrodes

We measure electronic conductance through single conjugated molecules bonded to Au metal electrodes with direct Au-C covalent bonds using the scanning tunneling microscope based break-junction technique. We start with molecules terminated with trimethyltin end groups that cleave off in situ resulting in formation of a direct covalent sigma bond between the carbon backbone and the gold metal electrodes. The molecular carbon backbone used in this study consist of a conjugated pi-system that has one terminal methylene group on each end, which bonds to the electrodes, achieving large electronic coupling of the electrodes to the pi-system. The junctions formed with the prototypical example of 1,4-dimethylenebenzene show a conductance approaching one conductance quantum (G0 = 2e2/h). Junctions formed with methylene terminated oligophenyls with two to four phenyl units show a hundred-fold increase in conductance compared with junctions formed with amine-linked oligophenyls. The conduction mechanism for these longer oligophenyls is tunneling as they exhibit an exponential dependence of conductance with oligomer length. In addition, density functional theory based calculations for the Au-xylylene-Au junction show near-resonant transmission with a cross-over to tunneling for the longer oligomers.Comment: Accepted to the Journal of the American Chemical Society as a Communication

Swift-UVOT detection of GRB 050318

We present observations of GRB 050318 by the Ultra-Violet and Optical Telescope (UVOT) on-board the Swift observatory. The data are the first detections of a Gamma Ray Burst (GRB) afterglow decay by the UVOT instrument, launched specifically to open a new window on these transient sources. We showcase UVOTs ability to provide multi-color photometry and the advantages of combining UVOT data with simultaneous and contemporaneous observations from the high-energy detectors on the Swift spacecraft. Multiple filters covering 1,800-6,000 Angstroms reveal a red source with spectral slope steeper than the simultaneous X-ray continuum. Spectral fits indicate that the UVOT colors are consistent with dust extinction by systems at z = 1.2037 and z = 1.4436, redshifts where absorption systems have been pre-identified. However, the data can be most-easily reproduced with models containing a foreground system of neutral gas redshifted by z = 2.8 +/- 0.3. For both of the above scenarios, spectral and decay slopes are, for the most part, consistent with fireball expansion into a uniform medium, provided a cooling break occurs between the energy ranges of the UVOT and Swifts X-ray instrumentation.Comment: 15 pages, 4 figures, ApJ Letters, in pres