An Enhanced Features Extractor for a Portfolio of Constraint Solvers

Recent research has shown that a single arbitrarily efficient solver can be significantly outperformed by a portfolio of possibly slower on-average solvers. The solver selection is usually done by means of (un)supervised learning techniques which exploit features extracted from the problem specification. In this paper we present an useful and flexible framework that is able to extract an extensive set of features from a Constraint (Satisfaction/Optimization) Problem defined in possibly different modeling languages: MiniZinc, FlatZinc or XCSP. We also report some empirical results showing that the performances that can be obtained using these features are effective and competitive with state of the art CSP portfolio techniques

MDL Convergence Speed for Bernoulli Sequences

The Minimum Description Length principle for online sequence estimation/prediction in a proper learning setup is studied. If the underlying model class is discrete, then the total expected square loss is a particularly interesting performance measure: (a) this quantity is finitely bounded, implying convergence with probability one, and (b) it additionally specifies the convergence speed. For MDL, in general one can only have loss bounds which are finite but exponentially larger than those for Bayes mixtures. We show that this is even the case if the model class contains only Bernoulli distributions. We derive a new upper bound on the prediction error for countable Bernoulli classes. This implies a small bound (comparable to the one for Bayes mixtures) for certain important model classes. We discuss the application to Machine Learning tasks such as classification and hypothesis testing, and generalization to countable classes of i.i.d. models.Comment: 28 page

Self-Modification of Policy and Utility Function in Rational Agents

Any agent that is part of the environment it interacts with and has versatile actuators (such as arms and fingers), will in principle have the ability to self-modify -- for example by changing its own source code. As we continue to create more and more intelligent agents, chances increase that they will learn about this ability. The question is: will they want to use it? For example, highly intelligent systems may find ways to change their goals to something more easily achievable, thereby `escaping' the control of their designers. In an important paper, Omohundro (2008) argued that goal preservation is a fundamental drive of any intelligent system, since a goal is more likely to be achieved if future versions of the agent strive towards the same goal. In this paper, we formalise this argument in general reinforcement learning, and explore situations where it fails. Our conclusion is that the self-modification possibility is harmless if and only if the value function of the agent anticipates the consequences of self-modifications and use the current utility function when evaluating the future.Comment: Artificial General Intelligence (AGI) 201

On the Computability of Solomonoff Induction and Knowledge-Seeking

Solomonoff induction is held as a gold standard for learning, but it is known to be incomputable. We quantify its incomputability by placing various flavors of Solomonoff's prior M in the arithmetical hierarchy. We also derive computability bounds for knowledge-seeking agents, and give a limit-computable weakly asymptotically optimal reinforcement learning agent.Comment: ALT 201

Optimistic Agents are Asymptotically Optimal

We use optimism to introduce generic asymptotically optimal reinforcement learning agents. They achieve, with an arbitrary finite or compact class of environments, asymptotically optimal behavior. Furthermore, in the finite deterministic case we provide finite error bounds.Comment: 13 LaTeX page

Bayesian reinforcement learning with exploration

We consider a general reinforcement learning problem and show that carefully combining the Bayesian optimal policy and an exploring policy leads to minimax sample-complexity bounds in a very general class of (history-based) environments. We also prove lower bounds and show that the new algorithm displays adaptive behaviour when the environment is easier than worst-case

Electron correlation in C_(4N+2) carbon rings: aromatic vs. dimerized structures

The electronic structure of C_(4N+2) carbon rings exhibits competing many-body effects of Huckel aromaticity, second-order Jahn-Teller and Peierls instability at large sizes. This leads to possible ground state structures with aromatic, bond angle or bond length alternated geometry. Highly accurate quantum Monte Carlo results indicate the existence of a crossover between C_10 and C_14 from bond angle to bond length alternation. The aromatic isomer is always a transition state. The driving mechanism is the second-order Jahn-Teller effect which keeps the gap open at all sizes.Comment: Submitted for publication: 4 pages, 3 figures. Corrected figure

Time consistent discounting

A possibly immortal agent tries to maximise its summed discounted rewards over time, where discounting is used to avoid infinite utilities and encourage the agent to value current rewards more than future ones. Some commonly used discount functions lead to time-inconsistent behavior where the agent changes its plan over time. These inconsistencies can lead to very poor behavior. We generalise the usual discounted utility model to one where the discount function changes with the age of the agent. We then give a simple characterisation of time-(in)consistent discount functions and show the existence of a rational policy for an agent that knows its discount function is time-inconsistent

Comparison of single-nucleotide polymorphisms and microsatellites in detecting quantitative trait loci for alcoholism: The Collaborative Study on the Genetics of Alcoholism

BACKGROUND: The feasibility of effectively analyzing high-density single nucleotide polymorphism (SNP) maps in whole genome scans of complex traits is not known. The purpose of this study was to compare variance components linkage results using different density marker maps in data from the Collaborative Study on the Genetics of Alcoholism (COGA). Marker maps having an average spacing of 10 cM (microsatellite), 0.78 cM (SNP1), and 0.31 cM (SNP2) were used to identify quantitative trait loci (QTLs) affecting maximum number of alcoholic drinks consumed in a 24-hour period (lnmaxalc). RESULTS: Heritability of lnmaxalc was estimated to be 15%. Multipoint variance components linkage analysis revealed similar linkage patterns among the three marker panels, with the SNP maps consistently yielding higher LOD scores. Robust LOD scores > 1.0 were observed on chromosomes 1 and 13 for all three marker maps. Additional LODs > 1.0 were observed on chromosome 4 with both SNP maps and on chromosomes 18 and 21 with the SNP2 map. Peak LOD scores for lnmaxalc were observed on chromosome 1, although none reached genome-wide statistical significance. Quantile-quantile plots revealed that the multipoint distribution of SNP results appeared to fit the asymptotic null distribution better than the twopoint results. CONCLUSION: In conclusion, variance-components linkage analysis using high-density SNP maps provided higher LOD scores compared with the standard microsatellite map, similar to studies using nonparametric linkage methods. Widespread application of SNP maps will depend on further improvements in the computational methods implemented in current software packages