Bayesian multitask inverse reinforcement learning

We generalise the problem of inverse reinforcement learning to multiple tasks, from multiple demonstrations. Each one may represent one expert trying to solve a different task, or as different experts trying to solve the same task. Our main contribution is to formalise the problem as statistical preference elicitation, via a number of structured priors, whose form captures our biases about the relatedness of different tasks or expert policies. In doing so, we introduce a prior on policy optimality, which is more natural to specify. We show that our framework allows us not only to learn to efficiently from multiple experts but to also effectively differentiate between the goals of each. Possible applications include analysing the intrinsic motivations of subjects in behavioural experiments and learning from multiple teachers.Comment: Corrected version. 13 pages, 8 figure

Power spectra of the natural input to the visual system

AbstractThe efficient coding hypothesis posits that sensory systems are adapted to the regularities of their signal input so as to reduce redundancy in the resulting representations. It is therefore important to characterize the regularities of natural signals to gain insight into the processing of natural stimuli. While measurements of statistical regularity in vision have focused on photographic images of natural environments it has been much less investigated, how the specific imaging process embodied by the organism’s eye induces statistical dependencies on the natural input to the visual system. This has allowed using the convenient assumption that natural image data are homogeneous across the visual field. Here we give up on this assumption and show how the imaging process in a human model eye influences the local statistics of the natural input to the visual system across the entire visual field. Artificial scenes with three-dimensional edge elements were generated and the influence of the imaging projection onto the back of a spherical model eye were quantified. These distributions show a strong radial influence of the imaging process on the resulting edge statistics with increasing eccentricity from the model fovea. This influence is further quantified through computation of the second order intensity statistics as a function of eccentricity from the center of projection using samples from the dead leaves image model. Using data from a naturalistic virtual environment, which allows generation of correctly projected images onto the model eye across the entire field of view, we quantified the second order dependencies as function of the position in the visual field using a new generalized parameterization of the power spectra. Finally, we compared this analysis with a commonly used natural image database, the van Hateren database, and show good agreement within the small field of view available in these photographic images. We conclude by providing a detailed quantitative analysis of the second order statistical dependencies of the natural input to the visual system across the visual field and demonstrating the importance of considering the influence of the sensory system on the statistical regularities of the input to the visual system

Display blindness? Looking again at the visibility of situated displays using eye tracking

Observational studies of situated displays have suggested that they are rarely looked at, and when they are it is typically only for a short period of time. Using a mobile eye tracker during a realistic shopping task in a shopping center, we show that people look at displays more than would be predicted from these observational studies, but still only short glances and often from quite far away. We characterize the patterns of eye-movements that precede looking at a display and discuss some of the design implications for the design of situated display technologies that are deployed in public space

Combinatorial Voter Control in Elections

Voter control problems model situations such as an external agent trying to affect the result of an election by adding voters, for example by convincing some voters to vote who would otherwise not attend the election. Traditionally, voters are added one at a time, with the goal of making a distinguished alternative win by adding a minimum number of voters. In this paper, we initiate the study of combinatorial variants of control by adding voters: In our setting, when we choose to add a voter~$v$, we also have to add a whole bundle $\kappa(v)$ of voters associated with $v$. We study the computational complexity of this problem for two of the most basic voting rules, namely the Plurality rule and the Condorcet rule.Comment: An extended abstract appears in MFCS 201

Heavy quarkonium: progress, puzzles, and opportunities

A golden age for heavy quarkonium physics dawned a decade ago, initiated by the confluence of exciting advances in quantum chromodynamics (QCD) and an explosion of related experimental activity. The early years of this period were chronicled in the Quarkonium Working Group (QWG) CERN Yellow Report (YR) in 2004, which presented a comprehensive review of the status of the field at that time and provided specific recommendations for further progress. However, the broad spectrum of subsequent breakthroughs, surprises, and continuing puzzles could only be partially anticipated. Since the release of the YR, the BESII program concluded only to give birth to BESIII; the $B$-factories and CLEO-c flourished; quarkonium production and polarization measurements at HERA and the Tevatron matured; and heavy-ion collisions at RHIC have opened a window on the deconfinement regime. All these experiments leave legacies of quality, precision, and unsolved mysteries for quarkonium physics, and therefore beg for continuing investigations. The plethora of newly-found quarkonium-like states unleashed a flood of theoretical investigations into new forms of matter such as quark-gluon hybrids, mesonic molecules, and tetraquarks. Measurements of the spectroscopy, decays, production, and in-medium behavior of c\bar{c}, b\bar{b}, and b\bar{c} bound states have been shown to validate some theoretical approaches to QCD and highlight lack of quantitative success for others. The intriguing details of quarkonium suppression in heavy-ion collisions that have emerged from RHIC have elevated the importance of separating hot- and cold-nuclear-matter effects in quark-gluon plasma studies. This review systematically addresses all these matters and concludes by prioritizing directions for ongoing and future efforts.Comment: 182 pages, 112 figures. Editors: N. Brambilla, S. Eidelman, B. K. Heltsley, R. Vogt. Section Coordinators: G. T. Bodwin, E. Eichten, A. D. Frawley, A. B. Meyer, R. E. Mitchell, V. Papadimitriou, P. Petreczky, A. A. Petrov, P. Robbe, A. Vair

Learning from multimedia and hypermedia

Computer-based multimedia and hypermedia resources (e.g., the world wide web) have become one of the primary sources of academic information for a majority of pupils and students. In line with this expansion in the field of education, the scientific study of learning from multimedia and hypermedia has become a very active field of research. In this chapter we provide a short overview with regard to research on learning with multimedia and hypermedia. In two review sections, we describe the educational benefits of multiple representations and of learner control, as these are the two defining characteristics of hypermedia. In a third review section we describe recent scientific trends in the field of multimedia/hypermedia learning. In all three review sections we will point to relevant European work on multimedia/hypermedia carried out within the last 5 years, and often carried out within the Kaleidoscope Network of Excellence. According to the interdisciplinary nature of the field this work might come not only from psychology, but also from technology or pedagogy. Comparing the different research activities on multimedia and hypermedia that have dominated the international scientific discourse in the last decade reveals some important differences. Most important, a gap seems to exist between researchers mainly interested in a “serious” educational use of multimedia/ hypermedia and researchers mainly interested in “serious” experimental research on learning with multimedia/hypermedia. Recent discussions about the pros and cons of “design-based research” or “use-inspired basic research” can be seen as a direct consequence of an increasing awareness of the tensions within these two different cultures of research on education

Preference elicitation and inverse reinforcement learning

We state the problem of inverse reinforcement learning in terms of preference elicitation, resulting in a principled (Bayesian) statistical formulation. This generalises previous work on Bayesian inverse reinforcement learning and allows us to obtain a posterior distribution on the agent\u27s preferences, policy and optionally, the obtained reward sequence, from observations. We examine the relation of the resulting approach to other statistical methods for inverse reinforcement learning via analysis and experimental results. We show that preferences can be determined accurately, even if the observed agent\u27s policy is sub-optimal with respect to its own preferences. In that case, significantly improved policies with respect to the agent\u27s preferences are obtained, compared to both other methods and to the performance of the demonstrated policy. \ua9 2011 Springer-Verlag

Cryptographic Protocols for Secure Second-Price Auctions

In recent years auctions have become more and more important in the field of multiagent systems as useful mechanisms for resource allocation, task assignment and last but not least electronic commerce. In many cases the Vickrey (second-price sealed-bid) auction is used as a protocol that prescribes how the individual agents have to interact in order to come to an agreement. The main reasons for choosing the Vickrey auction are the existence of a dominant strategy equilibrium, the low bandwidth and time consumption due to just one round of bidding and the (theoretical) privacy of bids. This paper specifies properties that are needed to ensure the accurate and secret execution of Vickrey auctions and provides a classification of different forms of collusion. We approach the two major security concerns of the Vickrey auction: the vulnerability to a lying auctioneer and the reluctance of bidders to reveal their private valuations. We then propose a novel technique that allows to securely perform second-price auctions

Biologically Plausible Multi-Dimensional Reinforcement Learning in Neural Networks

Abstract. How does the brain learn to map multi-dimensional sensory inputs to multi-dimensional motor outputs when it can only observe single rewards for the coordinated outputs of the whole network of neurons that make up the brain? We introduce Multi-AGREL, a novel, biologically plausible multi-layer neural network model for multi-dimensional reinforcement learning. We demonstrate that Multi-AGREL can learn non-linear mappings from inputs to multi-dimensional outputs by using only scalar reward feedback. We further show that in Multi-AGREL, the changes in the connection weights follow the gradient that minimizes global prediction error, and that all information required for synaptic plasticity is locally present.