15 research outputs found
Bayesian multitask inverse reinforcement learning
We generalise the problem of inverse reinforcement learning to multiple
tasks, from multiple demonstrations. Each one may represent one expert trying
to solve a different task, or as different experts trying to solve the same
task. Our main contribution is to formalise the problem as statistical
preference elicitation, via a number of structured priors, whose form captures
our biases about the relatedness of different tasks or expert policies. In
doing so, we introduce a prior on policy optimality, which is more natural to
specify. We show that our framework allows us not only to learn to efficiently
from multiple experts but to also effectively differentiate between the goals
of each. Possible applications include analysing the intrinsic motivations of
subjects in behavioural experiments and learning from multiple teachers.Comment: Corrected version. 13 pages, 8 figure
Power spectra of the natural input to the visual system
AbstractThe efficient coding hypothesis posits that sensory systems are adapted to the regularities of their signal input so as to reduce redundancy in the resulting representations. It is therefore important to characterize the regularities of natural signals to gain insight into the processing of natural stimuli. While measurements of statistical regularity in vision have focused on photographic images of natural environments it has been much less investigated, how the specific imaging process embodied by the organismâs eye induces statistical dependencies on the natural input to the visual system. This has allowed using the convenient assumption that natural image data are homogeneous across the visual field. Here we give up on this assumption and show how the imaging process in a human model eye influences the local statistics of the natural input to the visual system across the entire visual field. Artificial scenes with three-dimensional edge elements were generated and the influence of the imaging projection onto the back of a spherical model eye were quantified. These distributions show a strong radial influence of the imaging process on the resulting edge statistics with increasing eccentricity from the model fovea. This influence is further quantified through computation of the second order intensity statistics as a function of eccentricity from the center of projection using samples from the dead leaves image model. Using data from a naturalistic virtual environment, which allows generation of correctly projected images onto the model eye across the entire field of view, we quantified the second order dependencies as function of the position in the visual field using a new generalized parameterization of the power spectra. Finally, we compared this analysis with a commonly used natural image database, the van Hateren database, and show good agreement within the small field of view available in these photographic images. We conclude by providing a detailed quantitative analysis of the second order statistical dependencies of the natural input to the visual system across the visual field and demonstrating the importance of considering the influence of the sensory system on the statistical regularities of the input to the visual system
Display blindness? Looking again at the visibility of situated displays using eye tracking
Observational studies of situated displays have suggested that they are rarely looked at, and when they are it is typically only for a short period of time. Using a mobile eye tracker during a realistic shopping task in a shopping center, we show that people look at displays more than would be predicted from these observational studies, but still only short glances and often from quite far away. We characterize the patterns of eye-movements that precede looking at a display and discuss some of the design implications for the design of situated display technologies that are deployed in public space
Combinatorial Voter Control in Elections
Voter control problems model situations such as an external agent trying to
affect the result of an election by adding voters, for example by convincing
some voters to vote who would otherwise not attend the election. Traditionally,
voters are added one at a time, with the goal of making a distinguished
alternative win by adding a minimum number of voters. In this paper, we
initiate the study of combinatorial variants of control by adding voters: In
our setting, when we choose to add a voter~, we also have to add a whole
bundle of voters associated with . We study the computational
complexity of this problem for two of the most basic voting rules, namely the
Plurality rule and the Condorcet rule.Comment: An extended abstract appears in MFCS 201
Heavy quarkonium: progress, puzzles, and opportunities
A golden age for heavy quarkonium physics dawned a decade ago, initiated by
the confluence of exciting advances in quantum chromodynamics (QCD) and an
explosion of related experimental activity. The early years of this period were
chronicled in the Quarkonium Working Group (QWG) CERN Yellow Report (YR) in
2004, which presented a comprehensive review of the status of the field at that
time and provided specific recommendations for further progress. However, the
broad spectrum of subsequent breakthroughs, surprises, and continuing puzzles
could only be partially anticipated. Since the release of the YR, the BESII
program concluded only to give birth to BESIII; the -factories and CLEO-c
flourished; quarkonium production and polarization measurements at HERA and the
Tevatron matured; and heavy-ion collisions at RHIC have opened a window on the
deconfinement regime. All these experiments leave legacies of quality,
precision, and unsolved mysteries for quarkonium physics, and therefore beg for
continuing investigations. The plethora of newly-found quarkonium-like states
unleashed a flood of theoretical investigations into new forms of matter such
as quark-gluon hybrids, mesonic molecules, and tetraquarks. Measurements of the
spectroscopy, decays, production, and in-medium behavior of c\bar{c}, b\bar{b},
and b\bar{c} bound states have been shown to validate some theoretical
approaches to QCD and highlight lack of quantitative success for others. The
intriguing details of quarkonium suppression in heavy-ion collisions that have
emerged from RHIC have elevated the importance of separating hot- and
cold-nuclear-matter effects in quark-gluon plasma studies. This review
systematically addresses all these matters and concludes by prioritizing
directions for ongoing and future efforts.Comment: 182 pages, 112 figures. Editors: N. Brambilla, S. Eidelman, B. K.
Heltsley, R. Vogt. Section Coordinators: G. T. Bodwin, E. Eichten, A. D.
Frawley, A. B. Meyer, R. E. Mitchell, V. Papadimitriou, P. Petreczky, A. A.
Petrov, P. Robbe, A. Vair
Learning from multimedia and hypermedia
Computer-based multimedia and hypermedia resources (e.g., the world wide web) have become one of the primary sources of academic information for a majority of pupils and students. In line with this expansion in the field of education, the scientific study of learning from multimedia and hypermedia has become a very active field of research. In this chapter we provide a short overview with regard to research on learning with multimedia and hypermedia. In two review sections, we describe the educational benefits of multiple representations and of learner control, as these are the two defining characteristics of hypermedia. In a third review section we describe recent scientific trends in the field of multimedia/hypermedia learning. In all three review sections we will point to relevant European work on multimedia/hypermedia carried out within the last 5 years, and often carried out within the Kaleidoscope Network of Excellence. According to the interdisciplinary nature of the field this work might come not only from psychology, but also from technology or pedagogy. Comparing the different research activities on multimedia and hypermedia that have dominated the international scientific discourse in the last decade reveals some important differences. Most important, a gap seems to exist between researchers mainly interested in a âseriousâ educational use of multimedia/ hypermedia and researchers mainly interested in âseriousâ experimental research on learning with multimedia/hypermedia. Recent discussions about the pros and cons of âdesign-based researchâ or âuse-inspired basic researchâ can be seen as a direct consequence of an increasing awareness of the tensions within these two different cultures of research on education
Preference elicitation and inverse reinforcement learning
We state the problem of inverse reinforcement learning in terms of preference elicitation, resulting in a principled (Bayesian) statistical formulation. This generalises previous work on Bayesian inverse reinforcement learning and allows us to obtain a posterior distribution on the agent\u27s preferences, policy and optionally, the obtained reward sequence, from observations. We examine the relation of the resulting approach to other statistical methods for inverse reinforcement learning via analysis and experimental results. We show that preferences can be determined accurately, even if the observed agent\u27s policy is sub-optimal with respect to its own preferences. In that case, significantly improved policies with respect to the agent\u27s preferences are obtained, compared to both other methods and to the performance of the demonstrated policy. \ua9 2011 Springer-Verlag
Cryptographic Protocols for Secure Second-Price Auctions
In recent years auctions have become more and more important in the field of multiagent systems as useful mechanisms for resource allocation, task assignment and last but not least electronic commerce. In many cases the Vickrey (second-price sealed-bid) auction is used as a protocol that prescribes how the individual agents have to interact in order to come to an agreement. The main reasons for choosing the Vickrey auction are the existence of a dominant strategy equilibrium, the low bandwidth and time consumption due to just one round of bidding and the (theoretical) privacy of bids. This paper specifies properties that are needed to ensure the accurate and secret execution of Vickrey auctions and provides a classification of different forms of collusion. We approach the two major security concerns of the Vickrey auction: the vulnerability to a lying auctioneer and the reluctance of bidders to reveal their private valuations. We then propose a novel technique that allows to securely perform second-price auctions
Biologically Plausible Multi-Dimensional Reinforcement Learning in Neural Networks
Abstract. How does the brain learn to map multi-dimensional sensory inputs to multi-dimensional motor outputs when it can only observe single rewards for the coordinated outputs of the whole network of neurons that make up the brain? We introduce Multi-AGREL, a novel, biologically plausible multi-layer neural network model for multi-dimensional reinforcement learning. We demonstrate that Multi-AGREL can learn non-linear mappings from inputs to multi-dimensional outputs by using only scalar reward feedback. We further show that in Multi-AGREL, the changes in the connection weights follow the gradient that minimizes global prediction error, and that all information required for synaptic plasticity is locally present.