371 research outputs found
Learning with Opponent-Learning Awareness
Multi-agent settings are quickly gathering importance in machine learning.
This includes a plethora of recent work on deep multi-agent reinforcement
learning, but also can be extended to hierarchical RL, generative adversarial
networks and decentralised optimisation. In all these settings the presence of
multiple learning agents renders the training problem non-stationary and often
leads to unstable training or undesired final results. We present Learning with
Opponent-Learning Awareness (LOLA), a method in which each agent shapes the
anticipated learning of the other agents in the environment. The LOLA learning
rule includes a term that accounts for the impact of one agent's policy on the
anticipated parameter update of the other agents. Results show that the
encounter of two LOLA agents leads to the emergence of tit-for-tat and
therefore cooperation in the iterated prisoners' dilemma, while independent
learning does not. In this domain, LOLA also receives higher payouts compared
to a naive learner, and is robust against exploitation by higher order
gradient-based methods. Applied to repeated matching pennies, LOLA agents
converge to the Nash equilibrium. In a round robin tournament we show that LOLA
agents successfully shape the learning of a range of multi-agent learning
algorithms from literature, resulting in the highest average returns on the
IPD. We also show that the LOLA update rule can be efficiently calculated using
an extension of the policy gradient estimator, making the method suitable for
model-free RL. The method thus scales to large parameter and input spaces and
nonlinear function approximators. We apply LOLA to a grid world task with an
embedded social dilemma using recurrent policies and opponent modelling. By
explicitly considering the learning of the other agent, LOLA agents learn to
cooperate out of self-interest. The code is at github.com/alshedivat/lola
Absence of remote earthquake triggering within the Coso and Salton Sea geothermal production fields
Geothermal areas are long recognized to be susceptible to remote earthquake triggering, probably due to the high seismicity rates and presence of geothermal fluids. However, anthropogenic injection and extraction activity may alter the stress state and fluid flow within the geothermal fields. Here we examine the remote triggering phenomena in the Coso geothermal field and its surrounding areas to assess possible anthropogenic effects. We find that triggered earthquakes are absent within the geothermal field but occur in the surrounding areas. Similar observation is also found in the Salton Sea geothermal field. We hypothesize that continuous geothermal operation has eliminated any significant differential pore pressure between fractures inside the geothermal field through flushing geothermal precipitations and sediments out of clogged fractures. To test this hypothesis, we analyze the pore-pressure-driven earthquake swarms, and they are found to occur outside or on the periphery of the geothermal production field. Therefore, our results suggest that the geothermal operation has changed the subsurface fracture network, and differential pore pressure is the primary controlling factor of remote triggering in geothermal fields
Optimization of second-harmonic generation from touching plasmonic wires
We employ transformation optics to optimize the generic nonlinear wave
interaction of second-harmonic generation from a pair of touching metallic
wires. We demonstrate a 10 orders of magnitude increase in the second-harmonic
scattering cross-section by increasing the background permittivity and a 5
orders of magnitude increase in efficiency with respect to a single wire. These
results have clear implications for the design of nanostructured metallic
frequency-conversion devices. Finally, we exploit our analytic solution of a
non-trivial nanophotonic geometry as a platform for performing a critical
comparison of the strengths, weaknesses and validity of other prevailing
theoretical approaches previously employed for nonlinear wave interactions at
the nanoscale
On The Problem of Vacuum energy in Brane Theories
We point out that modern brane theories suffer from a severe vacuum energy
problem. To be specific, the Casimir energy associated with the matter fields
confined to the brane, is stemming from the one and the same localization
mechanism which forms the brane itself, and is thus generically unavoidable.
Possible practical solutions are discussed, including in particular
spontaneously broken supersymmetry, and quantum mechanically induced brane
tension.Comment: 9 pages, 1 figure, to be published in Phys. Lett.
Reference Architecture for Collaborative Design
Issues and themes of Collaborative Design (CD) addressed by research done so far are so extensive that when running a project of collaborative design, people may lack directions or guidelines to support the whole picture. Hence, developing reference architecture for CD is important and necessary in the academic and the empirical fields. Reference architecture provides the systematic, elementary skeleton and can be extended and adapted to diverse, changing environments. It also provides a comprehensive framework and enables practices implemented more thoroughly and easily. The reference architecture developed in this re-search is formed along three dimensions: decision aspect, design stage, and collaboration scope. There are five elements in the dimension of decision aspect: (1) participant, (2) product, (3) process, (4) organization, and (5) information. The dimension of design stage includes three stages: (1) planning and concepting, (2) system-level design and detail design, and (3) testing and prototyping. The dimension of collaboration scope includes three types of collaboration: (1) cross-functional, (2) cross-company, and (3) cross-industry. Because of the three reference dimensions, a cubic architecture is developed. The cubic reference architecture helps decision-makers in dealing with implementing a CD project or activity. It also serves as a guideline for CD system developers or people involved in the design collaboration to figure out their own responsibility functions and their relations with other members. Demonstration of how to use the reference architecture in developing design collaboration activities and specifying the details for cross-company CD is also provided in this research
On prisms, M\"obius ladders and the cycle space of dense graphs
For a graph X, let f_0(X) denote its number of vertices, d(X) its minimum
degree and Z_1(X;Z/2) its cycle space in the standard graph-theoretical sense
(i.e. 1-dimensional cycle group in the sense of simplicial homology theory with
Z/2-coefficients). Call a graph Hamilton-generated if and only if the set of
all Hamilton circuits is a Z/2-generating system for Z_1(X;Z/2). The main
purpose of this paper is to prove the following: for every s > 0 there exists
n_0 such that for every graph X with f_0(X) >= n_0 vertices, (1) if d(X) >=
(1/2 + s) f_0(X) and f_0(X) is odd, then X is Hamilton-generated, (2) if d(X)
>= (1/2 + s) f_0(X) and f_0(X) is even, then the set of all Hamilton circuits
of X generates a codimension-one subspace of Z_1(X;Z/2), and the set of all
circuits of X having length either f_0(X)-1 or f_0(X) generates all of
Z_1(X;Z/2), (3) if d(X) >= (1/4 + s) f_0(X) and X is square bipartite, then X
is Hamilton-generated. All these degree-conditions are essentially
best-possible. The implications in (1) and (2) give an asymptotic affirmative
answer to a special case of an open conjecture which according to [European J.
Combin. 4 (1983), no. 3, p. 246] originates with A. Bondy.Comment: 33 pages; 5 figure
Learning from Demonstration in the Wild
Learning from demonstration (LfD) is useful in settings where hand-coding
behaviour or a reward function is impractical. It has succeeded in a wide range
of problems but typically relies on manually generated demonstrations or
specially deployed sensors and has not generally been able to leverage the
copious demonstrations available in the wild: those that capture behaviours
that were occurring anyway using sensors that were already deployed for another
purpose, e.g., traffic camera footage capturing demonstrations of natural
behaviour of vehicles, cyclists, and pedestrians. We propose Video to Behaviour
(ViBe), a new approach to learn models of behaviour from unlabelled raw video
data of a traffic scene collected from a single, monocular, initially
uncalibrated camera with ordinary resolution. Our approach calibrates the
camera, detects relevant objects, tracks them through time, and uses the
resulting trajectories to perform LfD, yielding models of naturalistic
behaviour. We apply ViBe to raw videos of a traffic intersection and show that
it can learn purely from videos, without additional expert knowledge.Comment: Accepted to the IEEE International Conference on Robotics and
Automation (ICRA) 2019; extended version with appendi
The missing link in gravitational-wave astronomy: A summary of discoveries waiting in the decihertz range
Since 2015 the gravitational-wave observations of LIGO and Virgo have transformed our understanding of compact-object binaries. In the years to come, ground-based gravitational-wave observatories such as LIGO, Virgo, and their successors will increase in sensitivity, discovering thousands of stellar-mass binaries. In the 2030s, the space-based LISA will provide gravitational-wave observations of massive black holes binaries. Between the ∼10–103 Hz band of ground-based observatories and the ∼10−4–10− 1 Hz band of LISA lies the uncharted decihertz gravitational-wave band. We propose a Decihertz Observatory to study this frequency range, and to complement observations made by other detectors. Decihertz observatories are well suited to observation of intermediate-mass (∼102–104M⊙) black holes; they will be able to detect stellar-mass binaries days to years before they merge, providing early warning of nearby binary neutron star mergers and measurements of the eccentricity of binary black holes, and they will enable new tests of general relativity and the Standard Model of particle physics. Here we summarise how a Decihertz Observatory could provide unique insights into how black holes form and evolve across cosmic time, improve prospects for both multimessenger astronomy and multiband gravitational-wave astronomy, and enable new probes of gravity, particle physics and cosmology.publishedVersio
Synaptic dynamics contribute to long-term single neuron response fluctuations
Firing rate variability at the single neuron level is characterized by long-memory processes and complex statistics over a wide range of time scales (from milliseconds up to several hours). Here, we focus on the contribution of non-stationary efficacy of the ensemble of synapses-activated in response to a given stimulus-on single neuron response variability. We present and validate a method tailored for controlled and specific long-term activation of a single cortical neuron in vitro via synaptic or antidromic stimulation, enabling a clear separation between two determinants of neuronal response variability: membrane excitability dynamics vs. synaptic dynamics. Applying this method we show that, within the range of physiological activation frequencies, the synaptic ensemble of a given neuron is a key contributor to the neuronal response variability, long-memory processes and complex statistics observed over extended time scales. Synaptic transmission dynamics impact on response variability in stimulation rates that are substantially lower compared to stimulation rates that drive excitability resources to fluctuate. Implications to network embedded neurons are discussed. \ua9 2014 Reinartz, Biro, Gal, Giugliano and Marom
- …