24,676 research outputs found
-Learning: A Collaborative Distributed Strategy for Multi-Agent Reinforcement Learning Through Consensus + Innovations
The paper considers a class of multi-agent Markov decision processes (MDPs),
in which the network agents respond differently (as manifested by the
instantaneous one-stage random costs) to a global controlled state and the
control actions of a remote controller. The paper investigates a distributed
reinforcement learning setup with no prior information on the global state
transition and local agent cost statistics. Specifically, with the agents'
objective consisting of minimizing a network-averaged infinite horizon
discounted cost, the paper proposes a distributed version of -learning,
-learning, in which the network agents collaborate by means of
local processing and mutual information exchange over a sparse (possibly
stochastic) communication network to achieve the network goal. Under the
assumption that each agent is only aware of its local online cost data and the
inter-agent communication network is \emph{weakly} connected, the proposed
distributed scheme is almost surely (a.s.) shown to yield asymptotically the
desired value function and the optimal stationary control policy at each
network agent. The analytical techniques developed in the paper to address the
mixed time-scale stochastic dynamics of the \emph{consensus + innovations}
form, which arise as a result of the proposed interactive distributed scheme,
are of independent interest.Comment: Submitted to the IEEE Transactions on Signal Processing, 33 page
Distributed Linear Parameter Estimation: Asymptotically Efficient Adaptive Strategies
The paper considers the problem of distributed adaptive linear parameter
estimation in multi-agent inference networks. Local sensing model information
is only partially available at the agents and inter-agent communication is
assumed to be unpredictable. The paper develops a generic mixed time-scale
stochastic procedure consisting of simultaneous distributed learning and
estimation, in which the agents adaptively assess their relative observation
quality over time and fuse the innovations accordingly. Under rather weak
assumptions on the statistical model and the inter-agent communication, it is
shown that, by properly tuning the consensus potential with respect to the
innovation potential, the asymptotic information rate loss incurred in the
learning process may be made negligible. As such, it is shown that the agent
estimates are asymptotically efficient, in that their asymptotic covariance
coincides with that of a centralized estimator (the inverse of the centralized
Fisher information rate for Gaussian systems) with perfect global model
information and having access to all observations at all times. The proof
techniques are mainly based on convergence arguments for non-Markovian mixed
time scale stochastic approximation procedures. Several approximation results
developed in the process are of independent interest.Comment: Submitted to SIAM Journal on Control and Optimization journal.
Initial Submission: Sept. 2011. Revised: Aug. 201
Distributed Constrained Recursive Nonlinear Least-Squares Estimation: Algorithms and Asymptotics
This paper focuses on the problem of recursive nonlinear least squares
parameter estimation in multi-agent networks, in which the individual agents
observe sequentially over time an independent and identically distributed
(i.i.d.) time-series consisting of a nonlinear function of the true but unknown
parameter corrupted by noise. A distributed recursive estimator of the
\emph{consensus} + \emph{innovations} type, namely , is
proposed, in which the agents update their parameter estimates at each
observation sampling epoch in a collaborative way by simultaneously processing
the latest locally sensed information~(\emph{innovations}) and the parameter
estimates from other agents~(\emph{consensus}) in the local neighborhood
conforming to a pre-specified inter-agent communication topology. Under rather
weak conditions on the connectivity of the inter-agent communication and a
\emph{global observability} criterion, it is shown that at every network agent,
the proposed algorithm leads to consistent parameter estimates. Furthermore,
under standard smoothness assumptions on the local observation functions, the
distributed estimator is shown to yield order-optimal convergence rates, i.e.,
as far as the order of pathwise convergence is concerned, the local parameter
estimates at each agent are as good as the optimal centralized nonlinear least
squares estimator which would require access to all the observations across all
the agents at all times. In order to benchmark the performance of the proposed
distributed estimator with that of the centralized nonlinear
least squares estimator, the asymptotic normality of the estimate sequence is
established and the asymptotic covariance of the distributed estimator is
evaluated. Finally, simulation results are presented which illustrate and
verify the analytical findings.Comment: 28 pages. Initial Submission: Feb. 2016, Revised: July 2016,
Accepted: September 2016, To appear in IEEE Transactions on Signal and
Information Processing over Networks: Special Issue on Inference and Learning
over Network
Gauging kinematical and internal symmetry groups for extended systems: the Galilean one-time and two-times harmonic oscillators
The possible external couplings of an extended non-relativistic classical
system are characterized by gauging its maximal dynamical symmetry group at the
center-of-mass. The Galilean one-time and two-times harmonic oscillators are
exploited as models. The following remarkable results are then obtained: 1) a
peculiar form of interaction of the system as a whole with the external gauge
fields; 2) a modification of the dynamical part of the symmetry
transformations, which is needed to take into account the alteration of the
dynamics itself, induced by the {\it gauge} fields. In particular, the
Yang-Mills fields associated to the internal rotations have the effect of
modifying the time derivative of the internal variables in a scheme of minimal
coupling (introduction of an internal covariant derivative); 3) given their
dynamical effect, the Yang-Mills fields associated to the internal rotations
apparently define a sort of Galilean spin connection, while the Yang-Mills
fields associated to the quadrupole momentum and to the internal energy have
the effect of introducing a sort of dynamically induced internal metric in the
relative space.Comment: 32 pages, LaTex using the IOP preprint macro package (ioplppt.sty
available at: http://www.iop.org/). The file is available at:
http://www.fis.unipr.it/papers/1995.html The file is a uuencoded tar gzip
file with the IOP preprint style include
A magnetized torus for modeling Sgr A* millimeter images and spectra
Context. The supermassive black hole, Sagittarius (Sgr) A*, in the centre of
our Galaxy has the largest angular size in the sky among all astrophysical
black holes. Its shadow, assuming no rotation, spans ~ 50 microarcsec.
Resolving such dimensions has long been out of reach for astronomical
instruments until a new generation of interferometers being operational during
this decade. Of particular interest is the Event Horizon Telescope (EHT) with
resolution ~ 20 microarcsec in the millimeter-wavelength range 0.87 mm - 1.3
mm. Aims. We investigate the ability of the fully general relativistic
Komissarov (2006) analytical magnetized torus model to account for observable
constraints at Sgr A* in the centimeter and millimeter domains. The impact of
the magnetic field geometry on the observables is also studied. Methods. We
calculate ray-traced centimeter- and millimeter-wavelength synchrotron spectra
and images of a magnetized accretion torus surrounding the central black hole
in Sgr A*. We assume stationarity, axial symmetry, constant specific angular
momentum and polytropic equation of state. A hybrid population of thermal and
non-thermal electrons is considered. Results. We show that the torus model is
capable of reproducing spectral constraints in the millimeter domain, and in
particular in the observable domain of the EHT. However, the torus model is not
yet able to fit the centimeter spectrum. 1.3 mm images at high inclinations are
in agreement with observable constraints. Conclusions. The ability of the torus
model to account for observations of Sgr A* in the millimeter domain is
interesting in the perspective of the future EHT. Such an analytical model
allows very fast computations. It will thus be a suitable test bed for
investigating large domains of physical parameters, as well as non-black-hole
compact object candidates and alternative theories of gravity.Comment: Major changes wrt the June 2014 version. Accepted by A&
Leptons from Dark Matter Annihilation in Milky Way Subhalos
Numerical simulations of dark matter collapse and structure formation show
that in addition to a large halo surrounding the baryonic component of our
galaxy, there also exists a significant number of subhalos that extend hundreds
of kiloparsecs beyond the edge of the observable Milky Way. We find that for
dark matter (DM) annihilation models, galactic subhalos can significantly
modify the spectrum of electrons and positrons as measured at our galactic
position. Using data from the recent Via Lactea II simulation we include the
subhalo contribution of electrons and positrons as boundary source terms for
simulations of high energy cosmic ray propagation with a modified version of
the publicly available GALPROP code. Focusing on the DM DM -> 4e annihilation
channel, we show that including subhalos leads to a better fit to both the
Fermi and PAMELA data. The best fit gives a dark matter particle mass of 1.2
TeV, for boost factors of 90 in the main halo and 1950-3800 in the subhalos
(depending on assumptions about the background), in contrast to the 0.85 TeV
mass that gives the best fit in the main halo-only scenario. These fits suggest
that at least a third of the observed electron cosmic rays from DM annihilation
could come from subhalos, opening up the possibility of a relaxation of recent
stringent constraints from inverse Compton gamma rays originating from the
high-energy leptons.Comment: 8 pages, 13 figures; added referenc
Methyl 2-(4-ferrocenylbenzamido)thiophene-3-carboxylate and ethyl 2-(4-ferrocenylbenzamido)-1,3-thiazole-4-acetate, a unique ferrocen
The conformations and hydrogen bonding in the thiophene and thiazole title compounds, [Fe(Câ
Hâ
)(CââHââNOâS)], (I), and [Fe(Câ
Hâ
)(CââHââNâOâS)], (II), are discussed. The sequence (Câ
Hâ)-(CâHâ)-(CONH)-(CâHâS)-(COâMe) of rings and moieties in (I) is close to being planar; all consecutive interplanar angles are less than 10°. An intramolecular N-H...O=Cester hydrogen bond [graph set S(6), N...O = 2.768 (2) Ă
and N-H...O = 134 (2)°] effects the molecular planarity, and aggregation occurs via hydrogen-bonded chains formed from intermolecular Car-H...O=Cester/amide interactions along [010], with C...O distances ranging from 3.401 (3) to 3.577 (2) Ă
. The thiazole system in (II) crystallizes with two molecules in the asymmetric unit; these differ in the conformation along their long molecular axes; for example, the interplanar angle between the phenylene (CâHâ) and thiazole (CâNS) rings is 8.1 (2)° in one molecule and 27.66 (14)° in the other. Intermolecular N-H...O=Cester hydrogen bonds [N...O = 2.972 (4) and 2.971 (3) Ă
], each augmented by a Cphenylene-H...O=Cester interaction [3.184 (5) and 3.395 (4) Ă
], form motifs with graph set RÂčâ(7) and generate chains along [100]. The amide C=O groups do not participate in hydrogen bonding. Compound (II) is the first reported ferrocenyl-containing thiazole structure
Lattice model study of the thermodynamic interplay of polymer crystallization and liquid-liquid demixing
We report Monte Carlo simulations of a lattice-polymer model that can account
for both polymer crystallization and liquid-liquid demixing in solutions of
semiflexible homopolymers. In our model, neighboring polymer segments can have
isotropic interactions that affect demixing, and anisotropic interactions that
are responsible for freezing. However, our simulations show that the isotropic
interactions also have a noticeable effect on the freezing curve, as do the
anisotropic interactions on demixing. As the relative strength of the isotropic
interactions is reduced, the liquid-liquid demixing transition disappears below
the freezing curve. A simple, extended Flory-Huggins theory accounts quite well
for the phase behavior observed in the simulations.Comment: Revtex, 7 pages, the content accepted by J. Chem. Phy
- âŠ