QDQD-Learning: A Collaborative Distributed Strategy for Multi-Agent Reinforcement Learning Through Consensus + Innovations

The paper considers a class of multi-agent Markov decision processes (MDPs), in which the network agents respond differently (as manifested by the instantaneous one-stage random costs) to a global controlled state and the control actions of a remote controller. The paper investigates a distributed reinforcement learning setup with no prior information on the global state transition and local agent cost statistics. Specifically, with the agents' objective consisting of minimizing a network-averaged infinite horizon discounted cost, the paper proposes a distributed version of $Q$-learning, $\mathcal{QD}$-learning, in which the network agents collaborate by means of local processing and mutual information exchange over a sparse (possibly stochastic) communication network to achieve the network goal. Under the assumption that each agent is only aware of its local online cost data and the inter-agent communication network is \emph{weakly} connected, the proposed distributed scheme is almost surely (a.s.) shown to yield asymptotically the desired value function and the optimal stationary control policy at each network agent. The analytical techniques developed in the paper to address the mixed time-scale stochastic dynamics of the \emph{consensus + innovations} form, which arise as a result of the proposed interactive distributed scheme, are of independent interest.Comment: Submitted to the IEEE Transactions on Signal Processing, 33 page

Distributed Linear Parameter Estimation: Asymptotically Efficient Adaptive Strategies

The paper considers the problem of distributed adaptive linear parameter estimation in multi-agent inference networks. Local sensing model information is only partially available at the agents and inter-agent communication is assumed to be unpredictable. The paper develops a generic mixed time-scale stochastic procedure consisting of simultaneous distributed learning and estimation, in which the agents adaptively assess their relative observation quality over time and fuse the innovations accordingly. Under rather weak assumptions on the statistical model and the inter-agent communication, it is shown that, by properly tuning the consensus potential with respect to the innovation potential, the asymptotic information rate loss incurred in the learning process may be made negligible. As such, it is shown that the agent estimates are asymptotically efficient, in that their asymptotic covariance coincides with that of a centralized estimator (the inverse of the centralized Fisher information rate for Gaussian systems) with perfect global model information and having access to all observations at all times. The proof techniques are mainly based on convergence arguments for non-Markovian mixed time scale stochastic approximation procedures. Several approximation results developed in the process are of independent interest.Comment: Submitted to SIAM Journal on Control and Optimization journal. Initial Submission: Sept. 2011. Revised: Aug. 201

Purely electromagnetic spacetimes

Electrovacuum solutions devoid of usual mass sources are classified in the case of one, two and three commuting Killing vectors. Three branches of solutions exist. Electromagnetically induced mass terms appear in some of them.Comment: 8 page

Frequency-dependent (ac) Conduction in Disordered Composites: a Percolative Study

In a recent paper [Phys. Rev. B{\bf57}, 3375 (1998)], we examined in detail the nonlinear (electrical) dc response of a random resistor cum tunneling bond network ($RRTN$, introduced by us elsewhere to explain nonlinear response of metal-insulator type mixtures). In this work which is a sequel to that paper, we consider the ac response of the $RRTN$-based correlated $RC$ ($CRC$) model. Numerical solutions of the Kirchoff's laws for the $CRC$ model give a power-law exponent (= 0.7 near $p = p_c$) of the modulus of the complex ac conductance at moderately low frequencies, in conformity with experiments on various types of disordered systems. But, at very low frequencies, it gives a simple quadratic or linear dependence on the frequency depending upon whether the system is percolating or not. We do also discuss the effective medium approximation ($EMA$) of our $CRC$ and the traditional random $RC$ network model, and discuss their comparative successes and shortcomings.Comment: Revised and reduced version with 17 LaTeX pages plus 8 JPEG figure

Global oscillation analysis of solar neutrino data with helioseismically constrained fluxes

A seismic model for the Sun calculated using the accurate helioseismic data predicts a lower $^{8}{B}$ neutrino flux as compared to the standard solar model (SSM). However, there persists a discrepancy between the predicted and measured neutrino fluxes and it seems necessary to invoke neutrino oscillations to explain the measurements. In this work, we have performed a global, unified oscillation analysis of the latest solar neutrino data (including the results of SNO charged current rate) using the seismic model fluxes as theoretical predictions. We determine the best-fit values of the neutrino oscillation parameters and the $\chi^2_{\mathrm min}$ for both $\nu_e-\nu_{\mathrm active}$ and $\nu_e -\nu_{\mathrm sterile}$ cases and present the allowed parameter regions in the $\Delta m^2 - \tan^2 \theta$ plane for $\nu_e-\nu_{\mathrm active}$ transition. The results are compared with those obtained using the latest SSM by Bahcall and his collaborators.Comment: Version to appear in Phys. Rev.

Distributed Constrained Recursive Nonlinear Least-Squares Estimation: Algorithms and Asymptotics

This paper focuses on the problem of recursive nonlinear least squares parameter estimation in multi-agent networks, in which the individual agents observe sequentially over time an independent and identically distributed (i.i.d.) time-series consisting of a nonlinear function of the true but unknown parameter corrupted by noise. A distributed recursive estimator of the \emph{consensus} + \emph{innovations} type, namely $\mathcal{CIWNLS}$, is proposed, in which the agents update their parameter estimates at each observation sampling epoch in a collaborative way by simultaneously processing the latest locally sensed information~(\emph{innovations}) and the parameter estimates from other agents~(\emph{consensus}) in the local neighborhood conforming to a pre-specified inter-agent communication topology. Under rather weak conditions on the connectivity of the inter-agent communication and a \emph{global observability} criterion, it is shown that at every network agent, the proposed algorithm leads to consistent parameter estimates. Furthermore, under standard smoothness assumptions on the local observation functions, the distributed estimator is shown to yield order-optimal convergence rates, i.e., as far as the order of pathwise convergence is concerned, the local parameter estimates at each agent are as good as the optimal centralized nonlinear least squares estimator which would require access to all the observations across all the agents at all times. In order to benchmark the performance of the proposed distributed $\mathcal{CIWNLS}$ estimator with that of the centralized nonlinear least squares estimator, the asymptotic normality of the estimate sequence is established and the asymptotic covariance of the distributed estimator is evaluated. Finally, simulation results are presented which illustrate and verify the analytical findings.Comment: 28 pages. Initial Submission: Feb. 2016, Revised: July 2016, Accepted: September 2016, To appear in IEEE Transactions on Signal and Information Processing over Networks: Special Issue on Inference and Learning over Network

Learning-Based Distributed Detection-Estimation in Sensor Networks with Unknown Sensor Defects

We consider the problem of distributed estimation of an unknown deterministic scalar parameter (the target signal) in a wireless sensor network (WSN), where each sensor receives a single snapshot of the field. We assume that the observation at each node randomly falls into one of two modes: a valid or an invalid observation mode. Specifically, mode one corresponds to the desired signal plus noise observation mode (\emph{valid}), and mode two corresponds to the pure noise mode (\emph{invalid}) due to node defect or damage. With no prior information on such local sensing modes, we introduce a learning-based distributed procedure, called the mixed detection-estimation (MDE) algorithm, based on iterative closed-loop interactions between mode learning (detection) and target estimation. The online learning step re-assesses the validity of the local observations at each iteration, thus refining the ongoing estimation update process. The convergence of the MDE algorithm is established analytically. Asymptotic analysis shows that, in the high signal-to-noise ratio (SNR) regime, the MDE estimation error converges to that of an ideal (centralized) estimator with perfect information about the node sensing modes. This is in contrast to the estimation performance of a naive average consensus based distributed estimator (without mode learning), whose estimation error blows up with an increasing SNR.Comment: 15 pages, 2 figures, submitted to TS

Studies on trematode parasites of air breathing fishes of Awangsoi Lake, Manipur

The present investigation deals with a systematic survey of trematode parasites of air breathing fishes from Awangsoi Lake collected during 2008-2009. The air breathing fishes found in Awangsoi Lake are Channa punctatus, Clarias batrachus, Channa striatus, Channa orientalis, Anabas testudineus and Heteropneustes fossilis. During the study period the following 5 species of trematodes were collected : Clinostomum complanatum, Allocreadium handia, Allocreadium fasciatusi, Astiotrema reniferum and Genarcopsis goppo. The percentage of abundance was found to be maximum in Astiotrema reniferum and Anabas testudineus harbours the maximum number of parasites