7,189 research outputs found
Ensemble Inhibition and Excitation in the Human Cortex: an Ising Model Analysis with Uncertainties
The pairwise maximum entropy model, also known as the Ising model, has been
widely used to analyze the collective activity of neurons. However, controversy
persists in the literature about seemingly inconsistent findings, whose
significance is unclear due to lack of reliable error estimates. We therefore
develop a method for accurately estimating parameter uncertainty based on
random walks in parameter space using adaptive Markov Chain Monte Carlo after
the convergence of the main optimization algorithm. We apply our method to the
spiking patterns of excitatory and inhibitory neurons recorded with
multielectrode arrays in the human temporal cortex during the wake-sleep cycle.
Our analysis shows that the Ising model captures neuronal collective behavior
much better than the independent model during wakefulness, light sleep, and
deep sleep when both excitatory (E) and inhibitory (I) neurons are modeled;
ignoring the inhibitory effects of I-neurons dramatically overestimates
synchrony among E-neurons. Furthermore, information-theoretic measures reveal
that the Ising model explains about 80%-95% of the correlations, depending on
sleep state and neuron type. Thermodynamic measures show signatures of
criticality, although we take this with a grain of salt as it may be merely a
reflection of long-range neural correlations.Comment: 17 pages, 8 figure
Natural Wake-Sleep Algorithm
The benefits of using the natural gradient are well known in a wide range of
optimization problems. However, for the training of common neural networks the
resulting increase in computational complexity sets a limitation to its
practical application. Helmholtz Machines are a particular type of generative
model composed of two Sigmoid Belief Networks (SBNs), acting as an encoder and
a decoder, commonly trained using the Wake-Sleep (WS) algorithm and its
reweighted version RWS. For SBNs, it has been shown how the locality of the
connections in the graphical structure induces sparsity in the Fisher
information matrix. The resulting block diagonal structure can be efficiently
exploited to reduce the computational complexity of the Fisher matrix inversion
and thus compute the natural gradient exactly, without the need of
approximations. We present a geometric adaptation of well-known methods from
the literature, introducing the Natural Wake-Sleep (NWS) and the Natural
Reweighted Wake-Sleep (NRWS) algorithms. We present an experimental analysis of
the novel geometrical algorithms based on the convergence speed and the value
of the log-likelihood, both with respect to the number of iterations and the
time complexity and demonstrating improvements on these aspects over their
respective non-geometric baselines.Comment: 19 pages, 9 figure
Two Timescale Convergent Q-learning for Sleep--Scheduling in Wireless Sensor Networks
In this paper, we consider an intrusion detection application for Wireless
Sensor Networks (WSNs). We study the problem of scheduling the sleep times of
the individual sensors to maximize the network lifetime while keeping the
tracking error to a minimum. We formulate this problem as a
partially-observable Markov decision process (POMDP) with continuous
state-action spaces, in a manner similar to (Fuemmeler and Veeravalli [2008]).
However, unlike their formulation, we consider infinite horizon discounted and
average cost objectives as performance criteria. For each criterion, we propose
a convergent on-policy Q-learning algorithm that operates on two timescales,
while employing function approximation to handle the curse of dimensionality
associated with the underlying POMDP. Our proposed algorithm incorporates a
policy gradient update using a one-simulation simultaneous perturbation
stochastic approximation (SPSA) estimate on the faster timescale, while the
Q-value parameter (arising from a linear function approximation for the
Q-values) is updated in an on-policy temporal difference (TD) algorithm-like
fashion on the slower timescale. The feature selection scheme employed in each
of our algorithms manages the energy and tracking components in a manner that
assists the search for the optimal sleep-scheduling policy. For the sake of
comparison, in both discounted and average settings, we also develop a function
approximation analogue of the Q-learning algorithm. This algorithm, unlike the
two-timescale variant, does not possess theoretical convergence guarantees.
Finally, we also adapt our algorithms to include a stochastic iterative
estimation scheme for the intruder's mobility model. Our simulation results on
a 2-dimensional network setting suggest that our algorithms result in better
tracking accuracy at the cost of only a few additional sensors, in comparison
to a recent prior work
- …