Two Timescale Convergent Q-learning for Sleep--Scheduling in Wireless Sensor Networks

In this paper, we consider an intrusion detection application for Wireless Sensor Networks (WSNs). We study the problem of scheduling the sleep times of the individual sensors to maximize the network lifetime while keeping the tracking error to a minimum. We formulate this problem as a partially-observable Markov decision process (POMDP) with continuous state-action spaces, in a manner similar to (Fuemmeler and Veeravalli [2008]). However, unlike their formulation, we consider infinite horizon discounted and average cost objectives as performance criteria. For each criterion, we propose a convergent on-policy Q-learning algorithm that operates on two timescales, while employing function approximation to handle the curse of dimensionality associated with the underlying POMDP. Our proposed algorithm incorporates a policy gradient update using a one-simulation simultaneous perturbation stochastic approximation (SPSA) estimate on the faster timescale, while the Q-value parameter (arising from a linear function approximation for the Q-values) is updated in an on-policy temporal difference (TD) algorithm-like fashion on the slower timescale. The feature selection scheme employed in each of our algorithms manages the energy and tracking components in a manner that assists the search for the optimal sleep-scheduling policy. For the sake of comparison, in both discounted and average settings, we also develop a function approximation analogue of the Q-learning algorithm. This algorithm, unlike the two-timescale variant, does not possess theoretical convergence guarantees. Finally, we also adapt our algorithms to include a stochastic iterative estimation scheme for the intruder's mobility model. Our simulation results on a 2-dimensional network setting suggest that our algorithms result in better tracking accuracy at the cost of only a few additional sensors, in comparison to a recent prior work

Quantitative multi-objective verification for probabilistic systems

We present a verification framework for analysing multiple quantitative objectives of systems that exhibit both nondeterministic and stochastic behaviour. These systems are modelled as probabilistic automata, enriched with cost or reward structures that capture, for example, energy usage or performance metrics. Quantitative properties of these models are expressed in a specification language that incorporates probabilistic safety and liveness properties, expected total cost or reward, and supports multiple objectives of these types. We propose and implement an efficient verification framework for such properties and then present two distinct applications of it: firstly, controller synthesis subject to multiple quantitative objectives; and, secondly, quantitative compositional verification. The practical applicability of both approaches is illustrated with experimental results from several large case studies

Pairing in the continuum: the quadrupole response of the Borromean nucleus 6He

The ground state and low-lying continuum states of 6He are found within a shell model scheme, in a basis of two-particle states built out of continuum p-states of the unbound 5He nucleus, using a simple pairing contact-delta interaction. This accounts for the Borromean character of the bound ground state, revealing its composition. We investigate the quadrupole response of the system and we put our calculations into perspective with the latest experimental results. The calculated quadrupole strength distribution reproduces the narrow 2+ resonance, while a second wider peak is found at about 3.9 MeV above the g.s. energy.Comment: 5 pages, 5 figure

Electric multipole response of the halo nucleus 6^6He

The role of different continuum components in the weakly-bound nucleus $^6$He is studied by coupling unbound spd-waves of $^5$He by means of simple pairing contact-delta interaction. The results of our previous investigations in a model space containing only p-waves, showed the collective nature of the ground state and allowed the calculation of the electric quadrupole transitions. We extend this simple model by including also sd-continuum neutron states and we investigate the electric monopole, dipole and octupole response of the system for transitions to the continuum, discussing the contribution of different configurations.Comment: 22 pages, 10 figure

Proteinopathy, oxidative stress and mitochondrial dysfunction: cross talk in alzheimer’s disease and parkinson’s disease

Alzheimer's disease and Parkinson's disease are two common neurodegenerative diseases of the elderly people that have devastating effects in terms of morbidity and mortality. The predominant form of the disease in either case is sporadic with uncertain etiology. The clinical features of Parkinson's disease are primarily motor deficits, while the patients of Alzheimer's disease present with dementia and cognitive impairment. Though neuronal death is a common element in both the disorders, the postmortem histopathology of the brain is very characteristic in each case and different from each other. In terms of molecular pathogenesis, however, both the diseases have a significant commonality, and proteinopathy (abnormal accumulation of misfolded proteins), mitochondrial dysfunction and oxidative stress are the cardinal features in either case. These three damage mechanisms work in concert, reinforcing each other to drive the pathology in the aging brain for both the diseases; very interestingly, the nature of interactions among these three damage mechanisms is very similar in both the diseases, and this review attempts to highlight these aspects. In the case of Alzheimer's disease, the peptide amyloid beta (A beta) is responsible for the proteinopathy, while alpha-synuclein plays a similar role in Parkinson's disease. The expression levels of these two proteins and their aggregation processes are modulated by reactive oxygen radicals and transition metal ions in a similar manner. In turn, these proteins - as oligomers or in aggregated forms - cause mitochondrial impairment by apparently following similar mechanisms. Understanding the common nature of these interactions may, therefore, help us to identify putative neuroprotective strategies that would be beneficial in both the clinical conditions

Electric and magnetic response to the continuum for A=7 isobars in a dicluster model

Mirror isobars $^7$Li and $^7$Be are investigated in a dicluster model. The magnetic dipole moments and the magnetic dipole response to the continuum are calculated in this framework. The magnetic contribution is found to be small with respect to electric dipole and quadrupole excitations even at astrophysical energies, at a variance with the case of deuteron. Energy weighted molecular sum rules are evaluated and a formula for the molecular magnetic dipole sum rule is found which matches the numerical calculations. Cross-sections for photo-dissociation and radiative capture as well as the S-factor for reactions of astrophysical significance are calculated with good agreement with known experimental data.Comment: Accepted in EPJ

Application of edge-based finite elements and vector ABCs in 3D scattering

A finite element absorbing boundary condition (FE-ABC) solution of the scattering by arbitrary 3-D structures is considered. The computational domain is discretized using edge-based tetrahedral elements. In contrast to the node-based elements, edge elements can treat geometries with sharp edges, are divergence-less, and easily satisfy the field continuity condition across dielectric interfaces. They do, however, lead to a higher unknown count but this is balanced by the greater sparsity of the resulting finite element matrix. Thus, the computation time required to solve such a system iteratively with a given degree of accuracy is less than the traditional node-based approach. The purpose is to examine the derivation and performance of the ABC's when applied to 2-D and 3-D problems and to discuss the specifics of our FE-ABC implementation

Binary Black Holes in Dense Star Clusters: Exploring the Theoretical Uncertainties

Recent N-body simulations predict that large numbers of stellar black holes (BHs) could remain bound to globular clusters (GCs) at present, and merging BH--BH binaries are produced dynamically in significant numbers. We systematically vary "standard" assumptions made by numerical simulations related to, e.g., BH formation, stellar winds, binary properties of high-mass stars, and IMF within existing uncertainties, and study the effects on the evolution of the structural properties of GCs, and the BHs in GCs. We find that variations in initial assumptions can set otherwise identical initial clusters on completely different evolutionary paths, significantly affecting their present observable properties, or even affecting the cluster's very survival to the present. However, these changes usually do not affect the numbers or properties of local BH--BH mergers. The only exception is that variations in the assumed winds and IMF can change the masses and numbers of local BH--BH mergers, respectively. All other variations (e.g., in initial binary properties and binary fraction) leave the masses and numbers of locally merging BH--BH binaries largely unchanged. This is in contrast to binary population synthesis models for the field, where results are very sensitive to many uncertain parameters in the initial binary properties and binary stellar-evolution physics. We find that weak winds are required for producing GW150914-like mergers from GCs at low redshifts. LVT151012 can be produced in GCs modeled both with strong and weak winds. GW151226 is lower-mass than typical mergers from GCs modeled with weak winds, but is similar to mergers from GCs modeled with strong winds.Comment: 25 pages, 20 figures, 4 tables, ApJ in pres