Tsitsiklis, “Linear Stochastic Approximation Driven by Slowly Varying

www.elsevier.com/locate/sysconl

Actor-critic algorithms

Abstract. In this article, we propose and analyze a class of actor-critic algorithms. These are two-time-scale algorithms in which the critic uses temporal difference learning with a linearly parameterized approximation architecture, and the actor is updated in an approximate gradient direction, based on information provided by the critic. We show that the features for the critic should ideally span a subspace prescribed by the choice of parameterization of the actor. We study actor-critic algorithms for Markov decision processes with Polish state and action spaces. We state and prove two results regarding their convergence

Association of GSTT1 and GSTM1 polymorphisms in South Indian Epilepsy Patients

783-787Experimental studies suggest that oxidative stress is one of the contributing factors in the onset of epileptic seizures. Glutathione S-transferases (GSTs) are able to conjugate electrophilic compounds, and thus possess neuroprotective role by removing exogenous and endogenous oxidants, detoxifying therapeutic drugs, environmental toxins through conjugation with glutathione (GSH). Several studies from different ethnic groups showed that polymorphisms of the GST gene have been associated with Epilepsy. In the present study, we investigated the association of GST polymorphism in the South Indian epilepsy patients population. A total 371 samples (110 cases and 261 controls) were genotyped for the GST1 and GSTM1 polymorphism by multiplex PCR method. We observed a significant association of GSTT1 null polymorphism in patients with epilepsy. The frequency of the GSTT1 null genotype was found to be significantly higher in cases (35.45 %) than the controls (18.39 %) (OR: 2.44, 95%CI: 1.4-4.02, P <0.0001). In contrast, the frequency of the GSTM1 null variant was significantly lower in cases (11.81%) than controls (32.95%) (OR: 0.27, 95%CI: 0.14-0.51, P <0.001) indicating a protective role. These results indicated that individuals who have GSTT1 null variant are at higher risk for developing seizure than those of GSTT1 wild genotype. On the other hand, individuals carrying GSTM1 null variant showed protective role against seizure. Further, these two null variants did not show any significant association with antiepileptic drug-induced skin rash

On Differentiability of Average Cost in Parameterized Markov Chains 1 Overview

The purpose of this appendix is to prove Theorem 4.6 in [5] and establish various facts used in verifying some of the assumptions of Theorem A.1 therein. We use the same notation and assumptions as in [5], and to simplify the analysis, we assume that the parameter θ is scalar. The setting used here is similar (but not identical) to the one in [4]. The major difference is that the setting of [4] allows only RSPs of the form Uk = f(Xk,Wk), where the parameter θ enters only through the distribution of Wk which does not depend on x. Incontrast, our setting allows for any RSPs of the form Uk = fθ(Xk,Wk), as long as the distribution µθ(u|x) offθ(x, U) satisfies the required assumptions. The differentiability results presented in this report are based on the following key lemma (see [3, Corollary 2.2.1]). Lemma 1.1. Consider a parameterized family of random variables {Fθ(ω); θ ∈ R} on a Polish space Ω. Suppose the following hold: 1. For each θ0, there exists some d&gt;1 and Kd&gt; 0 such that, E |Fθ − Fθ0 |d � &lt;Kd|θ − θ0 | d, for all θ sufficiently close to θ0. 2. There is a family of random variables {fθ} such tha

Additive Manufacturing Processes: Selective Laser Melting, Electron Beam Melting and Binder Jetting—Selection Guidelines

Additive manufacturing (AM), also known as 3D printing or rapid prototyping, is gaining increasing attention due to its ability to produce parts with added functionality and increased complexities in geometrical design, on top of the fact that it is theoretically possible to produce any shape without limitations. However, most of the research on additive manufacturing techniques are focused on the development of materials/process parameters/products design with different additive manufacturing processes such as selective laser melting, electron beam melting, or binder jetting. However, we do not have any guidelines that discuss the selection of the most suitable additive manufacturing process, depending on the material to be processed, the complexity of the parts to be produced, or the design considerations. Considering the very fact that no reports deal with this process selection, the present manuscript aims to discuss the different selection criteria that are to be considered, in order to select the best AM process (binder jetting/selective laser melting/electron beam melting) for fabricating a specific component with a defined set of material properties

On De Finetti Coherence and Kolmogorov Probability

This article addresses the problem of existence of a countably additive probability measure in the sense of Kolmogorov that is consistent with a probability assignment to a family of sets which is coherent in the sense of De Finetti. Key words: probability assignment, coherence condition, subjective probability, countably additive probability This work done while visiting the Laboratory for Information and Decision Systems. Massachusetts Institute of Technology. This research supported by Grant No. III 5(12)/96-ET from the Department of Science and Technology, Government of India and the U.S. Army Research O#ce under the MURI Grant: Data Fusion in Large Arrays of Microsensors DAAD19-00-1-0466