On the Convergence of Stochastic Iterative Dynamic Programming Algorithms

Recent developments in the area of reinforcement learning have yielded a number of new algorithms for the prediction and control of Markovian environments. These algorithms, including the TD(lambda) algorithm of Sutton (1988) and the Q-learning algorithm of Watkins (1989), can be motivated heuristically as approximations to dynamic programming (DP). In this paper we provide a rigorous proof of convergence of these DP-based learning algorithms by relating them to the powerful techniques of stochastic approximation theory via a new convergence theorem. The theorem establishes a general class of convergent algorithms to which both TD(lambda) and Q-learning belong

A case of rapid-onset daptomycin-induced neutropenia

A 75-year-old Caucasian woman was transferred to a tertiary academic medical center for management of a foul smelling, non-draining, unstageable sacral pressure ulcer. Upon admission, the patient's laboratory data was notable for a white blood cell count (WBC) of 14.90 x 10-3/[mu]L and absolute neutrophil count of 12.24 x 10-3 /[mu]L. After a complicated hospital course the patient was initiated on daptomycin 6 mg/kg (300 mg) daily in response to wound cultures detecting Enterococcus faecium susceptible only to daptomycin and linezolid. Starting 96 hours after daptomycin initiation, there was a profound reduction in the patient's WBC, reaching a nadir of 1.63 x 10-3 /[mu]L after 12 days of therapy. Prior to documented resolution of her blood dyscrasia the patient was transferred to hospice care and use of antimicrobial therapy was withdrawn. The Naranjo Adverse Drug Reaction Probability Score demonstrated a probable relationship (score equal to 6) between daptomycin and the development of neutropenia. Unlike previous reports of daptomycin-induced neutropenia and/or thrombocytopenia, where patients received prolonged courses of daptomycin with doses ranging from 6-10 mg/kg, this case describes a much more rapid onset of neutropenia that occurred at a dose of 6 mg/kg/day. This case reports highlights the risk for daptomycin to induce neutropenia in patients receiving traditional dosing, not just at higher doses, and within a shorter time-frame than previously documented. Healthcare providers should be cognizant of this adverse reaction, and diligent in monitoring for adverse events, associated with daptomycin use even when used for short courses or at traditional doses

Learning and Predicting Dynamic Behavior with Graphical Multiagent Models

Factored models of multiagent systems address the complexity of joint behavior by exploiting locality in agent interactions. History-dependent graphical multiagent models (hGMMs) further capture dynamics by conditioning behavior on history. The challenges of modeling real human behavior motivated us to extend the hGMM representation by distinguishing two types of agent interactions. This distinction opens the opportunity for learning dependence networks that are different from given graphical structures representing observed agent interactions. We propose a greedy algorithm for learning hGMMs from time-series data, inducing both graphical structure and parameters. Our empirical study employs human-subject experiment data for a dynamic consensus scenario, where agents on a network attempt to reach a unanimous vote. We show that the learned hGMMs directly expressing joint behavior outperform alternatives in predicting dynamic human voting behavior, and end-game vote results. Analysis of learned graphical structures reveals patterns of action dependence not directly reflected in the original experiment networks

Receptionist rECognition and rEferral of PaTients with Stroke (RECEPTS) study - protocol of a mixed methods study

BACKGROUND: As the first point of contact for patients and witnesses of stroke, General Practice receptionists can be instrumental in deciding the urgency of clinical contact. Despite the considerable complexity of this task, reception staff are not clinically trained. Minimising the time taken to access thrombolysis is crucial in acute stroke as treatment must be initiated within 4.5 hours of the onset, and the earlier the better, to achieve the best outcomes. Research suggests that patients who first contact their General Practice following the onset of stroke symptoms are less likely to receive thrombolysis, in part due to significant delays within Primary Care.This study therefore aims to understand the role of General Practice receptionists, with particular interest in receptionist's ability to recognise people who may be suffering from a stroke and to handle such patients as a medical emergency. METHODS: The Receptionist rECognition and rEferral of PaTients with Stroke (RECEPTS) study will be a Primary Care based mixed methods study. 60 General Practices in the West Midlands will be recruited. Each practice will receive 10 unannounced simulated patient telephone calls, after the 10 calls questionnaires will be administered to each receptionist. These will examine the behaviour of receptionists towards patients presenting in Primary Care with stroke symptoms, and their knowledge of stroke symptoms. An embedded qualitative study will use interviews and focus groups to investigate the views of General Practice staff on the receptionists' role in patient referral and whether training in this area would be helpful. DISCUSSION: The results of the RECEPTS study will have important implications for providers of Primary Care. The study will establish current practice in UK primary care in terms of General Practice receptionists' knowledge of the presentation and appropriate referral of those who may be suffering a stroke. It will highlight training needs and how such training might be best delivered

The association between prehospital care and in-hospital treatment decisions in acute stroke: a cohort study.

BACKGROUND: Hospital prealerting in acute stroke improves the timeliness of subsequent treatment, but little is known about the impact of prehospital assessments on in-hospital care. OBJECTIVE: Examine the association between prehospital assessments and notification by emergency medical service staff on the subsequent acute stroke care pathway. METHODS: This was a cohort study of linked patient medical records. Consenting patients with a diagnosis of stroke were recruited from two urban hospitals. Data from patient medical records were extracted and entered into a Cox regression analysis to investigate the association between time to CT request and recording of onset time, stroke recognition (using the Face Arm Speech Test (FAST)) and sending of a prealert message. RESULTS: 151 patients (aged 71±15 years) travelled to hospital via ambulance and were eligible for this analysis. Time of symptom onset was recorded in 61 (40%) cases, the FAST test was positive in 114 (75%) and a prealert message was sent in 65 (44%). Following adjustment for confounding, patients who had time of onset recorded (HR 0.73, 95% CI 0.52 to 1.03), were FAST-positive (HR 0.54, 95% CI 0.37 to 0.80) or were prealerted (HR 0.26, 95% CI 0.18 to 0.38), were more likely to receive a timely CT request in hospital. CONCLUSIONS: This study highlights the importance of hospital prealerting, accurate stroke recognition, and recording of onset time. Those not recognised with stroke in a prehospital setting appear to be excluded from the possibility of rapid treatment in hospital, even before they have been seen by a specialist

Neurodevelopmental disorders in children aged 2-9 years: Population-based burden estimates across five regions in India.

BACKGROUND: Neurodevelopmental disorders (NDDs) compromise the development and attainment of full social and economic potential at individual, family, community, and country levels. Paucity of data on NDDs slows down policy and programmatic action in most developing countries despite perceived high burden. METHODS AND FINDINGS: We assessed 3,964 children (with almost equal number of boys and girls distributed in 2-<6 and 6-9 year age categories) identified from five geographically diverse populations in India using cluster sampling technique (probability proportionate to population size). These were from the North-Central, i.e., Palwal (N = 998; all rural, 16.4% non-Hindu, 25.3% from scheduled caste/tribe [SC-ST] [these are considered underserved communities who are eligible for affirmative action]); North, i.e., Kangra (N = 997; 91.6% rural, 3.7% non-Hindu, 25.3% SC-ST); East, i.e., Dhenkanal (N = 981; 89.8% rural, 1.2% non-Hindu, 38.0% SC-ST); South, i.e., Hyderabad (N = 495; all urban, 25.7% non-Hindu, 27.3% SC-ST) and West, i.e., North Goa (N = 493; 68.0% rural, 11.4% non-Hindu, 18.5% SC-ST). All children were assessed for vision impairment (VI), epilepsy (Epi), neuromotor impairments including cerebral palsy (NMI-CP), hearing impairment (HI), speech and language disorders, autism spectrum disorders (ASDs), and intellectual disability (ID). Furthermore, 6-9-year-old children were also assessed for attention deficit hyperactivity disorder (ADHD) and learning disorders (LDs). We standardized sample characteristics as per Census of India 2011 to arrive at district level and all-sites-pooled estimates. Site-specific prevalence of any of seven NDDs in 2-<6 year olds ranged from 2.9% (95% CI 1.6-5.5) to 18.7% (95% CI 14.7-23.6), and for any of nine NDDs in the 6-9-year-old children, from 6.5% (95% CI 4.6-9.1) to 18.5% (95% CI 15.3-22.3). Two or more NDDs were present in 0.4% (95% CI 0.1-1.7) to 4.3% (95% CI 2.2-8.2) in the younger age category and 0.7% (95% CI 0.2-2.0) to 5.3% (95% CI 3.3-8.2) in the older age category. All-site-pooled estimates for NDDs were 9.2% (95% CI 7.5-11.2) and 13.6% (95% CI 11.3-16.2) in children of 2-<6 and 6-9 year age categories, respectively, without significant difference according to gender, rural/urban residence, or religion; almost one-fifth of these children had more than one NDD. The pooled estimates for prevalence increased by up to three percentage points when these were adjusted for national rates of stunting or low birth weight (LBW). HI, ID, speech and language disorders, Epi, and LDs were the common NDDs across sites. Upon risk modelling, noninstitutional delivery, history of perinatal asphyxia, neonatal illness, postnatal neurological/brain infections, stunting, LBW/prematurity, and older age category (6-9 year) were significantly associated with NDDs. The study sample was underrepresentative of stunting and LBW and had a 15.6% refusal. These factors could be contributing to underestimation of the true NDD burden in our population. CONCLUSIONS: The study identifies NDDs in children aged 2-9 years as a significant public health burden for India. HI was higher than and ASD prevalence comparable to the published global literature. Most risk factors of NDDs were modifiable and amenable to public health interventions