Reinforcement learning for continuous control

Reinforcement learning is an area in machine learning that concerns how agents should take actions in an environment to maximize its reward. Earlier works on reinforcement learning algorithms work well for discrete action and discrete observations. In the continuous environment the actions are infinite in number and observations are also infinite. So certain reinforcement algorithms were developed in order to learn on continuous spaces. Among those, two algorithms are Stochastic Synapse Reinforcement Learning (SSRL) and Deep Deterministic Policy Gradient (DDPG). Previous works used these algorithms on different environments separately in order to understand the behaviour of the algorithms. In this thesis, a comparison study between between SSRL and DDPG is made by using them on the same continuous environments and observing how each algorithm behaves on each environment and what kind of strengths and weaknesses can be inferred by comparing the algorithms. The algorithms are made to run on two continuous environments, namely mountain car continuous and pendulum. They are run 10 times for each set of time steps like 2000, 3000, 4000 for 1000 episodes each and the cumulative reward at the end of each episode is found. The episode is the length of the simulation at end of which the algorithm ends in a terminal state. The average and standard deviation of cumulative rewards across 10 repetitions for each time step and all the repetitions across different time steps are also collected. The results shows different trends across different experiments. Based on the results it can be inferred that overall SSRL performs consistently even though it does not gains rewards like DDPG whereas DDPG performs inconsistently but certain rewards it earns are higher than those of SSRL. Also in the case of the delayed-reinforcement pendulum environment both algorithms do not learn well, showing their weakness towards environments whose terminal state is not definite

Taz protects hematopoietic stem cells from an aging-dependent decrease in PU.1 activity

Specific functions of the immune system are essential to protect us from infections caused by pathogens such as viruses and bacteria. However, as we age, the immune system shows a functional decline that can be attributed in large part to age-associated defects in hematopoietic stem cells (HSCs)-the cells at the apex of the immune cell hierarchy. Here, we find that the Hippo pathway coactivator TAZ is potently induced in old HSCs and protects these cells from functional decline. We identify Clca3a1 as a TAZ-induced gene that allows us to trace TAZ activity in vivo. Using CLCA3A1 as a marker, we can isolate "young-like" HSCs from old mice. Mechanistically, Taz acts as coactivator of PU.1 and to some extent counteracts the gradual loss of PU.1 expression during HSC aging. Our work thus uncovers an essential role for Taz in a previously undescribed fail-safe mechanism in aging HSCs. Immune system function declines with age, a consequence of defects in hematopoietic stem cells (HSCs). Here the authors show that TAZ buffers age-related loss of PU.1 activity to maintain HSC functionality and identify the surface protein Clca3a1 as a marker of "young-like" HSCs, even in old mice

Effects of antiplatelet therapy on stroke risk by brain imaging features of intracerebral haemorrhage and cerebral small vessel diseases: subgroup analyses of the RESTART randomised, open-label trial

Background Findings from the RESTART trial suggest that starting antiplatelet therapy might reduce the risk of recurrent symptomatic intracerebral haemorrhage compared with avoiding antiplatelet therapy. Brain imaging features of intracerebral haemorrhage and cerebral small vessel diseases (such as cerebral microbleeds) are associated with greater risks of recurrent intracerebral haemorrhage. We did subgroup analyses of the RESTART trial to explore whether these brain imaging features modify the effects of antiplatelet therapy

Effects of antiplatelet therapy after stroke due to intracerebral haemorrhage (RESTART): a randomised, open-label trial

Background: Antiplatelet therapy reduces the risk of major vascular events for people with occlusive vascular disease, although it might increase the risk of intracranial haemorrhage. Patients surviving the commonest subtype of intracranial haemorrhage, intracerebral haemorrhage, are at risk of both haemorrhagic and occlusive vascular events, but whether antiplatelet therapy can be used safely is unclear. We aimed to estimate the relative and absolute effects of antiplatelet therapy on recurrent intracerebral haemorrhage and whether this risk might exceed any reduction of occlusive vascular events. Methods: The REstart or STop Antithrombotics Randomised Trial (RESTART) was a prospective, randomised, open-label, blinded endpoint, parallel-group trial at 122 hospitals in the UK. We recruited adults (≥18 years) who were taking antithrombotic (antiplatelet or anticoagulant) therapy for the prevention of occlusive vascular disease when they developed intracerebral haemorrhage, discontinued antithrombotic therapy, and survived for 24 h. Computerised randomisation incorporating minimisation allocated participants (1:1) to start or avoid antiplatelet therapy. We followed participants for the primary outcome (recurrent symptomatic intracerebral haemorrhage) for up to 5 years. We analysed data from all randomised participants using Cox proportional hazards regression, adjusted for minimisation covariates. This trial is registered with ISRCTN (number ISRCTN71907627). Findings: Between May 22, 2013, and May 31, 2018, 537 participants were recruited a median of 76 days (IQR 29–146) after intracerebral haemorrhage onset: 268 were assigned to start and 269 (one withdrew) to avoid antiplatelet therapy. Participants were followed for a median of 2·0 years (IQR [1·0– 3·0]; completeness 99·3%). 12 (4%) of 268 participants allocated to antiplatelet therapy had recurrence of intracerebral haemorrhage compared with 23 (9%) of 268 participants allocated to avoid antiplatelet therapy (adjusted hazard ratio 0·51 [95% CI 0·25–1·03]; p=0·060). 18 (7%) participants allocated to antiplatelet therapy experienced major haemorrhagic events compared with 25 (9%) participants allocated to avoid antiplatelet therapy (0·71 [0·39–1·30]; p=0·27), and 39 [15%] participants allocated to antiplatelet therapy had major occlusive vascular events compared with 38 [14%] allocated to avoid antiplatelet therapy (1·02 [0·65–1·60]; p=0·92). Interpretation: These results exclude all but a very modest increase in the risk of recurrent intracerebral haemorrhage with antiplatelet therapy for patients on antithrombotic therapy for the prevention of occlusive vascular disease when they developed intracerebral haemorrhage. The risk of recurrent intracerebral haemorrhage is probably too small to exceed the established benefits of antiplatelet therapy for secondary prevention

Effects of antiplatelet therapy after stroke due to intracerebral haemorrhage (RESTART): a randomised, open-label trial

Background: Antiplatelet therapy reduces the risk of major vascular events for people with occlusive vascular disease, although it might increase the risk of intracranial haemorrhage. Patients surviving the commonest subtype of intracranial haemorrhage, intracerebral haemorrhage, are at risk of both haemorrhagic and occlusive vascular events, but whether antiplatelet therapy can be used safely is unclear. We aimed to estimate the relative and absolute effects of antiplatelet therapy on recurrent intracerebral haemorrhage and whether this risk might exceed any reduction of occlusive vascular events. Methods: The REstart or STop Antithrombotics Randomised Trial (RESTART) was a prospective, randomised, open-label, blinded endpoint, parallel-group trial at 122 hospitals in the UK. We recruited adults (≥18 years) who were taking antithrombotic (antiplatelet or anticoagulant) therapy for the prevention of occlusive vascular disease when they developed intracerebral haemorrhage, discontinued antithrombotic therapy, and survived for 24 h. Computerised randomisation incorporating minimisation allocated participants (1:1) to start or avoid antiplatelet therapy. We followed participants for the primary outcome (recurrent symptomatic intracerebral haemorrhage) for up to 5 years. We analysed data from all randomised participants using Cox proportional hazards regression, adjusted for minimisation covariates. This trial is registered with ISRCTN (number ISRCTN71907627). Findings: Between May 22, 2013, and May 31, 2018, 537 participants were recruited a median of 76 days (IQR 29–146) after intracerebral haemorrhage onset: 268 were assigned to start and 269 (one withdrew) to avoid antiplatelet therapy. Participants were followed for a median of 2·0 years (IQR [1·0– 3·0]; completeness 99·3%). 12 (4%) of 268 participants allocated to antiplatelet therapy had recurrence of intracerebral haemorrhage compared with 23 (9%) of 268 participants allocated to avoid antiplatelet therapy (adjusted hazard ratio 0·51 [95% CI 0·25–1·03]; p=0·060). 18 (7%) participants allocated to antiplatelet therapy experienced major haemorrhagic events compared with 25 (9%) participants allocated to avoid antiplatelet therapy (0·71 [0·39–1·30]; p=0·27), and 39 [15%] participants allocated to antiplatelet therapy had major occlusive vascular events compared with 38 [14%] allocated to avoid antiplatelet therapy (1·02 [0·65–1·60]; p=0·92). Interpretation: These results exclude all but a very modest increase in the risk of recurrent intracerebral haemorrhage with antiplatelet therapy for patients on antithrombotic therapy for the prevention of occlusive vascular disease when they developed intracerebral haemorrhage. The risk of recurrent intracerebral haemorrhage is probably too small to exceed the established benefits of antiplatelet therapy for secondary prevention

Effects of antiplatelet therapy after stroke due to intracerebral haemorrhage (RESTART): a randomised, open-label trial

BACKGROUND: Antiplatelet therapy reduces the risk of major vascular events for people with occlusive vascular disease, although it might increase the risk of intracranial haemorrhage. Patients surviving the commonest subtype of intracranial haemorrhage, intracerebral haemorrhage, are at risk of both haemorrhagic and occlusive vascular events, but whether antiplatelet therapy can be used safely is unclear. We aimed to estimate the relative and absolute effects of antiplatelet therapy on recurrent intracerebral haemorrhage and whether this risk might exceed any reduction of occlusive vascular events. METHODS: The REstart or STop Antithrombotics Randomised Trial (RESTART) was a prospective, randomised, open-label, blinded endpoint, parallel-group trial at 122 hospitals in the UK. We recruited adults (≥18 years) who were taking antithrombotic (antiplatelet or anticoagulant) therapy for the prevention of occlusive vascular disease when they developed intracerebral haemorrhage, discontinued antithrombotic therapy, and survived for 24 h. Computerised randomisation incorporating minimisation allocated participants (1:1) to start or avoid antiplatelet therapy. We followed participants for the primary outcome (recurrent symptomatic intracerebral haemorrhage) for up to 5 years. We analysed data from all randomised participants using Cox proportional hazards regression, adjusted for minimisation covariates. This trial is registered with ISRCTN (number ISRCTN71907627). FINDINGS: Between May 22, 2013, and May 31, 2018, 537 participants were recruited a median of 76 days (IQR 29-146) after intracerebral haemorrhage onset: 268 were assigned to start and 269 (one withdrew) to avoid antiplatelet therapy. Participants were followed for a median of 2·0 years (IQR [1·0- 3·0]; completeness 99·3%). 12 (4%) of 268 participants allocated to antiplatelet therapy had recurrence of intracerebral haemorrhage compared with 23 (9%) of 268 participants allocated to avoid antiplatelet therapy (adjusted hazard ratio 0·51 [95% CI 0·25-1·03]; p=0·060). 18 (7%) participants allocated to antiplatelet therapy experienced major haemorrhagic events compared with 25 (9%) participants allocated to avoid antiplatelet therapy (0·71 [0·39-1·30]; p=0·27), and 39 [15%] participants allocated to antiplatelet therapy had major occlusive vascular events compared with 38 [14%] allocated to avoid antiplatelet therapy (1·02 [0·65-1·60]; p=0·92). INTERPRETATION: These results exclude all but a very modest increase in the risk of recurrent intracerebral haemorrhage with antiplatelet therapy for patients on antithrombotic therapy for the prevention of occlusive vascular disease when they developed intracerebral haemorrhage. The risk of recurrent intracerebral haemorrhage is probably too small to exceed the established benefits of antiplatelet therapy for secondary prevention. FUNDING: British Heart Foundation

Global, regional, and national burden of traumatic brain injury and spinal cord injury, 1990-2016 : a systematic analysis for the Global Burden of Disease Study 2016

Background Traumatic brain injury (TBI) and spinal cord injury (SCI) are increasingly recognised as global health priorities in view of the preventability of most injuries and the complex and expensive medical care they necessitate. We aimed to measure the incidence, prevalence, and years of life lived with disability (YLDs) for TBI and SCI from all causes of injury in every country, to describe how these measures have changed between 1990 and 2016, and to estimate the proportion of TBI and SCI cases caused by different types of injury. Methods We used results from the Global Burden of Diseases, Injuries, and Risk Factors (GBD) Study 2016 to measure the global, regional, and national burden of TBI and SCI by age and sex. We measured the incidence and prevalence of all causes of injury requiring medical care in inpatient and outpatient records, literature studies, and survey data. By use of clinical record data, we estimated the proportion of each cause of injury that required medical care that would result in TBI or SCI being considered as the nature of injury. We used literature studies to establish standardised mortality ratios and applied differential equations to convert incidence to prevalence of long-term disability. Finally, we applied GBD disability weights to calculate YLDs. We used a Bayesian meta-regression tool for epidemiological modelling, used cause-specific mortality rates for non-fatal estimation, and adjusted our results for disability experienced with comorbid conditions. We also analysed results on the basis of the Socio-demographic Index, a compound measure of income per capita, education, and fertility. Findings In 2016, there were 27.08 million (95% uncertainty interval [UI] 24.30-30.30 million) new cases of TBI and 0.93 million (0.78-1.16 million) new cases of SCI, with age-standardised incidence rates of 369 (331-412) per 100 000 population for TBI and 13 (11-16) per 100 000 for SCI. In 2016, the number of prevalent cases of TBI was 55.50 million (53.40-57.62 million) and of SCI was 27.04 million (24 .98-30 .15 million). From 1990 to 2016, the age-standardised prevalence of TBI increased by 8.4% (95% UI 7.7 to 9.2), whereas that of SCI did not change significantly (-0.2% [-2.1 to 2.7]). Age-standardised incidence rates increased by 3.6% (1.8 to 5.5) for TBI, but did not change significantly for SCI (-3.6% [-7.4 to 4.0]). TBI caused 8.1 million (95% UI 6. 0-10. 4 million) YLDs and SCI caused 9.5 million (6.7-12.4 million) YLDs in 2016, corresponding to age-standardised rates of 111 (82-141) per 100 000 for TBI and 130 (90-170) per 100 000 for SCI. Falls and road injuries were the leading causes of new cases of TBI and SCI in most regions. Interpretation TBI and SCI constitute a considerable portion of the global injury burden and are caused primarily by falls and road injuries. The increase in incidence of TBI over time might continue in view of increases in population density, population ageing, and increasing use of motor vehicles, motorcycles, and bicycles. The number of individuals living with SCI is expected to increase in view of population growth, which is concerning because of the specialised care that people with SCI can require. Our study was limited by data sparsity in some regions, and it will be important to invest greater resources in collection of data for TBI and SCI to improve the accuracy of future assessments. Copyright (C) 2018 The Author(s). Published by Elsevier Ltd.Peer reviewe

Effects of antiplatelet therapy on stroke risk by brain imaging features of intracerebral haemorrhage and cerebral small vessel diseases:subgroup analyses of the RESTART randomised, open-label trial

Background: Findings from the RESTART trial suggest that starting antiplatelet therapy might reduce the risk of recurrent symptomatic intracerebral haemorrhage compared with avoiding antiplatelet therapy. Brain imaging features of intracerebral haemorrhage and cerebral small vessel diseases (such as cerebral microbleeds) are associated with greater risks of recurrent intracerebral haemorrhage. We did subgroup analyses of the RESTART trial to explore whether these brain imaging features modify the effects of antiplatelet therapy. Methods: RESTART was a prospective, randomised, open-label, blinded-endpoint, parallel-group trial at 122 hospitals in the UK that assessed whether starting antiplatelet therapy might reduce the risk of recurrent symptomatic intracerebral haemorrhage compared with avoiding antiplatelet therapy. For this prespecified subgroup analysis, consultant neuroradiologists masked to treatment allocation reviewed brain CT or MRI scans performed before randomisation to confirm participant eligibility and rate features of the intracerebral haemorrhage and surrounding brain. We followed participants for primary (recurrent symptomatic intracerebral haemorrhage) and secondary (ischaemic stroke) outcomes for up to 5 years (reported elsewhere). For this report, we analysed eligible participants with intracerebral haemorrhage according to their treatment allocation in primary subgroup analyses of cerebral microbleeds on MRI and in exploratory subgroup analyses of other features on CT or MRI. The trial is registered with the ISRCTN registry, number ISRCTN71907627. Findings: Between May 22, 2013, and May 31, 2018, 537 participants were enrolled, of whom 525 (98%) had intracerebral haemorrhage: 507 (97%) were diagnosed on CT (252 assigned to start antiplatelet therapy and 255 assigned to avoid antiplatelet therapy, of whom one withdrew and was not analysed) and 254 (48%) underwent the required brain MRI protocol (122 in the start antiplatelet therapy group and 132 in the avoid antiplatelet therapy group). There were no clinically or statistically significant hazards of antiplatelet therapy on recurrent intracerebral haemorrhage in primary subgroup analyses of cerebral microbleed presence (2 or more) versus absence (0 or 1) (adjusted hazard ratio [HR] 0·30 [95% CI 0·08–1·13] vs 0·77 [0·13–4·61]; pinteraction=0·41), cerebral microbleed number 0–1 versus 2–4 versus 5 or more (HR 0·77 [0·13–4·62] vs 0·32 [0·03–3·66] vs 0·33 [0·07–1·60]; pinteraction=0·75), or cerebral microbleed strictly lobar versus other location (HR 0·52 [0·004–6·79] vs 0·37 [0·09–1·28]; pinteraction=0·85). There was no evidence of heterogeneity in the effects of antiplatelet therapy in any exploratory subgroup analyses (all pinteraction>0·05). Interpretation: Our findings exclude all but a very modest harmful effect of antiplatelet therapy on recurrent intracerebral haemorrhage in the presence of cerebral microbleeds. Further randomised trials are needed to replicate these findings and investigate them with greater precision. Funding: British Heart Foundation

Effects of antiplatelet therapy on stroke risk by brain imaging features of intracerebral haemorrhage and cerebral small vessel diseases : subgroup analyses of the RESTART randomised, open-label trial

Background: Findings from the RESTART trial suggest that starting antiplatelet therapy might reduce the risk of recurrent symptomatic intracerebral haemorrhage compared with avoiding antiplatelet therapy. Brain imaging features of intracerebral haemorrhage and cerebral small vessel diseases (such as cerebral microbleeds) are associated with greater risks of recurrent intracerebral haemorrhage. We did subgroup analyses of the RESTART trial to explore whether these brain imaging features modify the effects of antiplatelet therapy. Methods: RESTART was a prospective, randomised, open-label, blinded-endpoint, parallel-group trial at 122 hospitals in the UK that assessed whether starting antiplatelet therapy might reduce the risk of recurrent symptomatic intracerebral haemorrhage compared with avoiding antiplatelet therapy. For this prespecified subgroup analysis, consultant neuroradiologists masked to treatment allocation reviewed brain CT or MRI scans performed before randomisation to confirm participant eligibility and rate features of the intracerebral haemorrhage and surrounding brain. We followed participants for primary (recurrent symptomatic intracerebral haemorrhage) and secondary (ischaemic stroke) outcomes for up to 5 years (reported elsewhere). For this report, we analysed eligible participants with intracerebral haemorrhage according to their treatment allocation in primary subgroup analyses of cerebral microbleeds on MRI and in exploratory subgroup analyses of other features on CT or MRI. The trial is registered with the ISRCTN registry, number ISRCTN71907627. Findings: Between May 22, 2013, and May 31, 2018, 537 participants were enrolled, of whom 525 (98%) had intracerebral haemorrhage: 507 (97%) were diagnosed on CT (252 assigned to start antiplatelet therapy and 255 assigned to avoid antiplatelet therapy, of whom one withdrew and was not analysed) and 254 (48%) underwent the required brain MRI protocol (122 in the start antiplatelet therapy group and 132 in the avoid antiplatelet therapy group). There were no clinically or statistically significant hazards of antiplatelet therapy on recurrent intracerebral haemorrhage in primary subgroup analyses of cerebral microbleed presence (2 or more) versus absence (0 or 1) (adjusted hazard ratio [HR] 0·30 [95% CI 0·08–1·13] vs 0·77 [0·13–4·61]; pinteraction=0·41), cerebral microbleed number 0–1 versus 2–4 versus 5 or more (HR 0·77 [0·13–4·62] vs 0·32 [0·03–3·66] vs 0·33 [0·07–1·60]; pinteraction=0·75), or cerebral microbleed strictly lobar versus other location (HR 0·52 [0·004–6·79] vs 0·37 [0·09–1·28]; pinteraction=0·85). There was no evidence of heterogeneity in the effects of antiplatelet therapy in any exploratory subgroup analyses (all pinteraction>0·05). Interpretation: Our findings exclude all but a very modest harmful effect of antiplatelet therapy on recurrent intracerebral haemorrhage in the presence of cerebral microbleeds. Further randomised trials are needed to replicate these findings and investigate them with greater precision. Funding: British Heart Foundation