Hierarchical Policy Search via Return-Weighted Density Estimation

Learning an optimal policy from a multi-modal reward function is a challenging problem in reinforcement learning (RL). Hierarchical RL (HRL) tackles this problem by learning a hierarchical policy, where multiple option policies are in charge of different strategies corresponding to modes of a reward function and a gating policy selects the best option for a given context. Although HRL has been demonstrated to be promising, current state-of-the-art methods cannot still perform well in complex real-world problems due to the difficulty of identifying modes of the reward function. In this paper, we propose a novel method called hierarchical policy search via return-weighted density estimation (HPSDE), which can efficiently identify the modes through density estimation with return-weighted importance sampling. Our proposed method finds option policies corresponding to the modes of the return function and automatically determines the number and the location of option policies, which significantly reduces the burden of hyper-parameters tuning. Through experiments, we demonstrate that the proposed HPSDE successfully learns option policies corresponding to modes of the return function and that it can be successfully applied to a challenging motion planning problem of a redundant robotic manipulator.Comment: The 32nd AAAI Conference on Artificial Intelligence (AAAI 2018), 9 page

Machine learning approach for computing optical properties of a photonic crystal fiber

Photonic crystal fibers (PCFs) are the specialized optical waveguides that led to many interesting applications ranging from nonlinear optical signal processing to high-power fiber amplifiers. In this paper, machine learning techniques are used to compute various optical properties including effective index, effective mode area, dispersion and confinement loss for a solid-core PCF. These machine learning algorithms based on artificial neural networks are able to make accurate predictions of above-mentioned optical properties for usual parameter space of wavelength ranging from 0.5-1.8 µm, pitch from 0.8-2.0 µm, diameter by pitch from 0.6-0.9 and number of rings as 4 or 5 in a silica solid-core PCF. We demonstrate the use of simple and fast-training feed-forward artificial neural networks that predicts the output for unknown device parameters faster than conventional numerical simulation techniques. Computation runtimes required with neural networks (for training and testing) and Lumerical MODE solutions are also compared

The Bidirectional Relationship Between Obstructive Sleep Apnea and Metabolic Disease

Obstructive sleep apnea (OSA) is a common sleep disorder, effecting 17% of the total population and 40–70% of the obese population (1, 2). Multiple studies have identified OSA as a critical risk factor for the development of obesity, diabetes, and cardiovascular diseases (3–5). Moreover, emerging evidence indicates that metabolic disorders can exacerbate OSA, creating a bidirectional relationship between OSA and metabolic physiology. In this review, we explore the relationship between glycemic control, insulin, and leptin as both contributing factors and products of OSA. We conclude that while insulin and leptin action may contribute to the development of OSA, further research is required to determine the mechanistic actions and relative contributions independent of body weight. In addition to increasing our understanding of the etiology, further research into the physiological mechanisms underlying OSA can lead to the development of improved treatment options for individuals with OSA

Association of Obstructive Sleep Apnea With Cardiovascular Outcomes in Patients With Acute Coronary Syndrome.

Background The prognostic significance of obstructive sleep apnea ( OSA ) in patients with acute coronary syndrome ( ACS ) in the contemporary era is unclear. We performed a large, prospective cohort study and did a landmark analysis to delineate the association of OSA with subsequent cardiovascular events after ACS onset. Methods and Results Between June 2015 and May 2017, consecutive eligible patients admitted for ACS underwent cardiorespiratory polygraphy during hospitalization. OSA was defined as an apnea-hypopnea index ≥15 events·h-1. The primary end point was major adverse cardiovascular and cerebrovascular event ( MACCE ), including cardiovascular death, myocardial infarction, stroke, ischemia-driven revascularization, or hospitalization for unstable angina or heart failure. OSA was present in 403 of 804 (50.1%) patients. During median follow-up of 1 year, cumulative incidence of MACCE was significantly higher in the OSA group than in the non- OSA group (log-rank, P=0.041). Multivariate analysis showed that OSA was nominally associated with incidence of MACCE (adjusted hazard ratio, 1.55; 95% CI, 0.94-2.57; P=0.085). In the landmark analysis, patients with OSA had 3.9 times the risk of incurring a MACCE after 1 year (adjusted hazard ratio, 3.87; 95% CI, 1.20-12.46; P=0.023), but no increased risk was found within 1-year follow-up (adjusted hazard ratio, 1.18; 95% CI, 0.67-2.09; P=0.575). No significant differences were found in the incidence of cardiovascular death, myocardial infarction, and ischemia-driven revascularization, except for a higher rate of hospitalization for unstable angina in the OSA group than in the non- OSA group (adjusted hazard ratio, 2.10; 95% CI, 1.09-4.05; P=0.027). Conclusions There was no independent correlation between OSA and 1-year MACCE after ACS . The increased risk associated with OSA was only observed after 1-year follow-up. Efficacy of OSA treatment as secondary prevention after ACS requires further investigation

Clinical significance of obstructive sleep apnea in patients with acute coronary syndrome in relation to diabetes status.

Objective: The prognostic significance of obstructive sleep apnea (OSA) in patients with acute coronary syndrome (ACS) according to diabetes mellitus (DM) status remains unclear. We aimed to elucidate the association of OSA with subsequent cardiovascular events in patients with ACS with or without DM. Research design and methods: In this prospective cohort study, consecutive eligible patients with ACS underwent cardiorespiratory polygraphy between June 2015 and May 2017. OSA was defined as an Apnea Hypopnea Index ≥15 events/hour. The primary end point was major adverse cardiovascular and cerebrovascular events (MACCEs), including cardiovascular death, myocardial infarction, stroke, ischemia-driven revascularization, or hospitalization for unstable angina or heart failure. Results: Among 804 patients, 248 (30.8%) had DM and 403 (50.1%) had OSA. OSA was associated with 2.5 times the risk of 1 year MACCE in patients with DM (22.3% vs 7.1% in the non-OSA group; adjusted HR (HR)=2.49, 95% CI 1.16 to 5.35, p=0.019), but not in patients without DM (8.5% vs 7.7% in the non-OSA group, adjusted HR=0.94, 95% CI 0.51 to 1.75, p=0.85). Patients with DM without OSA had a similar 1 year MACCE rate as patients without DM. The increased risk of events was predominately isolated to patients with OSA with baseline glucose or hemoglobin A1c levels above the median. Combined OSA and longer hypoxia duration (time with arterial oxygen saturation22 min) further increased the MACCE rate to 31.0% in patients with DM. Conclusions: OSA was associated with increased risk of 1 year MACCE following ACS in patients with DM, but not in non-DM patients. Further trials exploring the efficacy of OSA treatment in high-risk patients with ACS and DM are warranted

Preoperative STOP-BANG Scores and Postoperative Delirium and Coma in Thoracic Surgery Patients

Background Obstructive sleep apnea (OSA) is associated with higher rates of postoperative delirium. The relationship between preoperative OSA risk and postoperative delirium and coma in thoracic surgery patients hospitalized in the intensive care unit (ICU) is not well understood. This study tests the hypothesis that thoracic surgery patients hospitalized in ICU with a higher preoperative risk for OSA are more likely to develop postoperative delirium and coma, resulting in longer hospital stays. Methods Preoperative OSA risk was measured using the STOP-BANG questionnaire. STOP-BANG scores of ≥ 3 were defined as intermediate-high risk for OSA. 128 patients who underwent major thoracic surgery completed the STOP-BANG questionnaire preoperatively. The Richmond Agitation and Sedation Scale was used to assess level of consciousness. The Confusion Assessment Method for the ICU was used to assess for delirium. Linear regression was used to assess the relationship between risk of OSA and outcome measures. Results were adjusted for age, gender, body mass index, Charlson Comorbidity Index, instrumental activities of daily living, and surgery type. Results 96 out of 128 patients (76%) were in the intermediate-high risk OSA group. Adjusted analyses showed that the intermediate-high risk OSA group had a longer duration of postoperative ICU delirium and coma compared to the low risk OSA group (1.4 days ± 1.3 vs 0.9 days ± 1.4; P = 0.04). Total number of hospital days was not significantly different. Conclusions Higher preoperative risk for OSA in thoracic surgery patients was associated with a longer duration of postoperative delirium and coma