Hierarchical Policy Search via Return-Weighted Density Estimation

Learning an optimal policy from a multi-modal reward function is a challenging problem in reinforcement learning (RL). Hierarchical RL (HRL) tackles this problem by learning a hierarchical policy, where multiple option policies are in charge of different strategies corresponding to modes of a reward function and a gating policy selects the best option for a given context. Although HRL has been demonstrated to be promising, current state-of-the-art methods cannot still perform well in complex real-world problems due to the difficulty of identifying modes of the reward function. In this paper, we propose a novel method called hierarchical policy search via return-weighted density estimation (HPSDE), which can efficiently identify the modes through density estimation with return-weighted importance sampling. Our proposed method finds option policies corresponding to the modes of the return function and automatically determines the number and the location of option policies, which significantly reduces the burden of hyper-parameters tuning. Through experiments, we demonstrate that the proposed HPSDE successfully learns option policies corresponding to modes of the return function and that it can be successfully applied to a challenging motion planning problem of a redundant robotic manipulator.Comment: The 32nd AAAI Conference on Artificial Intelligence (AAAI 2018), 9 page

Newsletter : Center for Southeast Asian Studies, Kyoto University No.73

Looking Back over 50 Years/50年のあゆみを振りかえり … Kono Yasuyuki/河野 泰之 [3]CSEAS Celebrates 50th Anniversary/東南アジア研究所創立50周年記念式典・記念祝賀会の開催 … Hayami Yoko/速水 洋子 [4] The Life of a Research Institute (Kenkyu-sho no Inochi)/ 研究所のいのち … Tachimoto Narifumi/立本 成文 [5] Looking to the Past, Inspiring the Future/ 回顧と展望 … Mieno Fumiharu/三重野 文晴 [10]The SEASIA 2015 Conference: A Watershed in Southeast Asian Studies/SEASIA 2015 国際シンポジウム: 東南アジア研究における分水嶺 … Janus Isaac V. Nolasco/ジャヌス・イサーク・V・ノラスコ [26] Towards a Region of New Nations/ 新たな国々からなる一つの地域に向けて … Wang Gungwu/ワン・ガンウー [29] Region and Academe in Turbulent Times/ 激動の時代における地域と学問 … Pasuk Phongpaichit/パースック・ポーンパイチット [34]Three Cheers for the Hare that Did Not Stop Running to Take a Nap: In Celebration of the Life of Benedict Anderson (1936–2015)/ひと眠りも立ち止まりもしなかったウサギに乾杯!!!: ベネディクト・アンダーソンという生き方(1936–2015)を頌して … Kato Tsuyoshi/加藤 剛 [38] Southeast Asian Scholars at the Edge/ 瀬戸際に立つ東南アジア研究者たち … Loh Kah Seng/ロー・カー・セン [42

Modeling Megacity Medical System Response to a CBRNE Event

The collaborative effectiveness of the public health system (PHS) and the Army Medical Department (AMEDD) is limited in the case of a 10-kiloton (kt) nuclear event on a megacity due to an overall lack of knowledge and understanding among agencies. This study details an exhaustive analysis of the current medical response system using New York City as a case study. Through the problem definition phase of the Systems Decision Process (SDP), this report identifies operational gaps existing at different levels within the system. Identified operational gaps existed at the local, state, and federal levels in the areas of resources, communication, and planning within the following agencies: Sloan Kettering Memorial Hospital, the Office of Emergency Management (OEM), the Federal Emergency Management Agency (FEMA), Health and Human Services (HHS), and the United States Department of Veteran Affairs (VA). Evaluation of the operational gaps illustrated the areas which were most vulnerable. The current analysis suggests that the system in place requires adjustments of the identified gaps so that maximum efficiency can be achieved

Return to Battleship Island

No abstract available

Impact of the 2011 Tohoku Earthquake on the Use of Tidal Flats: A Case Study in Inner Tokyo Bay

Coastal areas have recently been revaluated to understand the relationship between ecosystem services and humans. However, devastation from the 2011 earthquake and tsunami in Japan might have limited the use of ecosystem services. Thus, we investigate the effects of the earthquake and tsunami on the use of tidal flats. We used questionnaires to evaluate changes in the number and anxiety of users for each type of use of the Sanbanze tidal flat in the innermost part of Tokyo Bay. We also conducted a field survey to verify the trends. We found that clamming and walking or jogging were the most common uses in the surveyed area. However, clamming activity decreased after the earthquake as indicated by field observation (73% decrease in spring long holiday season) and questionnaires (64%). Other clamming ground in Tokyo Bay also showed a similar decrease (62%). Many visitors were anxious about the damaged infrastructure. Compared to other types of use such as swimming or total tourism in the tourism statistics, the recovery phase was slower for clamming using the data of other locations. Considering these evidence, the innermost part of Tokyo Bay can be classified as an earthquake-devastated area, from the view point of ecosystem services. Key Words： behavioral change, tourism industry, Great East Japan Earthquake of 2011, marine leisure use, shellfish gathering, marine ecosystem servicesOpenAcces