Learning and Transfer of Modulated Locomotor Controllers

We study a novel architecture and training procedure for locomotion tasks. A high-frequency, low-level "spinal" network with access to proprioceptive sensors learns sensorimotor primitives by training on simple tasks. This pre-trained module is fixed and connected to a low-frequency, high-level "cortical" network, with access to all sensors, which drives behavior by modulating the inputs to the spinal network. Where a monolithic end-to-end architecture fails completely, learning with a pre-trained spinal module succeeds at multiple high-level tasks, and enables the effective exploration required to learn from sparse rewards. We test our proposed architecture on three simulated bodies: a 16-dimensional swimming snake, a 20-dimensional quadruped, and a 54-dimensional humanoid. Our results are illustrated in the accompanying video at https://youtu.be/sboPYvhpraQComment: Supplemental video available at https://youtu.be/sboPYvhpra

Playing Atari with Deep Reinforcement Learning

We present the first deep learning model to successfully learn control policies directly from high-dimensional sensory input using reinforcement learning. The model is a convolutional neural network, trained with a variant of Q-learning, whose input is raw pixels and whose output is a value function estimating future rewards. We apply our method to seven Atari 2600 games from the Arcade Learning Environment, with no adjustment of the architecture or learning algorithm. We find that it outperforms all previous approaches on six of the games and surpasses a human expert on three of them.Comment: NIPS Deep Learning Workshop 201

Müokardi revaskulariseerimine pärgarteri kroonilise täieliku oklusiooni korral

Südame isheemiatõbi põhjustas Eestis 2021. aastal 2169 surma, neist 475 ägeda müokardiinfarkti tõttu (1). Isheemiatõve suure tervise-, suremus- ja haiguskaotuse tõttu tuleb seda ajakohaselt menetleda. Eluviisi muutmine ja järjepidev ravimite kasutamine võib olla edukas, kuid suurel osal patsientidest annab tulemuse vaid müokardi revaskulariseerimine (2). Artiklis on tutvustatud üht kindlat südame isheemiatõve alavormi – kroonilist täielikku oklusiooni –, mille korral on pärgarteri haru olnud täielikult sulgunud kauem kui 3 kuud. Eesmärk on kirjeldada kroonilise täieliku oklusiooni ravimeetodeid, nende näidustusi ja tulemusi, keskendudes perkutaanse koronaarinterventsiooni arengule ning selle kasutusele kroonilise täieliku oklusiooni ravis väheinvasiivse alternatiivina kirurgilisele müokardi revaskulariseerimisele

Satellite Operations Simulator for Cyber Exercises (SatOpSim)

INTRODUCTION SatOpSim is a virtual simulator for satellite operations deployed in a cyber range, an environment for cyber exercises, testing and validation It enables cyber exercise participants to simulate attacking and defending a satellite mission, expanding their cybersecurity skills into the space domain SatOpSim is designed to empower development, testing and validation of satellite communication systems in controlled environment

Many-body theory interpretation of deep inelastic scattering

We analyze data on deep inelastic scattering of electrons from the proton using ideas from standard many-body theory involving {\em bound} constituents subject to {\em interactions}. This leads us to expect, at large three-momentum transfer ${\bf{q}}$, scaling in terms of the variable $\tilde{y}=\nu-{\bf |q|}$. The response at constant ${\bf |q|}$ scales well in this variable. Interaction effects are manifestly displayed in this approach. They are illustrated in two examples.Comment: 10 pages, 4 figure