QT Measurement and Heart Rate Correction during Hypoglycemia: Is There a Bias?

Introduction. Several studies show that hypoglycemia causes QT interval prolongation. The aim of this study was to investigate the effect of QT measurement methodology, heart rate correction, and insulin types during hypoglycemia. Methods. Ten adult subjects with type 1 diabetes had hypoglycemia induced by intravenous injection of two insulin types in a cross-over design. QT measurements were done using the slope-intersect (SI) and manual annotation (MA) methods. Heart rate correction was done using Bazett's (QTcB) and Fridericia's (QTcF) formulas. Results. The SI method showed significant prolongation at hypoglycemia for QTcB (42(6) ms; P < .001) and QTcF (35(6) ms; P < .001). The MA method showed prolongation at hypoglycemia for QTcB (7(2) ms, P < .05) but not QTcF. No difference in ECG variables between the types of insulin was observed. Discussion. The method for measuring the QT interval has a significant impact on the prolongation of QT during hypoglycemia. Heart rate correction may also influence the QT during hypoglycemia while the type of insulin is insignificant. Prolongation of QTc in this study did not reach pathologic values suggesting that QTc prolongation cannot fully explain the dead-in-bed syndrome

Combining Reinforcement Learning with a Local Control Algorithm

We explore combining reinforcement learning with a hand-crafted local controller in a manner suggested by the chaotic control algorithm of Vincent, Schmitt and Vincent (1994). A closedloop controller is designed using conventional means that creates a domain of attraction about a target state. Chaotic behavior is used or induced to bring the system into this region, at which time the local controller is turned on to bring the system to the target state and stabilize it there. We describe experiments in which we use reinforcement learning instead of, and in addition to, chaotic behavior to learn an efficient policy for driving the system into the local controller’s domain of attraction. Using a simulated double pendulum, we illustrate how this method allows reinforcement learning to be effective in a problem that cannot be easily solved by reinforcement learning alone, and we show how reinforcement learning can improve upon the chaotic control algorithm when the domain of attraction can only be approximately determined. Similar results are shown using the Hénon map. This is a simple and effective way of extending reinforcement learning to more difficult problems

Learning macro-actions in reinforcement learning

We present a method for automatically constructing macro-actions from scratch from primitive actions during the reinforcement learning process. The overall idea is to reinforce the tendency to perform actionbafter actionaif such a pattern of actions has been rewarded. We test the method on a bicycle task, the car-on-the-hill task, the race-track task and some grid-world tasks. For the bicycle and race-track tasks the use of macro-actions approximately halves the learning time, while for one of the grid-world tasks the learning time is reduced by a factor of 5. The method did not work for the car-on-the-hill task for reasons we discuss in the conclusion.

Learning to Drive a Bicycle using Reinforcement Learning and Shaping

We present and solve a real-world problem of learning to drive a bicycle. We solve the problem by online reinforcement learning using the Sarsa()-algorithm. Then we solve the composite problem of learning to balance a bicycle and then drive to a goal. In our approach the reinforcement function is independent of the task the agent tries to learn to solve. 1 Introduction Here we consider the problem of learning to balance on a bicycle. Having done this we want to drive the bicycle to a goal. The second problem is not as straightforward as it may seem. The learning agent has to solve two problems at the same time: Balancing on the bicycle and driving to a specific place. Recently, ideas from behavioural psychology have been adapted by reinforcement learning to solve this type of problem. We will return to this in section 3. In reinforcement learning an agent interacts with an environment or a system. At each time step the agent receives information on the state of the system and chooses ..

Learning and Control in a Chaotic System

Most real world problems can be solved at least partially by simple, nonadaptive means. When we want to use an adaptive learning technique like reinforcement learning for practical purposes, we need to be able to cooperate with one or several of these partial solutions. In this paper we consider the problem of making a reinforcement learning agent cooperate with a hand-crafted local controller and a global chaotic controller, and designing a regime that improves upon these controller. 1 Introduction There exist a lot of methods for finding hand-crafted solutions to problems or methods that work locally. It is reasonable to believe that a general learning algorithm could benefit from those methods when trying to learn to solve the problem. We could think of local methods as a limited part of the state space that has a suggestion for a solution, while the agent is on its own in the rest of the state space. The situation could also be inverted: it could be required that the agent coopera..

Reinforcement Learning Based on Incomplete State Data

We construct and examine a network which is able to learn to control a system when parts of the state data from the system sometimes are missing. The network uses reinforcement learning and consists of an already existing agent like the actor-critic network introduced by Barto, Sutton and Anderson [Barto et al. 1983] and a novel expectation part. The network builds up an expectation of the next set of state data, and uses this expectation to choose the appropriate control action when parts of the state data are missing. 1 Introduction The step from supervised learning to reinforcement learning can roughly be characterised as a step towards less data about the system the agent has to control. It is thus possible to learn with less data than required for standard back-propagation learning¯but is it possible for an agent to learn a task with less data than required by standard reinforcement learning? As we shall see, the answer is yes. An agent may learn a task even when a component of t..

Combining reinforcement learning with a local control algorithm

We explore combining reinforcement learning with a hand-crafted local controller in a manner suggested by the chaotic control algorithm of Vincent, Schmitt and Vincent (1994). A closedloop controller is designed using conventional means that creates a domain of attraction about a target state. Chaotic behavior is used or induced to bring the system into this region, at which time the local controller is turned on to bring the system to the target state and stabilize it there. We describe experiments in which we use reinforcement learning instead of, and in addition to, chaotic behavior to learn an efficient policy for driving the system into the local controller’s domain of attraction. Using a simulated double pendulum, we illustrate how this method allows reinforcement learning to be effective in a problem that cannot be easily solved by reinforcement learning alone, and we show how reinforcement learning can improve upon the chaotic control algorithm when the domain of attraction can only be approximately determined. Similar results are shown using the Hénon map. This is a simple and effective way of extending reinforcement learning to more difficult problems. 1