Transfer learning for mobile robots

Abstract. In this project transfer learning can be defined as transferring previously learned knowledge to a new environment and making use of it to avoid obstacles. The feasibility of transfer learning was studied in a situation where a robot is given a task to navigate to a user-defined location in a virtual environment without hitting walls and utilizing reinforcement learning to teach the robot, which means that the robot will receive rewards according to the way it moves in the environment and how close it is to the goal location. In this project everything is done and tested in simulation. First the robot is trained in a standard environment, which is a simple hallway. It requires around 4000 iterations for the robot to learn better practices and reach the goal more frequently. When the training is done, the robot is moved to a test environment, which is otherwise similar to the standard one with the exception of a slanted floor, a ramp, in the beginning of the hallway. This proved to be an obstacle that the robot could not overcome without the help of sensor spoofing. Sensor spoofing in this case means inputting fake values to the robot’s laser sensor, which is responsible for detecting obstacles around the robot. The major target in this research was to transfer the previously learned data from the standard environment to the test environment and utilize sensor spoofing to help the robot overcome the slanted floor and eventually analyze if transfer learning helped the robot perform better. The performance can be compared by looking at the rewards received by the robot, since the robot receives highest rewards when reaching the goal location in the environment and negative rewards when crashing into walls. If transfer learning is beneficial for the robot, the robot should reach the goal point more frequently when making use of previously trained data from the standard environment and sensor spoofing in the test environment, compared to how it performs without them. This was also the result achieved. Even though the performance was not as good as it was without the ramp since without the ramp the robot reached the goal point every time after training around 200 episodes, the performance was better than it was without the trained model and sensor spoofing being used. As a result, transfer learning can be applied in virtual environments for mobile robots under certain restrictions. It can also be utilized in many other cases, this project is just one example. The codes and files used for this project are available on GitHub at https://github.com/lperala/Transfer_learning_for_mobile_robots.Tiivistelmä. Tässä projektissa oppimisen siirtäminen voidaan määrittää aiemmin opitun tiedon siirtämisenä uuteen ympäristöön ja sen hyödyntämisenä esteiden välttelyyn. Oppimisen siirtämisen toteutettavuutta tutkittiin tilanteessa, jossa robotille on annettu tehtävä navigoida käyttäjän määrittämään sijaintiin virtuaalisessa ympäristössä osumatta seiniin hyödyntäen vahvistavaa oppimista robotin opettamiseksi, joka tarkoittaa että robotti saa positiivisia palkkioita sen mukaan miten se liikkuu ympäristössä ja kuinka lähellä se on tavoitesijaintia. Tässä projektissa kaikki on tehty ja testattu simulaatiossa. Ensin robotti koulutetaan standardiympäristössä, joka on yksinkertainen käytävä. Robotti tarvitsee noin 4000 toistoa koulutusta, jotta se oppisi liikkumaan paremmin ja saavuttamaan tavoitesijaintinsa useammin. Kun koulutus on tehty, robotti siirretään testiympäristöön, joka on muuten samanlainen kuin standardiympäristö, mutta sisältää kaltevan rampin käytävän alussa. Tämä osottautui esteeksi, jonka yli robotti ei kyennyt liikkumaan ilman sensorin huijaamista. Sensorin huijaaminen tarkoitti tässä tapauksessa tekaistujen arvojen syöttämistä robotin lasersensorille, joka vastaa esteiden havaitsemisesta robotin ympärillä. Suurin tavoite projektissa oli siirtää aiemmin opittu data standardiympäristöstä testiympäristöön ja hyödyntää sensorin huijaamista auttaakseen robottia ylittämään ramppi ja lopulta analysoida oliko oppimisen siirtämisestä hyötyä robotin suoriutumisen kannalta. Suoriutumista voitiin tarkastella vertaamalla robotin keräämiä palkkioita, koska robotti saa isoimmat palkkionsa saavuttaessaan tavoitesijaintinsa ympäristössä ja taas negatiivisia palkkioita, mikäli se törmää seinään. Jos oppimisen siirtäminen on hyödyllistä, se tarkottaisi että robotti saavuttaisi tavoitesijainnin useammin kun se hyödyntää aiemmin opittua dataa kuin jos se suoriutuisi ilman opittua dataa. Tämä oli myös tulos johon päädyttiin. Vaikka suoriutuminen ei ollut yhtä hyvää kuin ilman ramppia, koska ilman ramppia robotti saavutti tavoitteensa jo 200 koulutusepisodin jälkeen, suoriutuminen oli parempaa kuin se oli täysin ilman koulutusta ja sensorin huijaamista. Tuloksena, oppimisen siirtämistä voidaan hyödyntää virtuaalisissa ympäristöissä mobiileille roboteille tiettyjen rajoituksin. Sitä voidaan myös hyödyntää monissa muissakin tapauksissa, tämä projekti on vain yksi esimerkki. Projektissa käytetyt tiedostot ovat saatavilla GitHubissa osoitteesta https://github.com/lperala/Transfer_learning_for_mobile_robots

Oral Levosimendan Increases Cerebral Blood Flow Velocities in Patients with a History of Stroke or Transient Ischemic Attack : A Pilot Safety Study

Peer reviewe

Levosimendan: current data, clinical use and future development.

Levosimendan is an inodilator indicated for the short-term treatment of acutely decompensated severe chronic heart failure, and in situations where conventional therapy is not considered adequate. The principal pharmacological effects of levosimendan are (a) increased cardiac contractility by calcium sensitisation of troponin C, (b) vasodilation, and (c) cardioprotection. These last two effects are related to the opening of sarcolemmal and mitochondrial potassium-ATP channels, respectively. Data from clinical trials indicate that levosimendan improves haemodynamics with no attendant significant increase in cardiac oxygen consumption and relieves symptoms of acute heart failure; these effects are not impaired or attenuated by the concomitant use of beta-blockers. Levosimendan also has favourable effects on neurohormone levels in heart failure patients. Levosimendan is generally well tolerated in acute heart failure patients: the most common adverse events encountered in this setting are hypotension, headache, atrial fibrillation, hypokalaemia and tachycardia. Levosimendan has also been studied in other therapeutic applications, particularly cardiac surgery - in which it has shown a range of beneficial haemodynamic and cardioprotective effects, and a favourable influence on clinical outcomes - and has been evaluated in repetitive dosing protocols in patients with advanced chronic heart failure. Levosimendan has shown preliminary positive effects in a range of conditions requiring inotropic support, including right ventricular failure, cardiogenic shock, septic shock, and Takotsubo cardiomyopathy

Levosimendan meta-analyses : Is there a pattern in the effect on mortality?

Background: Levosimendan is an inodilator developed for treatment of acute heart failure and other cardiac conditions where the use of an inodilator is considered appropriate. Levosimendan has been studied in different therapeutic settings including acutely decompensated chronic heart failure, advanced heart failure, right ventricular failure, cardiogenic shock, septic shock, and cardiac and non-cardiac surgery. This variety of data has been re-analysed in 25 meta-analyses from 15 different international research groups, based on different rationales to select the studies included. Methods: We here review all previously published meta-analyses on levosimendan to determine any common denominators for its effects on patient mortality. In addition, we also perform a comparative meta-analysis of the six phase II and III randomized double-blind trials which were taken into consideration by the regulatory authorities for the purpose of introducing levosimendan into the market. Results: Irrespective of clinical setting or comparator, all meta-analyses consistently show benefits for levosimendan, with lower relative risk (or odds ratio) for patient mortality. In 3/25 of the meta-analyses these beneficial trends did not reach statistical significance, while in 22/25 significance was reached. The relative risk is consistent overall, and very similar to that obtained in our own meta-analysis that considered only the 'regulatory' studies. Conclusion: The existing meta-analyses, now based on a population of over 6000 patients, provide the general message of significant benefits for levosimendan in terms of patient mortality. The weight of evidence is now clearly in favour of usefulness/efficacy of levosimendan, with data from multiple randomized trials and meta-analyses. (C) 2016 The Authors. Published by Elsevier Ireland Ltd.Peer reviewe

Repetitive levosimendan treatment in the management of advanced heart failure

Inotropes may be an appropriate treatment for patients with advanced heart failure (AdHF) who remain highly symptomatic despite optimized standard therapies. Objectives for inotrope use in these situations include relief of symptoms and improvement of quality of life, and reduction in unplanned hospitalizations and the costs associated with such episodes. All of these goals must be attained without compromising survival. Encouraging findings with intermittent cycles of intravenous levosimendan have emerged from a range of exploratory studies and from three larger controlled trials (LevoRep, LION-HEART, and LAICA) which offered some evidence of clinical advantage. In these settings, however, obtaining statistically robust data may prove elusive due to the difficulties of endpoint assessment in a complex medical condition with varying presentation and trajectory. Adoption of a composite clinical endpoint evaluated in a hierarchical manner may offer a workable solution to this problem. Such an instrument can explore the proposition that repetitive administration of levosimendan early in the period after discharge from an acute episode of worsening heart failure may be associated with greater subsequent clinical stability vis-à-vis standard therapy. The use of this methodology to develop a 'stability score' for each patient means that all participants in such a trial contribute to the overall outcome analysis through one or more of the hierarchical endpoints; this has helpful practical implications for the number of patients needed and the length of follow-up required to generate endpoint data. The LeoDOR study (NCT03437226), outlined in this review, has been designed to explore this new approach to outcome assessment in AdHF

Preoperative and perioperative use of levosimendan in cardiac surgery: European expert opinion

In cardiac surgery, postoperative low cardiac output has been shown to correlate with increased rates of organ failure and mortality. Catecholamines have been the standard therapy for many years, although they carry substantial risk for adverse cardiac and systemic effects, and have been reported to be associated with increased mortality. On the other hand, the calcium sensitiser and potassium channel opener levosimendan has been shown to improve cardiac function with no imbalance in oxygen consumption, and to have protective effects in other organs. Numerous clinical trials have indicated favourable cardiac and non-cardiac effects of preoperative and perioperative administration of levosimendan. A panel of 27 experts from 18 countries has now reviewed the literature on the use of levosimendan in on-pump and off-pump coronary artery bypass grafting and in heart valve surgery. This panel discussed the published evidence in these various settings, and agreed to vote on a set of questions related to the cardioprotective effects of levosimendan when administered preoperatively, with the purpose of reaching a consensus on which patients could benefit from the preoperative use of levosimendan and in which kind of procedures, and at which doses and timing should levosimendan be administered. Here, we present a systematic review of the literature to report on the completed and ongoing studies on levosimendan, including the newly commenced LEVO-CTS phase III study (NCT02025621), and on the consensus reached on the recommendations proposed for the use of preoperative levosimendan

Levosimendan in Acute and Advanced Heart Failure : an Expert Perspective on Posology and Therapeutic Application

Levosimendan, a calcium sensitizer and potassium channel-opener, is widely appreciated by many specialist heart failure practitioners for its effects on systemic and pulmonary hemodynamics and for the relief of symptoms of acute heart failure. The drug's impact on mortality in large randomized controlled trials has been inconsistent or inconclusive but, in contrast to conventional inotropes, there have been no indications of worsened survival and some signals of improved heart failure-related quality of life. For this reason, levosimendan has been proposed as a safer inodilator option than traditional agents in settings, such as advanced heart failure. Positive effects of levosimendan on renal function have also been described. At the HEART FAILURE 2018 congress of the Heart Failure Association of the European Society of Cardiology, safe and effective use levosimendan in acute and advanced heart failure was examined in a series of expert tutorials. The proceedings of those tutorials are summarized in this review, with special reference to advanced heart failure and heart failure with concomitant renal dysfunction. Meta-analysis of clinical trials data is supportive of a renal-protective effect of levosimendan, while physiological observations suggest that this effect is exerted at least in part via organ-specific effects that may include selective vasodilation of glomerular afferent arterioles and increased renal blood flow, with no compromise of renal oxygenation. These lines of evidence require further investigation and their clinical significance needs to be evaluated in specifically designed prospective trials.Peer reviewe

Intravenous levosimendan-norepinephrine combination during off-pump coronary artery bypass grafting in a hemodialysis patient with severe myocardial dysfunction

This the case of a 63 year-old man with end-stage renal disease (on chronic hemodialysis), unstable angina and significantly impaired myocardial contractility with low left ventricular ejection fraction, who underwent off-pump one vessel coronary bypass surgery. Combined continuous levosimendan and norepinephrine infusion (at 0.07 μg/kg/min and 0.05 μg/kg/min respectively) started immediately after anesthesia induction and continued for 24 hours. The levosimendan/norepinephrine combination helped maintain an appropriate hemodynamic profile, thereby contributing to uneventful completion of surgery and postoperative hemodynamic stability. Although levosimendan is considered contraindicated in ESRD patients, this case report suggests that combined perioperative levosimendan/norepinephrine administration can be useful in carefully selected hemodialysis patients with impaired myocardial contractility and ongoing myocardial ischemia, who undergo off-pump myocardial revascularization surgery

Repetitive use of levosimendan for treatment of chronic advanced heart failure: Clinical evidence, practical considerations, and perspectives: An expert panel consensus

Background The intravenous inodilator levosimendan was developed for the treatment of patients with acutely decompensated heart failure. In the last decade scientific and clinical interest has arisen for its repetitive or intermittent use in patients with advanced chronic, but not necessarily acutely decompensated, heart failure. Recent studies have suggested long-lasting favourable effects of levosimendan when administered repetitively, in terms of haemodynamic parameters, neurohormonal and inflammatory markers, and clinical outcomes. The existing data, however, requires further exploration to allow for definitive conclusions on the safety and clinical efficacy of repetitive use of levosimendan. Methods and results A panel of 30 experts from 15 countries convened to review and discuss the existing data, and agreed on the patient groups that can be considered to potentially benefit from intermittent treatment with levosimendan. The panel gave recommendations regarding patient dosing and monitoring, derived from the available evidence and from clinical experience. Conclusions The current data suggest that in selected patients and support out-of-hospital care, intermittent/repetitive levosimendan can be used in advanced heart failure to maintain patient stability. Further studies are needed to focus on morbidity and mortality outcomes, dosing intervals, and patient monitoring. Recommendations for the design of further clinical studies are made