Transfer learning for mobile robots

Abstract. In this project transfer learning can be defined as transferring previously learned knowledge to a new environment and making use of it to avoid obstacles. The feasibility of transfer learning was studied in a situation where a robot is given a task to navigate to a user-defined location in a virtual environment without hitting walls and utilizing reinforcement learning to teach the robot, which means that the robot will receive rewards according to the way it moves in the environment and how close it is to the goal location. In this project everything is done and tested in simulation. First the robot is trained in a standard environment, which is a simple hallway. It requires around 4000 iterations for the robot to learn better practices and reach the goal more frequently. When the training is done, the robot is moved to a test environment, which is otherwise similar to the standard one with the exception of a slanted floor, a ramp, in the beginning of the hallway. This proved to be an obstacle that the robot could not overcome without the help of sensor spoofing. Sensor spoofing in this case means inputting fake values to the robot’s laser sensor, which is responsible for detecting obstacles around the robot. The major target in this research was to transfer the previously learned data from the standard environment to the test environment and utilize sensor spoofing to help the robot overcome the slanted floor and eventually analyze if transfer learning helped the robot perform better. The performance can be compared by looking at the rewards received by the robot, since the robot receives highest rewards when reaching the goal location in the environment and negative rewards when crashing into walls. If transfer learning is beneficial for the robot, the robot should reach the goal point more frequently when making use of previously trained data from the standard environment and sensor spoofing in the test environment, compared to how it performs without them. This was also the result achieved. Even though the performance was not as good as it was without the ramp since without the ramp the robot reached the goal point every time after training around 200 episodes, the performance was better than it was without the trained model and sensor spoofing being used. As a result, transfer learning can be applied in virtual environments for mobile robots under certain restrictions. It can also be utilized in many other cases, this project is just one example. The codes and files used for this project are available on GitHub at https://github.com/lperala/Transfer_learning_for_mobile_robots.Tiivistelmä. Tässä projektissa oppimisen siirtäminen voidaan määrittää aiemmin opitun tiedon siirtämisenä uuteen ympäristöön ja sen hyödyntämisenä esteiden välttelyyn. Oppimisen siirtämisen toteutettavuutta tutkittiin tilanteessa, jossa robotille on annettu tehtävä navigoida käyttäjän määrittämään sijaintiin virtuaalisessa ympäristössä osumatta seiniin hyödyntäen vahvistavaa oppimista robotin opettamiseksi, joka tarkoittaa että robotti saa positiivisia palkkioita sen mukaan miten se liikkuu ympäristössä ja kuinka lähellä se on tavoitesijaintia. Tässä projektissa kaikki on tehty ja testattu simulaatiossa. Ensin robotti koulutetaan standardiympäristössä, joka on yksinkertainen käytävä. Robotti tarvitsee noin 4000 toistoa koulutusta, jotta se oppisi liikkumaan paremmin ja saavuttamaan tavoitesijaintinsa useammin. Kun koulutus on tehty, robotti siirretään testiympäristöön, joka on muuten samanlainen kuin standardiympäristö, mutta sisältää kaltevan rampin käytävän alussa. Tämä osottautui esteeksi, jonka yli robotti ei kyennyt liikkumaan ilman sensorin huijaamista. Sensorin huijaaminen tarkoitti tässä tapauksessa tekaistujen arvojen syöttämistä robotin lasersensorille, joka vastaa esteiden havaitsemisesta robotin ympärillä. Suurin tavoite projektissa oli siirtää aiemmin opittu data standardiympäristöstä testiympäristöön ja hyödyntää sensorin huijaamista auttaakseen robottia ylittämään ramppi ja lopulta analysoida oliko oppimisen siirtämisestä hyötyä robotin suoriutumisen kannalta. Suoriutumista voitiin tarkastella vertaamalla robotin keräämiä palkkioita, koska robotti saa isoimmat palkkionsa saavuttaessaan tavoitesijaintinsa ympäristössä ja taas negatiivisia palkkioita, mikäli se törmää seinään. Jos oppimisen siirtäminen on hyödyllistä, se tarkottaisi että robotti saavuttaisi tavoitesijainnin useammin kun se hyödyntää aiemmin opittua dataa kuin jos se suoriutuisi ilman opittua dataa. Tämä oli myös tulos johon päädyttiin. Vaikka suoriutuminen ei ollut yhtä hyvää kuin ilman ramppia, koska ilman ramppia robotti saavutti tavoitteensa jo 200 koulutusepisodin jälkeen, suoriutuminen oli parempaa kuin se oli täysin ilman koulutusta ja sensorin huijaamista. Tuloksena, oppimisen siirtämistä voidaan hyödyntää virtuaalisissa ympäristöissä mobiileille roboteille tiettyjen rajoituksin. Sitä voidaan myös hyödyntää monissa muissakin tapauksissa, tämä projekti on vain yksi esimerkki. Projektissa käytetyt tiedostot ovat saatavilla GitHubissa osoitteesta https://github.com/lperala/Transfer_learning_for_mobile_robots

Graft Neutrophil Sequestration and Concomitant Tissue Plasminogen Activator Release During Reperfusion in Clinical Kidney Transplantation

Background. Inflammation, coagulation, and fibrinolysis are tightly linked together. Reperfusion after transient ischemia activates both neutrophils, coagulation, and fibrinolysis. Experimental data suggest that tissue plasminogen activator (tPA) regulates renal neutrophil influx in kidney ischemia and reperfusion injury. Methods. In 30 patients undergoing kidney transplantation, we measured renal neutrophil sequestration and tPA release from blood samples drawn from the supplying artery and renal vein early after reperfusion. tPA antigen levels were measured using a commercial enzyme-linked immunosorbent assay kit. For each parameter, transrenal difference (Delta) was calculated by subtracting the value of the arterial sample (ingoing blood) from the value of the venous sample (outgoing blood). Results. Positive transrenal gradients of tPA antigen occurred at 1 minute [Delta = 14 (3-46) ng/mL, P <.01] and 5 minutes [Delta = 5 (-3 to 27) ng/mL, P <.01] after reperfusion. At 5 minutes after reperfusion, a negative transrenal gradient of neutrophils was observed [Delta = -0.17 (-1.45 to 0.24) x 10E9 cells/L, P <.001]. At 1 minute after reperfusion, neutrophil sequestration into the kidney (ie, negative transrenal neutrophil count) correlated significantly with tPA release from the kidney (ie, positive transrenal tPA concentration), (R = -0.513 and P = .006). Conclusions. The findings suggest a proinflammatory role for tPA in ischemia and reperfusion injury in human kidney transplantation.Peer reviewe

Neutron area survey instrument measurements in the EVIDOS project

Neutron survey instruments have been exposed at all the measurement locations used in the EVIDOS project. These results have an important impact in the interpretation of the results from the project, since operationally the survey instrument will be used for an initial assessment of and routine monitoring of the ambient dose equivalent dose rate. Additionally, since the response of these instruments is in some cases very well characterised, their systematic deviations from the reference quantities provide an important verification of the determination of those quantitie

Evaluation of individual dosimetry in mixed neutron and photon radiation fields (EVIDOS). Part II: conclusions and recommendations

The paper presents the main conclusions and recommendations derived from the EVIDOS project, which is supported by the European Commission within the 5th Framework Programme. EVIDOS aims at evaluating state of the art neutron dosimetry techniques in representative workplaces of the nuclear industry with complex mixed neutron-photon radiation fields. This analysis complements a series of individual papers which present detailed results and it summarises the main findings from a practical point of view. Conclusions and recommendations are given concerning characterisation of radiation fields, methods to derive radiation protection quantities and dosemeter result

Evaluation of individual dosimetry in mixed neutron and photon radiation fields (EVIDOS). Part I: scope and methods of the project

Supported by the European Commission, the EVIDOS project started in November 2001 with the broad goal of evaluating state of the art dosimetry techniques in representative workplaces of the nuclear industry. Seven European institutes joined efforts with end users at nuclear power plants, at fuel processing and reprocessing plants, and at transport and storage facilities. A comprehensive programme was devised to evaluate capabilities and limitations of standard and innovative personal dosemeters in relation to the mixed neutron-photon fields of concern to the nuclear industry. This paper describes the criteria behind the selection of dosimetry techniques and workplaces that were analysed, as well as the organisation of the measurement campaigns. Particular emphasis was placed on the evaluation of a variety of electronic personal dosemeters, either commercially available or previously developed by the partners. The estimates provided by these personal dosemeters were compared to reference values of dose equivalent quantities derived from spectrometry and fluence-to-dose equivalent conversion coefficients. Spectrometry was performed both with conventional multisphere and with some original instrumentation providing energy and direction resolution, based on silicon detectors and superheated drop detectors mounted on or in spherical moderators. The results were collected in a large, searchable database and are intended to be used in the harmonisation of dosimetric procedures for mixed radiation fields and for the approval of dosimetry services in Europ

Evaluation of individual monitoring in mixed neutron/photon fields: mid-term results from the EVIDOS project

EVIDOS is an EC sponsored project that aims at an evaluation and improvement of radiation protection dosimetry in mixed neutron/photon fields. This is performed through spectrometric and dosimetric investigations during different measurement campaigns in representative workplaces of the nuclear industry. The performance of routine and, in particular, novel personal dosemeters and survey instruments is tested in selected workplace fields. Reference values for the dose equivalent quantities, H*(10) and Hp(10) and the effective dose E, are determined using different spectrometers that provide the energy distribution of the neutron fluence and using newly developed devices that determine the energy and directional distribution of the neutron fluence. The EVIDOS project has passed the mid-term, and three measurement campaigns have been performed. This paper will give an overview and some new results from the third campaign that was held in Mol (Belgium), around the research reactor VENUS and in the MOX producing plant of Belgonucléair

Electronic neutron personal dosemeters: their performance in mixed radiation fields in nuclear power plants

This work describes spectral distributions of neutrons obtained as function of energy and direction at four workplace fields at the Krümmel reactor in Germany. Values of personal dose equivalent Hp(10) and effective dose E are determined for different directions of a person's orientation in these fields and readings of personal neutron dosemeters—especially electronic dosemeters—are discussed with respect to Hp(10) and

Summary of personal neutron dosemeter results obtained within the EVIDOS project

Within the EC project EVIDOS (‘Evaluation of Individual Dosimetry in Mixed Neutron and Photon Radiation Fields'), different types of active neutron personal dosemeters (and some passive ones) were tested in workplace fields at nuclear installations in Europe. The results of the measurements which have been performed up to now are summarised and compared to our currently best estimates of the personal dose equivalent Hp(10). Under- and over-readings by more than a factor of two for the same dosemeter in different workplace fields indicate that in most cases the use of field-specific correction factors is require

Individual neutron monitoring in workplaces with mixed neutron/photon radiation

EVIDOS (‘evaluation of individual dosimetry in mixed neutron and photon radiation fields') is an European Commission (EC)-sponsored project that aims at a significant improvement of radiation protection dosimetry in mixed neutron/photon fields via spectrometric and dosimetric investigations in representative workplaces of the nuclear industry. In particular, new spectrometry methods are developed that provide the energy and direction distribution of the neutron fluence from which the reference dosimetric quantities are derived and compared to the readings of dosemeters. The final results of the project will be a comprehensive set of spectrometric and dosimetric data for the workplaces and an analysis of the performance of dosemeters, including novel electronic dosemeters. This paper gives an overview of the project and focuses on the results from measurements performed in calibration fields with broad energy distributions (simulated workplace fields) and on the first results from workplaces in the nuclear industry, inside a boiling water reactor and around a spent fuel transport cas