Transfer learning for mobile robots

Abstract. In this project transfer learning can be defined as transferring previously learned knowledge to a new environment and making use of it to avoid obstacles. The feasibility of transfer learning was studied in a situation where a robot is given a task to navigate to a user-defined location in a virtual environment without hitting walls and utilizing reinforcement learning to teach the robot, which means that the robot will receive rewards according to the way it moves in the environment and how close it is to the goal location. In this project everything is done and tested in simulation. First the robot is trained in a standard environment, which is a simple hallway. It requires around 4000 iterations for the robot to learn better practices and reach the goal more frequently. When the training is done, the robot is moved to a test environment, which is otherwise similar to the standard one with the exception of a slanted floor, a ramp, in the beginning of the hallway. This proved to be an obstacle that the robot could not overcome without the help of sensor spoofing. Sensor spoofing in this case means inputting fake values to the robot’s laser sensor, which is responsible for detecting obstacles around the robot. The major target in this research was to transfer the previously learned data from the standard environment to the test environment and utilize sensor spoofing to help the robot overcome the slanted floor and eventually analyze if transfer learning helped the robot perform better. The performance can be compared by looking at the rewards received by the robot, since the robot receives highest rewards when reaching the goal location in the environment and negative rewards when crashing into walls. If transfer learning is beneficial for the robot, the robot should reach the goal point more frequently when making use of previously trained data from the standard environment and sensor spoofing in the test environment, compared to how it performs without them. This was also the result achieved. Even though the performance was not as good as it was without the ramp since without the ramp the robot reached the goal point every time after training around 200 episodes, the performance was better than it was without the trained model and sensor spoofing being used. As a result, transfer learning can be applied in virtual environments for mobile robots under certain restrictions. It can also be utilized in many other cases, this project is just one example. The codes and files used for this project are available on GitHub at https://github.com/lperala/Transfer_learning_for_mobile_robots.Tiivistelmä. Tässä projektissa oppimisen siirtäminen voidaan määrittää aiemmin opitun tiedon siirtämisenä uuteen ympäristöön ja sen hyödyntämisenä esteiden välttelyyn. Oppimisen siirtämisen toteutettavuutta tutkittiin tilanteessa, jossa robotille on annettu tehtävä navigoida käyttäjän määrittämään sijaintiin virtuaalisessa ympäristössä osumatta seiniin hyödyntäen vahvistavaa oppimista robotin opettamiseksi, joka tarkoittaa että robotti saa positiivisia palkkioita sen mukaan miten se liikkuu ympäristössä ja kuinka lähellä se on tavoitesijaintia. Tässä projektissa kaikki on tehty ja testattu simulaatiossa. Ensin robotti koulutetaan standardiympäristössä, joka on yksinkertainen käytävä. Robotti tarvitsee noin 4000 toistoa koulutusta, jotta se oppisi liikkumaan paremmin ja saavuttamaan tavoitesijaintinsa useammin. Kun koulutus on tehty, robotti siirretään testiympäristöön, joka on muuten samanlainen kuin standardiympäristö, mutta sisältää kaltevan rampin käytävän alussa. Tämä osottautui esteeksi, jonka yli robotti ei kyennyt liikkumaan ilman sensorin huijaamista. Sensorin huijaaminen tarkoitti tässä tapauksessa tekaistujen arvojen syöttämistä robotin lasersensorille, joka vastaa esteiden havaitsemisesta robotin ympärillä. Suurin tavoite projektissa oli siirtää aiemmin opittu data standardiympäristöstä testiympäristöön ja hyödyntää sensorin huijaamista auttaakseen robottia ylittämään ramppi ja lopulta analysoida oliko oppimisen siirtämisestä hyötyä robotin suoriutumisen kannalta. Suoriutumista voitiin tarkastella vertaamalla robotin keräämiä palkkioita, koska robotti saa isoimmat palkkionsa saavuttaessaan tavoitesijaintinsa ympäristössä ja taas negatiivisia palkkioita, mikäli se törmää seinään. Jos oppimisen siirtäminen on hyödyllistä, se tarkottaisi että robotti saavuttaisi tavoitesijainnin useammin kun se hyödyntää aiemmin opittua dataa kuin jos se suoriutuisi ilman opittua dataa. Tämä oli myös tulos johon päädyttiin. Vaikka suoriutuminen ei ollut yhtä hyvää kuin ilman ramppia, koska ilman ramppia robotti saavutti tavoitteensa jo 200 koulutusepisodin jälkeen, suoriutuminen oli parempaa kuin se oli täysin ilman koulutusta ja sensorin huijaamista. Tuloksena, oppimisen siirtämistä voidaan hyödyntää virtuaalisissa ympäristöissä mobiileille roboteille tiettyjen rajoituksin. Sitä voidaan myös hyödyntää monissa muissakin tapauksissa, tämä projekti on vain yksi esimerkki. Projektissa käytetyt tiedostot ovat saatavilla GitHubissa osoitteesta https://github.com/lperala/Transfer_learning_for_mobile_robots

Characterization of Mixed Monolayers of Phosphatidylcholine and a Dicationic Gemini Surfactant SS-1 with a Langmuir Balance: Effects Of DNA

AbstractMonolayers of a cationic gemini surfactant, 2,3-dimethoxy-1,4-bis(N-hexadecyl-N;N-dimethyl-ammonium)butane dibromide (abbreviated as SS-1) and its mixtures with 1-palmitoyl-2-oleoyl-sn-glycero-3-phosphocholine (POPC) were studied using a Langmuir balance. More specifically, we measured the force-area (π-A) curves and determined the elastic area compressibility modulus (Cs−1) as a function of lateral packing pressure and the mole fraction of the cationic lipid (XSS-1), with and without DNA in the subphase. Both SS-1 and POPC exhibited smooth compression isotherms, indicating their monolayers to be in the liquid expanded state. Even low contents (XSS-1<0.05) of SS-1 in a POPC monolayer condensed the film dramatically, up to 20% at 30mN/m. This effect is suggested to reflect reorientation of the P−-N+ dipole of the POPC headgroup. Accordingly, the magnitude of the condensing effect diminishes with XSS-1 and is not observed for mixed films of dioleoylglycerol and SS-1. Reorientation of the P−-N+ dipole is further supported by the pronounced increase in monolayer dipole potential ψ due to SS-1. The presence of DNA in the subphase affected the mixed POPC/SS-1 monolayers differently depending on the constituent lipid stoichiometry as well as on the DNA/SS-1 charge ratio. At a DNA concentration of 0.63μM (in base pairs) condensation of neat POPC monolayers was evident, and this effect remained up to XSS-1<0.5, corresponding to DNA/SS-1 charge ratio of 1.25. An expansion due to DNA, evident as an increase in ΔA/molecule, was observed at XSS-1>0.5. At a higher concentration of DNA (1.88μM base pairs) in the subphase corresponding to DNA/SS-1 charge ratio of 3.75 at XSS-1=0.5, condensation was observed at all values of XSS-1

Expression of inducible nitric oxide synthase in healthy pleura and in malignant mesothelioma

In this study we investigated the immunohistochemical expression of inducible nitric oxide synthase (iNOS) in a set of normal pleural mesothelial tissues, malignant mesotheliomas, mesothelioma cell lines and metastatic pleural adenocarcinomas. Furthermore, the expression of mRNA was assessed in four malignant mesothelioma cell lines in culture. Apoptosis and vascular density in malignant mesotheliomas was assessed by the TUNEL method and by immunohistochemistry with an antibody against FVIII-related antigen. Immunohistochemically mesothelial cells in non-neoplastic healthy pleural tissues were mostly negative for iNOS. Positivity for iNOS was observed in 28/38 (74%) and 24/25 (96%) of malignant mesotheliomas and metastatic pleural adenocarcinomas, respectively. Epithelial and mixed mesotheliomas expressed more often strong iNOS immunoreactivity compared to the sarcomatoid subtype (P = 0.023). Moreover, metastatic adenocarcinomas expressed more often iNOS positivity than mesotheliomas (P = 0.021). Experiments with the cell lines confirmed that malignant mesothelioma cells are capable of synthesizing iNOS. No significant association was found between iNOS expression and apoptosis or vascular density in malignant mesotheliomas. The higher expression of iNOS in the epithelial subtype of mesothelioma and pleural metastatic adenocarcinoma might be due to an increased sensitivity of these cell types to cytokine-mediated iNOS upregulation. The strong expression of iNOS suggests a putative role for NO in the growth and progression of these tumours. © 2000 Cancer Research Campaig

A minor role of asparaginase in predisposing to cerebral venous thromboses in adult acute lymphoblastic leukemia patients

Cerebral venous thrombosis (CVT) covers up to a third of all venous thromboses (VTs) detected in patients with acute lymphoblastic leukemia (ALL). It usually hampers patients' lives and may also endanger efficient leukemia treatment. Although many factors have been suggested to account for an elevated risk of VTs in patients with ALL, there still is a lack of studies focusing on CVTs and especially in the setting of adult ALL patients. We studied in our retrospective population-based cohort the occurrence, characteristics, as well as risk factors for VTs in 186 consecutively diagnosed Finnish adult ALL patients treated with a national pediatric-inspired treatment protocol ALL2000. In the risk factor analyses for VTs we found a distinction of the characteristics of the patients acquiring CVT from those with other kinds of VTs or without thrombosis. In contrast to previous studies we were also able to compare the effects of asparaginase in relation to CVT occurrence. Notably, more than half of the CVTs were diagnosed prior the administration of asparaginase which accentuates the role of other risk factors on the pathophysiology of CVT compared to truncal or central venous line (CVL) VTs in adult ALL patients

The effect of bone marrow microenvironment on the functional properties of the therapeutic bone marrow-derived cells in patients with acute myocardial infarction

<p>Abstract</p> <p>Background</p> <p>Treatment of acute myocardial infarction with stem cell transplantation has achieved beneficial effects in many clinical trials. The bone marrow microenvironment of ST-elevation myocardial infarction (STEMI) patients has never been studied even though myocardial infarction is known to cause an imbalance in the acid-base status of these patients. The aim of this study was to assess if the blood gas levels in the bone marrow of STEMI patients affect the characteristics of the bone marrow cells (BMCs) and, furthermore, do they influence the change in cardiac function after autologous BMC transplantation. The arterial, venous and bone marrow blood gas concentrations were also compared.</p> <p>Methods</p> <p>Blood gas analysis of the bone marrow aspirate and peripheral blood was performed for 27 STEMI patients receiving autologous stem cell therapy after percutaneous coronary intervention. Cells from the bone marrow aspirate were further cultured and the bone marrow mesenchymal stem cell (MSC) proliferation rate was determined by MTT assay and the MSC osteogenic differentiation capacity by alkaline phosphatase (ALP) activity assay. All the patients underwent a 2D-echocardiography at baseline and 4 months after STEMI.</p> <p>Results</p> <p>As expected, the levels of pO₂, pCO₂, base excess and HCO₃were similar in venous blood and bone marrow. Surprisingly, bone marrow showed significantly lower pH and Na⁺and elevated K⁺levels compared to arterial and venous blood. There was a positive correlation between the bone marrow pCO₂and HCO₃levels and MSC osteogenic differentiation capacity. In contrast, bone marrow pCO₂and HCO₃levels displayed a negative correlation with the proliferation rate of MSCs. Patients with the HCO₃level below the median value exhibited a more marked change in LVEF after BMC treatment than patients with HCO₃level above the median (11.13 ± 8.07% vs. 2.67 ± 11.89%, P = 0.014).</p> <p>Conclusions</p> <p>Low bone marrow pCO₂and HCO₃levels may represent the optimal environment for BMCs in terms of their efficacy in autologous stem cell therapy in STEMI patients.</p

p53 status of newly established acute myeloid leukaemia cell lines

We analysed the status of the p53 gene and protein in eight newly established acute myeloid leukaemia (AML) cell lines representing blast cells of either de novo leukaemia patients in first remission or patients with relapsed and chemotherapy-resistant disease causing their death. There were no mutations in the p53 gene in any of the cell lines as analysed by single-strand conformation polymorphism of amplified exons 5–8. However, the p53 protein was clearly and consistently expressed in all of these cell lines, as shown by immunohistochemistry, Western blotting and flow cytometry. The consistently expressed p53 protein was located in both the nucleus and the cytoplasm of all the cell lines and, as shown by flow cytometry, it was mostly in a conformation typical of the mutated protein. These AML cell lines offer a tool for studying the production and function of the p53 protein and its possible role in cell cycle regulation and chemoresistance as well as in the regulation of apoptosis in AML. © 1999 Cancer Research Campaig

A randomized phase II study of stem cell mobilization with cyclophosphamide plus G-CSF or G-CSF alone after lenalidomide-based induction in multiple myeloma

The most common means of mobilizing autologous stem cells is G-CSF alone or combined with cyclophosphamide (CY) to obtain sufficient CD34(+) cells for one to two transplants. There are few prospective, randomized studies investigating mobilization regimens in multiple myeloma (MM), especially after lenalidomide-based induction. We designed this prospective, randomized study to compare low-dose CY 2 g/m(2)+G-CSF (arm A) and G-CSF alone (arm B) after lenalidomide-based up-front induction in MM. Of the 80 initially randomized patients, 69 patients were evaluable, 34 and 35 patients in arms A and B, respectively. The primary end point was the proportion of patients achieving a yield of >= 3x10(6)/kg CD34(+) cells with 1 - 2 aphereses, which was achieved in 94% and 77% in arms A and B, respectively (P = 0.084). The median number of aphereses needed to reach the yield of >= 3x10(6)/kg was lower in arm A than in arm B (1 vs 2, P = 0.035). Two patients needed plerixafor in arm A and five patients in arm B (P = 0.428). Although CY-based mobilization was more effective, G-CSF alone was successful in a great majority of patients to reach the defined collection target after three cycles of lenalidomide-based induction.Peer reviewe

Resource Allocation Algorithm for GSM-OSC Cellular Systems

We consider one of the latest feature included in the Release 9 of the GSM/EDGE standard: the Orthogonal Sub Channel (OSC) transmission scheme. OSC aims at doubling the cell capacity by multiplexing two co-cell users on the same radio resource. In this work we deal with the challenge of finding the optimum pairing strategy among co-cell OSC users exploiting the Adaptive QPSK (AQPSK) modulation in both up- and down-link scenarios. The aim of the proposed scheduling algorithm is to i) find the best association among the users and the available OSC logical channels, and ii) select the optimum transmitting powers. The criterion for optimization is the minimization of the overall transmitted power constrained to service quality targets. The proposed scheduling algorithm is performed locally at the BS, exploiting channel state information reported by the users. Numerical results show significant power saving provided by the algorithm in heterogeneous scenarios with variable cell load

Lipid domain morphologies in phosphatidylcholine−ceramide monolayers

In cells, one of the main roles of ceramide-enriched membrane domains is to recruit or exclude intracellular signaling molecules and receptors, thereby facilitating signal transduction cascades. Accordingly, in model membranes, even low contents of ceramide segregate into lateral domains. The impact of the N-acyl chain on this segregation and on the morphology of the domains remains to be explored. Using Langmuir monolayers, we have systematically studied binary mixtures of 1,2-dimyristoyl-sn-glycero-3-phosphocholine (DMPC) and ceramide (2:1, molar ratio) and varied the N-acyl chain length of ceramide from 2 to 24 carbon atoms (Cer2 to Cer24). Fluid Cer2, Cer6, and Cer8/DMPC mixtures were miscible at all surface pressures. Longer ceramides, however, formed surface pressure-dependent immiscible mixtures with DMPC. The domain morphology under fluorescence microscopy after including a trace amount of fluorescent NBD-phosphatidylcholine into DMPC/Cer mixtures was found to be very sensitive to the N-acyl chain length. Shorter ceramides (Cer10-Cer14) formed flower-like (seaweed) domains, whereas longer ceramides (N-acyl chain length >14 carbon atoms) formed round and regular domains. We attribute the formation of the flower patterns to diffusive morphological instabilities during domain growth