Transfer of Reinforcement Learning-Based Controllers from Model- to Hardware-in-the-Loop

The process of developing control functions for embedded systems is resource-, time-, and data-intensive, often resulting in sub-optimal cost and solutions approaches. Reinforcement Learning (RL) has great potential for autonomously training agents to perform complex control tasks with minimal human intervention. Due to costly data generation and safety constraints, however, its application is mostly limited to purely simulated domains. To use RL effectively in embedded system function development, the generated agents must be able to handle real-world applications. In this context, this work focuses on accelerating the training process of RL agents by combining Transfer Learning (TL) and X-in-the-Loop (XiL) simulation. For the use case of transient exhaust gas re-circulation control for an internal combustion engine, use of a computationally cheap Model-in-the-Loop (MiL) simulation is made to select a suitable algorithm, fine-tune hyperparameters, and finally train candidate agents for the transfer. These pre-trained RL agents are then fine-tuned in a Hardware-in-the-Loop (HiL) system via TL. The transfer revealed the need for adjusting the reward parameters when advancing to real hardware. Further, the comparison between a purely HiL-trained and a transferred agent showed a reduction of training time by a factor of 5.9. The results emphasize the necessity to train RL agents with real hardware, and demonstrate that the maturity of the transferred policies affects both training time and performance, highlighting the strong synergies between TL and XiL simulation

Cloud-Based Reinforcement Learning in Automotive Control Function Development

Automotive control functions are becoming increasingly complex and their development is becoming more and more elaborate, leading to a strong need for automated solutions within the development process. Here, reinforcement learning offers a significant potential for function development to generate optimized control functions in an automated manner. Despite its successful deployment in a variety of control tasks, there is still a lack of standard tooling solutions for function development based on reinforcement learning in the automotive industry. To address this gap, we present a flexible framework that couples the conventional development process with an open-source reinforcement learning library. It features modular, physical models for relevant vehicle components, a co-simulation with a microscopic traffic simulation to generate realistic scenarios, and enables distributed and parallelized training. We demonstrate the effectiveness of our proposed method in a feasibility study to learn a control function for automated longitudinal control of an electric vehicle in an urban traffic scenario. The evolved control strategy produces a smooth trajectory with energy savings of up to 14%. The results highlight the great potential of reinforcement learning for automated control function development and prove the effectiveness of the proposed framework

Elafin drives poor outcome in high grade serous ovarian cancers and basal-like breast tumors

High grade serous ovarian carcinoma (HGSOC) and basal-like breast cancer (BLBC) share many features including TP53 mutations, genomic instability and poor prognosis. We recently reported that Elafin is overexpressed by HGSOC and is associated with poor overall survival. Here, we confirmed that Elafin overexpression is associated with shorter survival in 1000 HGSOC patients. Elafin confers a proliferative advantage to tumor cells through activation of the MAP kinase pathway. This mitogenic effect can be neutralized by RNA interference, specific antibodies, and a MEK inhibitor. Elafin expression in patient-derived samples was also associated with chemoresistance and strongly correlates with bcl-xL expression. We extended these findings into examination of 1100 primary breast tumors and six breast cancer cell lines. We observed that Elafin is overexpressed and secreted specifically by BLBC tumors and cell lines, leading to a similar mitogenic effect through activation of the MAP kinase pathway. Here too, Elafin overexpression is associated with poor overall survival, suggesting that it may serve as a biomarker and therapeutic target in this setting

UCI Rocket Project

The UCI Rocket Project is an undergraduate engineering design team at the University of California, Irvine, dedicated to the design and development of liquid bi-propellant rockets. The team’s main goal is to design a liquid rocket capable of reaching 100 km for the Base11 Space Challenge. The current iteration of the rocket is designed to reach 45,000 ft as a preliminary verification of the team’s design and manufacturing capabilities. The design utilizes a pressure fed system with a propellant combination of liquid methane and liquid oxygen. As the team moves further into the testing and verification stages, development of the rocket capable of reaching 100 km begins. There are a total of four launch windows for the Base11 competition that begin in May 2020 and end in December 2021. The team has a dedicated lab space with the resources necessary for general design and assembly, but manufacturing is typically contracted to professional companies. As the space industry continues to grow, the UCI Rocket Project will continue giving undergraduate students relevant and impactful hands-on experience to prepare them for the challenges of working in the industry.Advisors: Professor Mark WalterProfessor Ken Meas

UCI Rocket Project

The UCI Rocket Project is an undergraduate engineering design team at the University of California, Irvine, dedicated to the design and development of liquid bi-propellant rockets. The team’s main goal is to design a liquid rocket capable of reaching 100 km for the Base11 Space Challenge. The current iteration of the rocket is designed to reach 45,000 ft as a preliminary verification of the team’s design and manufacturing capabilities. The design utilizes a pressure fed system with a propellant combination of liquid methane and liquid oxygen. As the team moves further into the testing and verification stages, development of the rocket capable of reaching 100 km begins. There are a total of four launch windows for the Base11 competition that begin in May 2020 and end in December 2021. The team has a dedicated lab space with the resources necessary for general design and assembly, but manufacturing is typically contracted to professional companies. As the space industry continues to grow, the UCI Rocket Project will continue giving undergraduate students relevant and impactful hands-on experience to prepare them for the challenges of working in the industry.Advisors: Professor Mark WalterProfessor Ken Meas