Batch Reinforcement Learning on the Industrial Benchmark: First Experiences

The Particle Swarm Optimization Policy (PSO-P) has been recently introduced and proven to produce remarkable results on interacting with academic reinforcement learning benchmarks in an off-policy, batch-based setting. To further investigate the properties and feasibility on real-world applications, this paper investigates PSO-P on the so-called Industrial Benchmark (IB), a novel reinforcement learning (RL) benchmark that aims at being realistic by including a variety of aspects found in industrial applications, like continuous state and action spaces, a high dimensional, partially observable state space, delayed effects, and complex stochasticity. The experimental results of PSO-P on IB are compared to results of closed-form control policies derived from the model-based Recurrent Control Neural Network (RCNN) and the model-free Neural Fitted Q-Iteration (NFQ). Experiments show that PSO-P is not only of interest for academic benchmarks, but also for real-world industrial applications, since it also yielded the best performing policy in our IB setting. Compared to other well established RL techniques, PSO-P produced outstanding results in performance and robustness, requiring only a relatively low amount of effort in finding adequate parameters or making complex design decisions

A Benchmark Environment Motivated by Industrial Control Problems

In the research area of reinforcement learning (RL), frequently novel and promising methods are developed and introduced to the RL community. However, although many researchers are keen to apply their methods on real-world problems, implementing such methods in real industry environments often is a frustrating and tedious process. Generally, academic research groups have only limited access to real industrial data and applications. For this reason, new methods are usually developed, evaluated and compared by using artificial software benchmarks. On one hand, these benchmarks are designed to provide interpretable RL training scenarios and detailed insight into the learning process of the method on hand. On the other hand, they usually do not share much similarity with industrial real-world applications. For this reason we used our industry experience to design a benchmark which bridges the gap between freely available, documented, and motivated artificial benchmarks and properties of real industrial problems. The resulting industrial benchmark (IB) has been made publicly available to the RL community by publishing its Java and Python code, including an OpenAI Gym wrapper, on Github. In this paper we motivate and describe in detail the IB's dynamics and identify prototypic experimental settings that capture common situations in real-world industry control problems

A Benchmark Environment Motivated by Industrial Control Problems

In the research area of reinforcement learning (RL), frequently novel and promising methods are developed and introduced to the RL community. However, although many researchers are keen to apply their methods on real-world problems, implementing such methods in real industry environments often is a frustrating and tedious process. Generally, academic research groups have only limited access to real industrial data and applications. For this reason, new methods are usually developed, evaluated and compared by using artificial software benchmarks. On one hand, these benchmarks are designed to provide interpretable RL training scenarios and detailed insight into the learning process of the method on hand. On the other hand, they usually do not share much similarity with industrial real-world applications. For this reason we used our industry experience to design a benchmark which bridges the gap between freely available, documented, and motivated artificial benchmarks and properties of real industrial problems. The resulting industrial benchmark (IB) has been made publicly available to the RL community by publishing its Java and Python code, including an OpenAI Gym wrapper, on Github. In this paper we motivate and describe in detail the IB's dynamics and identify prototypic experimental settings that capture common situations in real-world industry control problems

Wachstum und Charakterisierung von Seltenerdoxiden und Magnesiumoxid auf Galliumarsenid-Substraten

Die Erzeugung spinpolarisierter Ladungsträger in einem Halbleiter gilt als Grundvoraussetzung zur Realisierung spintronischer Bauelemente. Einen möglichen Ansatz zu deren Realisierung stellen Ferromagnet/Halbleiter(FM/HL)-Hybridstrukturen dar, deren Herstellung jedoch mit einigen Schwierigkeiten verbunden ist. Durch die Vermischung des ferromagnetischen Materials mit dem Halbleiter werden die elektronischen Eigenschaften der Hybridstruktur verändert und die Spininjektionseffizienz stark verringert. Durch das gezielte Einfügen einer dünnen Oxidschicht in den FM/HL-Grenzübergang kann die Diffusion unterdrückt, die Kristallqualität verbessert und die Effizienz der Struktur erhöht werden. Diese Arbeit beschäftigt sich mit dem Wachstum und der Charakterisierung dünner Oxidschichten, hergestellt mittels Molekularstrahlepitaxie. Zwei Seltenerdoxide, La2O3 und Lu2O3, werden auf GaAs-Substraten gewachsen und die Kristallqualität der Schichten miteinander verglichen. Mit der Heusler-Legierung Co2FeSi als Injektorschicht wird eine FM/Oxid/HL-Hybridstruktur auf Basis einer La2O3/GaAs(111)B-Struktur realisiert und magnetisch und elektrisch charakterisiert. Ein häufig verwendetes Barrierenmaterial in FM/HL-Hybridstrukturen ist Magnesiumoxid (MgO). In dieser Arbeit werden dünne MgO-Schichten auf GaAs(001) an der PHARAO-Wachstumsanlage am BESSY II erzeugt. Dies geschieht durch getrenntes Verdampfen von metallischem Mg bzw. Einleiten von molekularem Sauerstoff in die Wachstumskammer. Um die Oxidation des Halbleitersubstrats zu verhindern, wird vor dem MgO-Wachstum eine dünne Mg-Schicht abgeschieden. Abhängig von der Dicke dieser Schicht sind zwei in-plane-Orientierungen des MgO relativ zum GaAs kontrolliert einstellbar. Darüber hinaus werden Hybridstrukturen mit Eisen Fe als Injektorschicht und schrittweise erhöhter MgO-Schichtdicke gewachsen. Die Eindiffusion von Fe in das GaAs-Substrat nimmt mit zunehmender MgO-Schichtdicke um mehrere Größenordnungen ab.The generation of spin-polarized charge carriers in a semiconductor is a basic building block for the implemention of spintronic devices. A feasible approach to their implementation are ferromagnet/semiconductur(FM/SC) hybrid structures, whose fabrication is associated with some issues. The intermixing of the ferromagnetic material with the semiconductor leads to distortion of the electrical properties of the hybrid structure and the spin injection efficiency is reduced. By intentionally inserting a thin oxide layer into the FM/SC interface diffusion can be suppressed while the crystal quality and the spin injection efficiency of the structure are both increased. In this thesis the growth and characterization of thin oxide films fabricated by molecular beam epitaxy are discussed. Two rare earth oxides, La2O3 and Lu2O3, are grown on GaAs substrates and their crystal qualities are compared. Based on La2O3/GaAs(111)B full FM/SC hybrid structures are grown with the Heusler alloy Co2FeSi as injection layer and characterized by magnetic and electrical means. Another material used as a barrier in FM/SC hybrid structures is magnesium oxide (MgO). Here, thin MgO layers are grown on GaAs(001) at the PHARAO system at BESSY II. The growth is conducted by the separated evaporation of metallic Mg and introducing molecular oxygen into the growth chamber. To avoid oxidation of the semiconducting substrate a thin Mg layer is deposited prior to the MgO growth. Depending on the Mg layer thickness two different MgO in-plane orientations can be achieved with respect to the GaAs substrate. Furthermore, FM/SC hybrid structures with iron Fe as injection layer are grown while the MgO layer thickness is increased gradually. The indiffusion of Fe into the GaAs substrate is suppressed by several orders of magnitude with increasing MgO layer thickness

Achieving near-zero particle generation by simplicity of design—A compliant-mechanism-based gripper for clean-room environments

Lab Automation facilitates high-throughput processes and improves reproducibility and efficiency while removing human action, primary source of contaminating particles. Handling poses a risk of contamination due to close contact with the objects. We propose a novel gripper (CrocoGrip) relying on compliant mechanisms to reduce the amount of contaminating particles generated by the gripper rather than preventing their emission, the latter being the common approach in current grippers. Using a structured design approach including simplified motion models and Finite Element Methods, we developed a novel gripper that is actuated by linear solenoids and purely relies on deformation for its motion. As a result, abrasive behavior and, therefore, the generation of particles is reduced without the need for additional sealing. We experimentally proved that the number of particles emitted by the CrocoGrip fulfills the demands of ISO14644 class 5. Due to the monolithic design of the CrocoGrip and, as a result, the need for few components, we achieve a simplicity of design, making cleaning, sterilization and maintenance easy, even for nonexperts. Furthermore, all parts but the two solenoids can be sterilized through autoclaving. The gripping is performed by utilizing the deformation energy of the compliant mechanism, making the gripping energy-efficient and safe. By using interchangeable jaws, the CrocoGrip was able to handle a microplate in SBS-standard, a 50 mL Falcon tube, and a Ø60 mm Petri dish using a robotic arm. CrocoGrip exploits the advantages of compliant mechanisms, especially for applications requiring clean-room environments. This approach of CM-based grippers enables an increase in the cleanliness of handling processes without an increase in system complexity of the gripper to facilitate the lab automation of highly sensitive processes, such as in tissue engineering

A New Method for Reliable Rotary Tool Shank Use at Speeds above the Critical Speed Range

AbstractDynamic tool characteristics affect both the design and the use of rotary tools applied to high-speed cutting. At present, spindle-driven tools commonly operate at speed values below their critical bending speed. Currently, this is the only way to guarantee reliable tool operation. Consequently, the low natural frequency – in particular for tools with a high length-to-diameter ratio – often results in spindle speeds that utilise neither the cutting materials’ nor the machine tools’ potential. The paper elucidates a method to design tool shanks and tools to be run above the critical speed, as well as its implementation. The reliable and safe operation of these special tool shanks is based on the physical effects of self-centering and self-balancing, which come into play above the critical speed

FDA and EMA Approvals of New Breast Cancer Drugs—A Comparative Regulatory Analysis

Breast cancer is the most common cancer in women worldwide and the solid tumor type for which the highest number of drugs have been approved to date. This study examines new drug approvals for breast cancer by the United States Food and Drug Administration (FDA) and the European Medicines Agency (EMA), based on an analysis of regulatory documents from both agencies for the period from 1995 to 2018. Of the 29 breast cancer drugs approved over this time span, 17 received positive decisions from both the FDA and EMA, including all drugs licensed after 2008. Nineteen of the 25 FDA-approved drugs, but none of the EMA approvals, benefited from special regulatory pathways (such as fast track, breakthrough therapy, or priority review). In the U.S.A., four accelerated approvals were granted (of which one, for bevacizumab, was later revoked), while only two drugs received provisional approvals following EMA review. New breast cancer drugs were approved approximately twelve months earlier in the United States than in Europe. These results suggest that a broader use of special regulatory pathways by EMA could help to accelerate access to novel drugs for European breast cancer patients

Trajectory sampling and finite-size effects in first-principles stopping power calculations

Real-time time-dependent density functional theory (TDDFT) is presently the most accurate available method for computing electronic stopping powers from first principles. However, obtaining application-relevant results often involves either costly averages over multiple calculations or ad hoc selection of a representative ion trajectory. We consider a broadly applicable, quantitative metric for evaluating and optimizing trajectories in this context. This methodology enables rigorous analysis of the failure modes of various common trajectory choices in crystalline materials. Although randomly selecting trajectories is common practice in stopping power calculations in solids, we show that nearly 30% of random trajectories in an FCC aluminium crystal will not representatively sample the material over the time and length scales feasibly simulated with TDDFT, and unrepresentative choices incur errors of up to 60%. We also show that finite-size effects depend on ion trajectory via "ouroboros" effects beyond the prevailing plasmon-based interpretation, and we propose a cost-reducing scheme to obtain converged results even when expensive core-electron contributions preclude large supercells. This work helps to mitigate poorly controlled approximations in first-principles stopping power calculations, allowing 1-2 order of magnitude cost reductions for obtaining representatively averaged and converged results

Die Spontanheilungsversuche des vorderen Kreuzbands : eine experimentelle Untersuchung

Fragestellung Spontanheilungen von Rupturen des vorderen Kreuzbandes (VKB) wurden wiederholt mittels MRT beschrieben. In grundlegenden tierexperimentellen Untersuchungen wird geschlussfolgert, dass es keine Regeneration gäbe. Die Morphologie der Spontanheilung des VKB wurde im Tiermodell des Kaninchens untersucht. Methoden 32 männliche Kaninchen wurden medial arthrotomiert und das VKB durchtrennt. Die Durchtrennung erfolgte unter Sicht mittels Nr. 15 Skalpell, zusätzlich wurde die vollständige Trennung der Stümpfe durch mehrfache Instrumentenpassage kontrolliert. Eine Resektion von Fasern erfolgte nicht. Je 8 Tiere wurden 2, 4, 8 und 12 Wochen nach dem Eingriff getötet. Die Morphologie der Heilungsverläufe wurde nach Lo et al. [1] klassifiziert, neue Typen der Klassifikation hinzugefügt. Typ B (intrasynoviale Ruptur) und Typ C (knöcherner Ausriss) waren studienbedingt nicht möglich. Die Studie war vom Regierungspräsidium genehmigt. Ergebnisse Die untersuchten Tiere hatten einen unauffälligen postoperativen Verlauf. Bei unterschiedlicher Ausprägung hatten alle Tiere makroskopisch eine Knorpelschädigung. Nach chirurgischer Durchtrennung waren die VKBenden durchschnitten (mop ended Typ A). Dieser Typ wurde bei der Dissektion nicht mehr vorgefunden. Die VKBstümpfe waren bei Dissektion: retrahiert (Typ D), mit dem hinteren Kreuzband verwachsen (E), resorbiert (F), miteinander verbunden = narbengeheilt (G), oder es lag eine Kombination (H) verschiedener Typen vor. Als neue Typen wurden Verwachsungen mit dem Meniskus (I) oder mit der Gelenkkapsel bzw. dem Fettkörper (K) beschrieben. Die VKBstümpfe waren gering (G 1 = 7), deutlich (G2 = 3 bzw. 4 insges. bei zusätzlicher Verwachsung mit dem Innenmeniskus) oder hypertroph (G 3 = 3) miteinander verwachsen. [Tab. 1] Schlussfolgerungen Der VKBriss führt häufig zur Kniegelenksinstabilität und Osteoarthrose. Nach Durchtrennung des VKB wird in Studien der Osteoarthroseforschung im Kaninchenmodell der Befund nie detailliert. In einem systematischen Vergleich der Regenerationsfähigkeit von partiell und komplett durchtrennten VKBs fanden Hefti et al. [2] weder bei skeletall unreifen noch bei jungen erwachsenen Kaninchen eine Regeneration nach kompletter Durchtrennung. Sie berichteten als Regelfall die Resorption nach 3 Monaten, in 2 Fällen Verwachsungen mit dem hinteren Kreuzband. Die hier berichteten Daten sind im Gegensatz zu Hefti's Untersuchungen. Sie stimmen aber gut mit humanen MRT-Untersuchungen und Arthroskopien nach Kreuzbandverletzungen überein. Das VKB versucht durch Verwachsung eine Heilung der VKBstümpfe zu erzielen oder Anschluss an andere Kniegelenksstrukturen zu gewinnen. Somit besteht auch im Tiermodell des Kaninchens eine gewisse Spontanheilungsaktivität des VKB, deren biomechanische Stabilität allerdings oft unzureichend erschien. Eine Kniegelenksarthrose entwickelte sich in den untersuchten Gelenken