Transfer of Reinforcement Learning-Based Controllers from Model- to Hardware-in-the-Loop

The process of developing control functions for embedded systems is resource-, time-, and data-intensive, often resulting in sub-optimal cost and solutions approaches. Reinforcement Learning (RL) has great potential for autonomously training agents to perform complex control tasks with minimal human intervention. Due to costly data generation and safety constraints, however, its application is mostly limited to purely simulated domains. To use RL effectively in embedded system function development, the generated agents must be able to handle real-world applications. In this context, this work focuses on accelerating the training process of RL agents by combining Transfer Learning (TL) and X-in-the-Loop (XiL) simulation. For the use case of transient exhaust gas re-circulation control for an internal combustion engine, use of a computationally cheap Model-in-the-Loop (MiL) simulation is made to select a suitable algorithm, fine-tune hyperparameters, and finally train candidate agents for the transfer. These pre-trained RL agents are then fine-tuned in a Hardware-in-the-Loop (HiL) system via TL. The transfer revealed the need for adjusting the reward parameters when advancing to real hardware. Further, the comparison between a purely HiL-trained and a transferred agent showed a reduction of training time by a factor of 5.9. The results emphasize the necessity to train RL agents with real hardware, and demonstrate that the maturity of the transferred policies affects both training time and performance, highlighting the strong synergies between TL and XiL simulation

Genomic DNA isolation methods from honey bee (Apis mellifera L.) Spermatheca

The honey bee queen (Apis mellifera L.) has a polyandrous mating system, meaning that the queen mates on average with 17 drones from the surroundings in a congregation area. After the mating event, the spermatozoa of the drones are stored in an organ called spermatheca. Genetic analysis of the spermathecal content can provide an estimate of the genetic diversity and purity of the surrounding honey bee populations. This can be particularly useful for conservation and mating centers that need to monitor their populations’ genetic backgrounds. However, isolating enough DNA for genomic applications from such a small and complex matrix can be a challenge. Here, we compared the quantity and quality of DNA isolated using five methods: (i) phenol-chloroform-isopropanol, (ii) QIAamp DNA Minikit, (iii) QIAamp DNA Microkit, (iv) Macherey- Nagel Nucleospin Tissue, and (v) NEB Monarch Genomic DNA Purification Tissue. For each kit, when appropriate, variations including different isolation protocols, lysis incubation times, and the addition of RNA carrier were assayed. The quantity and quality of DNA extracted was assessed by spectrophotometric (SpectroStar®Nano LVis Plate) and fluorometric methods (Quantus ™ Fluorometer). Spectrophotometric quantification indicated nucleic acid concentrations ranging from 2.00 to 55.58 ng/μL, and in 91.43% of the cases, the A260/280 ratios were over 2.00, indicating an elevated presence of RNA. The fluorometric quantification, specific for double-stranded DNA, provided values ranging from 0.02 to 2.30 ng/μL. From the five methods, two alternative protocols of the commercial kit QIAamp DNA Microkit produced a sufficient DNA quantity (≥1.7 ng/μL measured by Quantus) for applications involving SNP genotyping, namely: the Tissue protocol with 6 hours of lysis incubation and the Tissue protocol with 3 hours of incubation, both with addition of RNA carrier. In contrast, overnight lysis decreased the DNA yield. The other methods generally produced low and/or inconsistent DNA recovery. According to our results, QIAamp DNA Microkit with the use of RNA carrier and lysis incubation times between 3 to 6 hours produce the required DNA quantities for SNP genotyping.Contributions of JW and EM were financed through the financial support of the German Federal Ministry for Food and Agriculture, through the intermediary of the Federal Office for Agriculture and Food (project no. 2818BM040).info:eu-repo/semantics/publishedVersio

A molecular tool to detect genetic introgression from Spermatheca content

The genetic integrity of Apis mellifera mellifera is threatened by introgression in many places of its native distribution, after recurrent importations of commercial queens typically belonging to the divergent C-lineage. A growing interest in keeping and protecting A. m. mellifera has motivated the development of conservation programs in many places of Europe. As part of the conservation efforts, isolated mating stations are set to avoid unwanted crosses, but these are not always effective as matings with unwanted drones are frequently reported. An interesting method to monitor the degree of isolation of mating stations could be through genetic analysis of the queen spermatheca contents. While this method implies that queens selected for monitoring are sacrificed, it can be a powerful way of assessing the effectiveness of mating stations because it would allow easy detection of unwanted alleles. Here, we developed an SNP-based tool suited to the analysis of DNA extracted from spermatheca or from pooled DNA of varying sources. To that end, we first designed an SNP panel from whole-genome sequence data generated from 228 drones, of which 148 belonged to the M-lineage (117 A. m. iberiensis and 31 A. m. mellifera) and 80 to the C-lineage (46 A. m. carnica and 34 A. m. ligustica). A total of 5,007 highly differentiated SNPs was found. Based on different criteria, 130 SNPs were selected to be included in the genotyping tool. This tool is based on the NEBNext Direct Genotyping Solution that allows high-throughput, sequence-based target genotyping of single-individual or pooled DNA. To assess the tool’s sensitivity and accuracy, 142 samples (DNA extracted from spermatheca and tissue, as well as known DNA mixtures) were genotyped. After removing the problematic SNPs, 81 were retained and these were able to provide an estimate of the pool introgression level with great accuracy. This tool represents a significant advance in the genetic analysis of honey bee colonies with a variety of applications, including breeding and conservation of A. m. mellifera.info:eu-repo/semantics/publishedVersio

Fresh Chicken as Main Risk Factor for Campylobacteriosis, Denmark

Increased consumption of fresh poultry in Denmark has contributed substantially to the increasing incidence of human campylobacteriosis

The WCET Tool Challenge 2011

Following the successful WCET Tool Challenges in 2006 and 2008, the third event in this series was organized in 2011, again with support from the ARTIST DESIGN Network of Excellence. Following the practice established in the previous Challenges, the WCET Tool Challenge 2011 (WCC'11) defined two kinds of problems to be solved by the Challenge participants with their tools, WCET problems, which ask for bounds on the execution time, and flow-analysis problems, which ask for bounds on the number of times certain parts of the code can be executed. The benchmarks to be used in WCC'11 were debie1, PapaBench, and an industrial-strength application from the automotive domain provided by Daimler AG. Two default execution platforms were suggested to the participants, the ARM7 as "simple target'' and the MPC5553/5554 as a "complex target,'' but participants were free to use other platforms as well. Ten tools participated in WCC'11: aiT, Astr\'ee, Bound-T, FORTAS, METAMOC, OTAWA, SWEET, TimeWeaver, TuBound and WCA

Combination and QCD analysis of charm production cross section measurements in deep-inelastic ep scattering at HERA

Measurements of open charm production cross sections in deep-inelastic ep scattering at HERA from the H1 and ZEUS Collaborations are combined. Reduced cross sections sigma(c (c) over bar)(red) for charm production are obtained in the kinematic range of photon virtuality 2.5 LT = Q(2) LT = 2000 GeV2 and Bjorken scaling variable 3 . 10(-5) LT = x LT = 5 . 10(-2). The combination method accounts for the correlations of the systematic uncertainties among the different data sets. The combined charm data together with the combined inclusive deep-inelastic scattering cross sections from HERA are used as input for a detailed NLO QCD analysis to study the influence of different heavy flavour schemes on the parton distribution functions. The optimal values of the charm mass as a parameter in these different schemes are obtained. The implications on the NLO predictions for W-+/- and Z production cross sections at the LHC are investigated. Using the fixed flavour number scheme, the running mass of the charm quark is determined

Verwendung von drohnenbrütigen Arbeiterinnen zur Zucht auf individuell exprimierte Merkmale der Honigbiene

Einige Zuchtmerkmale der Honigbiene lassen sich sinnvoll an einzelnen Arbeiterinnen messen. Dabei kann der Zuchtfortschritt potentiell beschleunigt werden, wenn Söhne der selektierten Arbeiterinnen zur Besamung herangezogen werden. Ziel dieser Arbeit war deshalb die Schaffung von Methoden zur Erzeugung von Söhnen selektierter Arbeitsbienen. Um die Fruchtbarkeit der selektierten Tiere zu fördern wurde zunächst vergeblich versucht, eine Hormonbehandlung zur gezielten Induktion der Eibildung zu entwickeln. Dabei konnten die Mechanismen der Fortpflanzungssteuerung bei Arbeitsbienen teilweise aufgeklärt werden. Um die Fruchtbarkeit in allen Bienen außer den selektierten zu unterdrücken wurde zunächst die Sterilisation durch harte Röntgenstrahlung erprobt. Die erhaltenen Tiere legten keine Eier, zeigten aber starke somatische Schäden. Die Aufzucht von Arbeiterinnen-Larven in überalterten Völkern ergab keine sterilen, sondern besonders fruchtbare Tiere. Schließlich gelang es unter Verwendung von Flugbienen als Begleittieren, in Jungbienen in über 60% der Fälle volle Ovarentwicklung zu erzielen. Auch konnten alle erzeugten Eier Jungbienen zugeordnet werden. Um die Eignung von Arbeiterinnen-gelegten Eiern für Zuchtzwecke zu prüfen, wurden Qualitätsparameter gemessen. Arbeiterinnen-gelegte Eier zeigten sich empfindlicher gegenüber Trockenheit und mechanischer Beschädigung als Königinnen-gelegte Eier. Alle in Völker verbrachten Arbeiterinnen-gelegten Eier wurden ausgefressen. Also wurde eine Methode zur Anfütterung von Larven im Brutschrank entwickelt. Die so erzeugten Larven überlebten aber ebenfalls nicht in Völkern. Deshalb wurde ein Verfahren entwickelt, mit dem aus über 50% der eingesetzten Larven durch Brutschrankaufzucht adulte Tiere erhalten werden können. Ein vorläufiges Protokoll zur Erzeugung von Söhnen individuell selektierter Arbeitsbienen wurde erarbeitet.Breeding of honey bees usually involves selection at the colony level. However, some important breeding traits exist that can also be measured at the scale of the individual worker bee. In these cases, genetic progress can potentially be accelerated by using drones which are sons of the selected individuals. The aim of this study was therefore to create methods for the production of offspring from individually selected workers. To enhance fertility of selected workers, an attempt was made to develop a hormonal treatment to induce worker fertility. It led to the partial elucidation of the endocrine regulation of reproduction in worker honey bees. Methods were tested to suppress fertility in all bees except the selected workers, in order to prevent competition among workers and enable identification of the eggs of selected bees. Method tested included the application of ultra-hard X-rays and rearing of larvae inside over-aged colonies. Finally, it could be shown that forager bees can be used as companion bees to selected workers. This method led to fully developed ovaries in more than 60% of bees representing the selected workers in these tests, who also were the only animals to produce eggs. The next subtask was the measurement of characteristics linked to viability in eggs from laying workers. Worker-laid eggs showed to be more sensitive to dryness and mechanical damage. When worker-laid eggs were inserted into queenless or queenright colonies for rearing, they were systematically cannibalised. Therefore, a method was developed to rear eggs into living larvae in an incubator. However, the larvae produced in this way were not accepted by colonies either. To circumvent this problem, a protocol was designed and validated that allows to rear worker-derived drone larvae into adults in vitro. The study led to a preliminary protocol for the production of drones that are sons of individually selected worker bees

Differences between queen- and worker-laid male eggs of the honey bee (

In this study, we investigated whether differences in the reproductive biology of honey bee (Apis mellifera) queens and laying workers are reflected in their eggs. We first tested the capacity of queen- and worker-laid male eggs to withstand dry conditions, by incubating samples at 30.0, 74.9, and 98.7% relative humidity. We found that worker-laid eggs were more sensitive to desiccation. Secondly, we measured the weight and quantities of vitellin, total protein, lipid, glycogen, and free carbohydrate in queen- and worker-laid eggs. Although worker-laid eggs were found to be heavier than queen-laid eggs in two of the four replicates, no systematic differences were found regarding nutrient content. Finally, we compared the duration of embryo development in the two egg types. Worker-laid eggs developed more slowly than queen-laid eggs in two out of three replicates, suggesting that they may only be partly mature at the moment they are laid. Possible causes and consequences of the observed differences are discussed

Determinanten beruflicher Karrieren unter den Bedingungen flexibilisierter Arbeitsmärkte: Eine Untersuchung des Berufseinstiegs von Hochschulabsolventen und -absolventinnen

Absolventen von Hochschulen finden heutzutage beim Berufseinstieg einen flexibilisierten Arbeitsmarkt vor. Nur einer Minderheit gelingt die Platzierung in unbefristeten Beschäftigungsverhältnissen. Es bestehen Mobilitätsbarrieren zwischen befristeten und unbefristeten Beschäftigungsverhältnissen. Absolventen in flexibilisierten Beschäftigungsverhältnissen sind zudem hinsichtlich der Qualitäten der Jobs im Vergleich zu unbefristet Beschäftigten benachteiligt. Die Flexibilisierung des Arbeitsmarkts bewirkt eine Verschiebung der Determinanten beruflichen Erfolgs. Bestimmte berufliche Wertorientierungen und soziales Kapital ("Kontakte") sind wichtig für beruflichen Erfolg. Schlüsselqualifikationen zeigen einen nur geringen Einfluss.Auch unter Akademikern sind Frauen gegenüber Männern beruflich benachteiligt. Der Bericht bereichert die Forschung zum Gender Wage Gap um den Einfluss geschlechtsspezifischer Orientierungen.Vorliegendes Papier ist die Dokumentation eines von der Hans-Böckler Stiftung geförderten Online-Panels. Panelteilnehmer sind Absolventen und Studierende kurz vor ihrem Abschluss. Die Studie verfolgte den Berufseinstieg der Absolventen über zwei Jahre und erfasste deren aktuelle Situation (Suche/Erwerbstätigkeit etc.)