Continuous Value Iteration (CVI) Reinforcement Learning and Imaginary Experience Replay (IER) for learning multi-goal, continuous action and state space controllers

This paper presents a novel model-free Reinforcement Learning algorithm for learning behavior in continuous action, state, and goal spaces. The algorithm approximates optimal value functions using non-parametric estimators. It is able to efficiently learn to reach multiple arbitrary goals in deterministic and nondeterministic environments. To improve generalization in the goal space, we propose a novel sample augmentation technique. Using these methods, robots learn faster and overall better controllers. We benchmark the proposed algorithms using simulation and a real-world voltage controlled robot that learns to maneuver in a non-observable Cartesian task space.Comment: Published in 2019 International Conference on Robotics and Automation (ICRA) 20-24 May 201

Conformational Preferences of 3-(Dimethylazinoyl)propanoic Acid as a Function of pH and Solvent; Intermolecular versus Intramolecular Hydrogen Bonding

The conformational equilibrium of 3-(dimethylazinoyl)propanoic acid (DMAPA, azinoyl = N^+(O^−) has a weak pH-dependence in D_2O, with a slight preference for trans in alkaline solutions. The acid ionization constants of the protonated amine oxide and carboxylic functional groups as determined by NMR spectroscopy were 7.9 × 10^(−4) and 6.3 × 10^(−6), respectively. The corresponding value of K_1/K_2 of 1.3 × 10^2 is not deemed large enough to provide experimental NMR evidence for a significant degree of intramolecular hydrogen bonding in D_2O. Conformational preferences of DMAPA are mostly close to statistical (gauche/trans = 2/1) in other protic solvents, e.g., alcohols. However, the un-ionized form of DMAPA appears to be strongly intramolecularly hydrogen-bonded and gauche in aprotic solvents

Retrospective Study on Ganglionic and Nerve Block Series as Therapeutic Option for Chronic Pain Patients with Refractory Neuropathic Pain

Objective. Current recommendations controversially discuss local infiltration techniques as specific treatment for refractory pain syndromes. Evidence of effectiveness remains inconclusive and local infiltration series are discussed as a therapeutic option in patients not responding to standard therapy. The aim of this study was to investigate the effectiveness of infiltration series with techniques such as sphenopalatine ganglion (SPG) block and ganglionic local opioid analgesia (GLOA) for the treatment of neuropathic pain in the head and neck area in a selected patient group. Methods. In a retrospective clinical study, 4960 cases presenting to our university hospital outpatient pain clinic between 2009 and 2016 were screened. Altogether, 83 patients with neuropathic pain syndromes receiving local infiltration series were included. Numeric rating scale (NRS) scores before, during, and after infiltration series, comorbidity, and psychological assessment were evaluated. Results. Maximum NRS before infiltration series was median 9 (IQR 8–10). During infiltration series, maximum NRS was reduced by mean 3.2 points (SD 3.3, p < 0.001) equaling a pain reduction of 41.0% (SD 40.4%). With infiltration series, mean pain reduction of at least 30% or 50% NRS was achieved in 54.2% or 44.6% of cases, respectively. In six percent of patients, increased pain intensity was noted. Initial improvement after the first infiltration was strongly associated with overall improvement throughout the series. Conclusion. This study suggests a beneficial effect of local infiltration series as a treatment option for refractory neuropathic pain syndromes in the context of a multimodal approach. This effect is both significant and clinically relevant and therefore highlights the need for further randomized controlled trials

Incidence of Anaplastic Large Cell Lymphoma and Breast-Implant-Associated Lymphoma—An Analysis of a Certified Tumor Registry over 17 Years

Background: Breast-implant-associated anaplastic large cell lymphoma (BI-ALCL) and primary breast ALCL are rare extranodal manifestations of non-Hodgkin lymphoma. The rarity of both diseases, along with unreleased sales data on breast implants and constant updates of classification systems impede the calculation of an exact incidence. Methods: The database of the Tumor Center Regensburg in Bavaria was searched for patients with CD30-positive and ALK-negative anaplastic large cell lymphoma between 2002 and 2018. These lymphomas were identified by the ICD-O-3 morphology code "97023" and were cross-checked by searching the diagnosis by name the and ICD-10 code C84.7. Furthermore, we tried to calculate the incidence rates and corresponding 95% confidence intervals, standardized to 1,000,000 implant years of breast-implant-associated anaplastic large cell lymphoma and primary breast anaplastic large cell lymphoma. Results: Twelve ALK-negative and CD30-positive anaplastic large cell lymphomas were identified out of 170,405 malignancies. No case was found within the breast tissue and none of the patients had a previous history of breast implant placement. In five cases, lymph node involvement in close proximity to the breast was observed. Conclusion: We found a low incidence of anaplastic large cell lymphoma and no association to breast implants in these patients. A review of the current literature revealed inconsistent use of classification systems for anaplastic large cell lymphomas and potential overestimation of cases

Effects of dietary curcumin or N-acetylcysteine on NF-κB activity and contractile performance in ambulatory and unloaded murine soleus

BACKGROUND: Unloading of skeletal muscle causes atrophy and loss of contractile function. In part, this response is believed to be mediated by the transcription factor nuclear factor-kappa B (NF-κB). Both curcumin, a component of the spice turmeric, and N-acetylcysteine (NAC), an antioxidant, inhibit activation of NF-κB by inflammatory stimuli, albeit by different mechanisms. In the present study, we tested the hypothesis that dietary curcumin or NAC supplementation would inhibit unloading-induced NF-κB activity in skeletal muscle and thereby protect muscles against loss of mass and function caused by prolonged unloading. METHODS: We used hindlimb suspension to unload the hindlimb muscles of adult mice. Animals had free access to drinking water or drinking water supplemented with 1% NAC and to standard laboratory diet or diet supplemented with 1% curcumin. For 11 days, half the animals in each dietary group were suspended by the tail (unloaded) and half were allowed to ambulate freely. RESULTS: Unloading caused a 51–53% loss of soleus muscle weight and cross-sectional area relative to freely-ambulating controls. Unloading also decreased total force and force per cross-sectional area developed by soleus. Curcumin supplementation decreased NF-κB activity measured in peripheral tissues of ambulatory mice by gel shift analysis. In unloaded animals, curcumin supplementation did not inhibit NF-κB activity or blunt the loss of muscle mass in soleus. In contrast, NAC prevented the increase in NF-κB activity induced by unloading but did not prevent losses of muscle mass or function. CONCLUSION: In conclusion, neither dietary curcumin nor dietary NAC prevents unloading-induced skeletal muscle dysfunction and atrophy, although dietary NAC does prevent unloading induced NF-κB activation

Effects of dietary curcumin or N-acetylcysteine on NF-kappaB activity and contractile performance in ambulatory and unloaded murine soleus

BACKGROUND: Unloading of skeletal muscle causes atrophy and loss of contractile function. In part, this response is believed to be mediated by the transcription factor nuclear factor-kappa B (NF-kappaB). Both curcumin, a component of the spice turmeric, and N-acetylcysteine (NAC), an antioxidant, inhibit activation of NF-kappaB by inflammatory stimuli, albeit by different mechanisms. In the present study, we tested the hypothesis that dietary curcumin or NAC supplementation would inhibit unloading-induced NF-kappaB activity in skeletal muscle and thereby protect muscles against loss of mass and function caused by prolonged unloading. METHODS: We used hindlimb suspension to unload the hindlimb muscles of adult mice. Animals had free access to drinking water or drinking water supplemented with 1% NAC and to standard laboratory diet or diet supplemented with 1% curcumin. For 11 days, half the animals in each dietary group were suspended by the tail (unloaded) and half were allowed to ambulate freely. RESULTS: Unloading caused a 51-53% loss of soleus muscle weight and cross-sectional area relative to freely-ambulating controls. Unloading also decreased total force and force per cross-sectional area developed by soleus. Curcumin supplementation decreased NF-kappaB activity measured in peripheral tissues of ambulatory mice by gel shift analysis. In unloaded animals, curcumin supplementation did not inhibit NF-kappaB activity or blunt the loss of muscle mass in soleus. In contrast, NAC prevented the increase in NF-kappaB activity induced by unloading but did not prevent losses of muscle mass or function. CONCLUSION: In conclusion, neither dietary curcumin nor dietary NAC prevents unloading-induced skeletal muscle dysfunction and atrophy, although dietary NAC does prevent unloading induced NF-kappaB activation

Challenges of temperature measurement during the friction stir welding process

The exact determination of the process zone temperature can be considered as an increasingly important role in the control and monitoring of the friction stir welding process (FSW). At present, temperature measurement is carried out with the aid of a temperature sensor integrated into the tool (usually thermocouples). Since these cannot be attached directly to the joining area, heat dissipation within the tool and to the environment cause measurement deviations as well as a time delay in the temperature measurement. The article describes a process and the challenges that arise in this process, how a direct temperature measurement during the process can be achieved by exploiting the thermoelectric effect between tool and workpiece, without changing the tool by introducing additional temperature sensors

Multi-Mode Love-Wave SAW Magnetic-Field Sensors

A surface-acoustic-wave (SAW) magnetic-field sensor utilizing fundamental, first- and second-order Love-wave modes is investigated. A 4.5   μ m SiO2 guiding layer on an ST-cut quartz substrate is coated with a 200 n m (Fe90Co10)78Si12B10 magnetostrictive layer in a delay-line configuration. Love-waves are excited and detected by two interdigital transducers (IDT). The delta-E effect in the magnetostrictive layer causes a phase change with applied magnetic field. A sensitivity of 1250 ° / m T is measured for the fundamental Love mode at 263 M Hz . For the first-order Love mode a value of 45 ° / m T is obtained at 352 M Hz . This result is compared to finite-element-method (FEM) simulations using one-dimensional (1D) and two-and-a-half-dimensional (2.5 D) models. The FEM simulations confirm the large drop in sensitivity as the first-order mode is close to cut-off. For multi-mode operation, we identify as a suitable geometry a guiding layer to wavelength ratio of h GL / λ ≈ 1.5 for an IDT pitch of p = 12   μ m . For this layer configuration, the first three modes are sufficiently far away from cut-off and show good sensitivity