Learning on a Budget via Teacher Imitation

Deep Reinforcement Learning (RL) techniques can benefit greatly from leveraging prior experience, which can be either self-generated or acquired from other entities. Action advising is a framework that provides a flexible way to transfer such knowledge in the form of actions between teacher-student peers. However, due to the realistic concerns, the number of these interactions is limited with a budget; therefore, it is crucial to perform these in the most appropriate moments. There have been several promising studies recently that address this problem setting especially from the student's perspective. Despite their success, they have some shortcomings when it comes to the practical applicability and integrity as an overall solution to the learning from advice challenge. In this paper, we extend the idea of advice reusing via teacher imitation to construct a unified approach that addresses both advice collection and advice utilisation problems. We also propose a method to automatically tune the relevant hyperparameters of these components on-the-fly to make it able to adapt to any task with minimal human intervention. The experiments we performed in 5 different Atari games verify that our algorithm either surpasses or performs on-par with its top competitors while being far simpler to be employed. Furthermore, its individual components are also found to be providing significant advantages alone

Dynamical Trajectory Replanning for Uncertain Environments

We propose a dynamical reference generator equipped with an augmented transient “replanning” subsystem that modulates a feedback controller’s efforts to force a mechanical plant to track the reference signal. The replanner alters the reference generator’s output in the face of unanticipated disturbances that drive up the tracking error. We demonstrate that the new reference generator cannot destabilize the tracker, that tracking errors converge in the absence of disturbance, and that the overall coupled reference-tracker system cannot be destabilized by disturbances of bounded energy. We report the results of simulation studies exploring the performance of this new design applied to a two dimensional point mass particle interacting with fixed but unknown terrain obstacles. For more information: Kod*La

Monitoring and disinfection of biofilm-associated sulfate reducing bacteria on different substrata in a simulated recirculating cooling tower system

Microbial biofilm and corrosion in cooling systems are the most common problems that damage expensive equipment, cause loss of production, and increase maintenance costs. Sulfate reducing bacteria were considered the major bacterial group involved in microbiologically influenced corrosion (MIC). We investigated the survival and enumeration of biofilm-associated SRB on coupons of galvanized steel, stainless steel, and copper, which are materials used in the manufacturing of cooling systems. We also investigated the effect of monochloromine on SRB as in mixed species mature biofilms formed on coupons by simulating recirculating cooling water conditions, due to the better penetration feature in biofilms than the residual chlorine. It was concluded that SRB count increased with time in bulk water and the surfaces (P < 0.01). Experimental results supported by statistical analyses show that monochloromine is poorly effective on SRB colonies formed on galvanized and stainless steel surfaces

A drift-diffusion model for robotic obstacle avoidance

We develop a stochastic framework for modeling and analysis of robot navigation in the presence of obstacles. We show that, with appropriate assumptions, the probability of a robot avoiding a given obstacle can be reduced to a function of a single dimensionless parameter which captures all relevant quantities of the problem. This parameter is analogous to the Peclet number considered in the literature on mass transport in advection-diffusion fluid flows. Using the framework we also compute statistics of the time required to escape an obstacle in an informative case. The results of the computation show that adding noise to the navigation strategy can improve performance. Finally, we present experimental results that illustrate these performance improvements on a robotic platform. For more information: Kod*La

The influence of the rapamycin-derivate SDZ RAD on the healing of airway anastomoses

Objective: Among the many immunosuppressive effects of SDZ RAD (40-0(2-hydroxyethyl)-rapamycin), a rapamycin derivative, is the inhibition of fibroblast proliferation. Since the long-term success of lung transplantation is limited by the development of bronchiolitis obliterans, a fibroblast-associated progressive luminal obstruction of the terminal bronchioli, the use of SDZ RAD as immunosuppressive in pulmonary graft recipients may counteract this process. However, reduction of fibroblast activity, posttransplant, may impair the healing of the bronchial anastomoses. Materials and methods: The cervical trachea in pigs was denuded, divided and re-anastomosed with Prolene 4-0 single stitches. Control animals (group 1, n=4) were without, and study animals (group 2, n=6) were with SDZ RAD therapy (1.25 mg/kg/day, p.o., 14 days). After 14 days, the pigs were sacrificed. The anastomoses were examined histologically, and breaking strength of tracheal strips of 5-mm width was measured. Results: All animals survived without complications. Serum levels of SDZ RAD were 30.9±8.7 ng/ml (recommended level 20-40 ng/ml). All anastomoses healed macroscopically without difference between the two groups. Breaking strength was significantly lower in the treated animals (group 1 vs. group 2: 11.75±0.35 vs. 7.69±1.39 N, P=0.01). Histology did not show a significant change in histoarchitecture between the groups. Conclusions: Although SDZ RAD significantly reduced the breaking strength of the tracheal anastomosis, no obvious histological differences between treated and untreated animals could be detected. Since this model does not reflect the clinical situation, further investigations are necessary to reveal the effect of SDZ RAD on airway wound healing in concert with a contemporary clinically used multidrug immunosuppressive regimen in allograft recipient

Evaluation of spinal-paraspinal parameters to determine segmentation of the vertebrae

Purpose: We aimed to evaluate whether lumbar vertebrae can be correctly numbered using auxiliary parameters. Material and methods: Vertebra corpus shape, O’Driscoll classification, lumbosacral axis angle, last two square vertebra dimensions, orifice of right renal artery (RRA), orifice of celiac truncus (CT), orifice of superior mesenteric artery (SMA), vena cava inferior confluence (CVC), abdominal aorta bifurcation (AB), and iliolumbar ligament were evaluated in this study. Results: Lumbosacral transitional vertebrae (LSTV) were observed in 13 (9%) patients. The most common locations of the paraspinal parameters were: RRA: L1 vertebrae (45%), SMA: L1 vertebrae (66%), CT: T12 vertebrae (46%), AB: L4 vertebrae (63%), and CVC: L4 vertebrae (52%). Conclusions: According to the results of our study, no single parameter in the magnetic resonance imaging can accurately indicate the number of vertebrae without counting the levels. As a result, we believe that these parameters may be suspicious in terms of the presence of LSTV rather than the correct level

Reluctance network model for the in-wheel motor of a series-hybrid truck using Tooth Contour Method

Recently, series-hybrid drivetrains are given more emphasis over parallel-hybrid drivetrains due to their simplicity, freedom in implementation and higher efficiency. However, the modeling phase of such machines takes either long calculation time with numerical methods such as Finite Element Analysis (FEA) or produces low accurate results with Magnetic Equivalent Circuit (MEC) models. In this paper a method is used to build a reluctance network where high accuracy is obtained yet still with acceptable calculation time

3D printed Artificial Cornea for Corneal Stromal Transplantation

The aim of this study is to understand the optical, biocompatible, and mechanical properties of chitosan (CS) and polyvinyl-alcohol (PVA) based corneal stroma constructs using 3D printing process. Corneal stroma is tested for biocompatibility with human adipose tissue-derived mesenchymal stem cells (hASCs). Physico-chemical and chemical characterization of the construct was performed using scanning electron microscopy (SEM), fourier transforms infrared spectroscopy (FTIR). Optical transmittance was analyzed using UV-Spectrophotometer. Results showed fabricated constructs have required shape and size. SEM images showed construct has thickness of 400 µm. The FTIR spectra demonstrated the presence of various predicted peaks. The swelling and degradation studies of 13%(wt)PVA and 13%(wt)PVA/(1, 3, 5)%(wt)CS showed to have high swelling ratios of 7 days and degradation times of 30 days, respectively. The light transmittance values of the fabricated cornea constructs decreased with CS addition slightly. Tensile strength values decreased with increasing CS ratio, but we found to support intraocular pressure (IOP) which ranges from 12 to 22 mm-Hg. Preliminary biostability studies showed that composite constructs were compatible with hASCs even after 30 days’ of degradation, showing potential for these cells to be differentiated to stroma layer in future. This study has implications for the rapid and custom fabrication of various cornea constructs for clinical applications