Learning on a Budget via Teacher Imitation

Deep Reinforcement Learning (RL) techniques can benefit greatly from leveraging prior experience, which can be either self-generated or acquired from other entities. Action advising is a framework that provides a flexible way to transfer such knowledge in the form of actions between teacher-student peers. However, due to the realistic concerns, the number of these interactions is limited with a budget; therefore, it is crucial to perform these in the most appropriate moments. There have been several promising studies recently that address this problem setting especially from the student's perspective. Despite their success, they have some shortcomings when it comes to the practical applicability and integrity as an overall solution to the learning from advice challenge. In this paper, we extend the idea of advice reusing via teacher imitation to construct a unified approach that addresses both advice collection and advice utilisation problems. We also propose a method to automatically tune the relevant hyperparameters of these components on-the-fly to make it able to adapt to any task with minimal human intervention. The experiments we performed in 5 different Atari games verify that our algorithm either surpasses or performs on-par with its top competitors while being far simpler to be employed. Furthermore, its individual components are also found to be providing significant advantages alone

Dynamical Trajectory Replanning for Uncertain Environments

We propose a dynamical reference generator equipped with an augmented transient “replanning” subsystem that modulates a feedback controller’s efforts to force a mechanical plant to track the reference signal. The replanner alters the reference generator’s output in the face of unanticipated disturbances that drive up the tracking error. We demonstrate that the new reference generator cannot destabilize the tracker, that tracking errors converge in the absence of disturbance, and that the overall coupled reference-tracker system cannot be destabilized by disturbances of bounded energy. We report the results of simulation studies exploring the performance of this new design applied to a two dimensional point mass particle interacting with fixed but unknown terrain obstacles. For more information: Kod*La

Monitoring and disinfection of biofilm-associated sulfate reducing bacteria on different substrata in a simulated recirculating cooling tower system

Microbial biofilm and corrosion in cooling systems are the most common problems that damage expensive equipment, cause loss of production, and increase maintenance costs. Sulfate reducing bacteria were considered the major bacterial group involved in microbiologically influenced corrosion (MIC). We investigated the survival and enumeration of biofilm-associated SRB on coupons of galvanized steel, stainless steel, and copper, which are materials used in the manufacturing of cooling systems. We also investigated the effect of monochloromine on SRB as in mixed species mature biofilms formed on coupons by simulating recirculating cooling water conditions, due to the better penetration feature in biofilms than the residual chlorine. It was concluded that SRB count increased with time in bulk water and the surfaces (P < 0.01). Experimental results supported by statistical analyses show that monochloromine is poorly effective on SRB colonies formed on galvanized and stainless steel surfaces

A drift-diffusion model for robotic obstacle avoidance

We develop a stochastic framework for modeling and analysis of robot navigation in the presence of obstacles. We show that, with appropriate assumptions, the probability of a robot avoiding a given obstacle can be reduced to a function of a single dimensionless parameter which captures all relevant quantities of the problem. This parameter is analogous to the Peclet number considered in the literature on mass transport in advection-diffusion fluid flows. Using the framework we also compute statistics of the time required to escape an obstacle in an informative case. The results of the computation show that adding noise to the navigation strategy can improve performance. Finally, we present experimental results that illustrate these performance improvements on a robotic platform. For more information: Kod*La

The influence of the rapamycin-derivate SDZ RAD on the healing of airway anastomoses

Objective: Among the many immunosuppressive effects of SDZ RAD (40-0(2-hydroxyethyl)-rapamycin), a rapamycin derivative, is the inhibition of fibroblast proliferation. Since the long-term success of lung transplantation is limited by the development of bronchiolitis obliterans, a fibroblast-associated progressive luminal obstruction of the terminal bronchioli, the use of SDZ RAD as immunosuppressive in pulmonary graft recipients may counteract this process. However, reduction of fibroblast activity, posttransplant, may impair the healing of the bronchial anastomoses. Materials and methods: The cervical trachea in pigs was denuded, divided and re-anastomosed with Prolene 4-0 single stitches. Control animals (group 1, n=4) were without, and study animals (group 2, n=6) were with SDZ RAD therapy (1.25 mg/kg/day, p.o., 14 days). After 14 days, the pigs were sacrificed. The anastomoses were examined histologically, and breaking strength of tracheal strips of 5-mm width was measured. Results: All animals survived without complications. Serum levels of SDZ RAD were 30.9±8.7 ng/ml (recommended level 20-40 ng/ml). All anastomoses healed macroscopically without difference between the two groups. Breaking strength was significantly lower in the treated animals (group 1 vs. group 2: 11.75±0.35 vs. 7.69±1.39 N, P=0.01). Histology did not show a significant change in histoarchitecture between the groups. Conclusions: Although SDZ RAD significantly reduced the breaking strength of the tracheal anastomosis, no obvious histological differences between treated and untreated animals could be detected. Since this model does not reflect the clinical situation, further investigations are necessary to reveal the effect of SDZ RAD on airway wound healing in concert with a contemporary clinically used multidrug immunosuppressive regimen in allograft recipient

Investigation of mitophagy biomarkers in corneal epithelium of keratoconus patients

Purpose: The pathological mechanisms of keratoconus (KC) have not been elucidated yet. Mitophagy is an important mechanism that eliminates damaged mitochondria under oxidative stress, and it could be one of the leading pathological causes of KC. This study aimed to find out the role of mitophagy in the keratoconic corneal epithelium. Methods: The corneal epithelia were collected from the 103 progressive KC patients and the 46 control subjects. The real-time quantitative PCR was performed for PTEN-putative kinase-1 (PINK1), PARKIN, p62, and BNIP3 gene expressions in 31 KC and 9 control subjects. Western blot analyses were performed to investigate the protein expressions of PINK1, PARKIN, LC3B, ATG5, and BECLIN in the remaining 109 corneal epithelium samples from 72 patients and 37 control subjects. Results: mRNA and protein expressions of PINK1 decreased significantly in the corneal epithelium of KC patients compared to the control subjects. No significant change was found in mRNA levels of PARKIN, p62, and BNIP3 in KC patients. The protein expression of PARKIN, LC3B, ATG5, and Beclin did not significantly differ between KC patients and control subjects. Gene expression levels of mitophagy biomarkers were not affected by the KC grade. Conclusions: PINK1/PARKIN-dependent mitophagy is affected in the keratoconic corneal epithelium. We found significant decreases in both mRNA and protein expressions of PINK1 in the keratoconic corneal epithelium. However, we did not observe any other significant change in mitophagy markers. Mitochondrial stress-related mitophagy pathways could be interrupted by the decreased levels of PINK1 in the keratoconic corneal epithelium, but solely PINK1 dysregulation is not likely to induce KC pathogenesis

Evaluation of spinal-paraspinal parameters to determine segmentation of the vertebrae

Purpose: We aimed to evaluate whether lumbar vertebrae can be correctly numbered using auxiliary parameters. Material and methods: Vertebra corpus shape, O’Driscoll classification, lumbosacral axis angle, last two square vertebra dimensions, orifice of right renal artery (RRA), orifice of celiac truncus (CT), orifice of superior mesenteric artery (SMA), vena cava inferior confluence (CVC), abdominal aorta bifurcation (AB), and iliolumbar ligament were evaluated in this study. Results: Lumbosacral transitional vertebrae (LSTV) were observed in 13 (9%) patients. The most common locations of the paraspinal parameters were: RRA: L1 vertebrae (45%), SMA: L1 vertebrae (66%), CT: T12 vertebrae (46%), AB: L4 vertebrae (63%), and CVC: L4 vertebrae (52%). Conclusions: According to the results of our study, no single parameter in the magnetic resonance imaging can accurately indicate the number of vertebrae without counting the levels. As a result, we believe that these parameters may be suspicious in terms of the presence of LSTV rather than the correct level

Reluctance network model for the in-wheel motor of a series-hybrid truck using Tooth Contour Method

Recently, series-hybrid drivetrains are given more emphasis over parallel-hybrid drivetrains due to their simplicity, freedom in implementation and higher efficiency. However, the modeling phase of such machines takes either long calculation time with numerical methods such as Finite Element Analysis (FEA) or produces low accurate results with Magnetic Equivalent Circuit (MEC) models. In this paper a method is used to build a reluctance network where high accuracy is obtained yet still with acceptable calculation time