A Maintenance Planning Framework using Online and Offline Deep Reinforcement Learning

Cost-effective asset management is an area of interest across several industries. Specifically, this paper develops a deep reinforcement learning (DRL) solution to automatically determine an optimal rehabilitation policy for continuously deteriorating water pipes. We approach the problem of rehabilitation planning in an online and offline DRL setting. In online DRL, the agent interacts with a simulated environment of multiple pipes with distinct lengths, materials, and failure rate characteristics. We train the agent using deep Q-learning (DQN) to learn an optimal policy with minimal average costs and reduced failure probability. In offline learning, the agent uses static data, e.g., DQN replay data, to learn an optimal policy via a conservative Q-learning algorithm without further interactions with the environment. We demonstrate that DRL-based policies improve over standard preventive, corrective, and greedy planning alternatives. Additionally, learning from the fixed DQN replay dataset in an offline setting further improves the performance. The results warrant that the existing deterioration profiles of water pipes consisting of large and diverse states and action trajectories provide a valuable avenue to learn rehabilitation policies in the offline setting, which can be further fine-tuned using the simulator.Comment: Published Neural Comput & Applic (2023), 12 pages, 8 Figur

New insights in chronic pain after common operations

The first part of the thesis addresses CPP after laparotomy judged to be caused by adhesions. It aims to assess long term results of laparoscopic adhesiolysis as a treatment for CPP. The second part of the thesis addresses CPP after inguinal hernia repair (CPIP). It aims to determine whether a self-gripping mesh for open inguinal hernia repair according to Lichtenstein influences the incidence of CPIP. The third part focuses on the methodological quality and comparability of studies addressing CPIP after Lichtenstein hernioplasty. It aims to assess whether study outcomes are valid and can be compared to each other to make firm conclusions about the best treatment or prevention method for CPIP

Superconductivity-Induced Transfer of In-Plane Spectral Weight in Bi2Sr2CaCu2O8: Resolving a Controversy

We present a detailed analysis of the superconductivity-induced redistribution of optical spectral weight in Bi2Sr2CaCu2O8 near optimal doping. It confirms the previous conclusion by Molegraaf et al. (Science 66, 2239 (2002)), that the integrated low-frequency spectral weight shows an extra increase below Tc. Since the region, where the change of the integrated spectral weight is not compensated, extends well above 2.5 eV, this transfer is caused by the transfer of spectral weight from interband to intraband region and only partially by the narrowing of the intraband peak. We show that the opposite assertion by Boris et al. (Science 304, 708 (2004)) regarding this compound, is unlikely the consequence of any obvious discrepancies between the actual experimental data.Comment: ReVTeX, 9 pages, 8 encapsulated postscript figures, several typo's correcte

Optical sum rule in metals with a strong interaction

The restricted optical sum rule and its dependence on the temperature, a superconducting gap and the cutoff energy have been investigated. As known this sum rule depends on the cutoff energy and the relaxation rate even for a homogeneous electron gas interacting with impurities or phonons. It is shown here that additional dependence of the spectral weight on a superconducting gap is very small in this model and this effect disappears totally when the relaxation rate is equal zero. The model metal with a single band is considered in details. It is well known that for this model there is the dependence of the sum rule on the temperature and the energy gap even in the case when the relaxation is absent. This dependence exists due to the smearing of the electron distribution function and it is expressed in the terms of Sommerfeld expansion. Here it is shown that these effects are considerably smaller than that of related with the relaxation rate if the band width is larger than the average phonon frequency. It is shown also that the experimental data about the temperature dependence of the spectral weight for the high- materials can be successfully explained in the framework approach based on the temperature dependence of the relaxation rateComment: 13 pages, 7 figures, the talk given on Internatinal coference on theoretical physics, april 11-16,2005, Mosco

In-plane optical spectral weight transfer in optimally doped Bi2_{2}Sr2_{2}Ca2_{2}Cu3_{3}O10_{10}

We examine the redistribution of the in-plane optical spectral weight in the normal and superconducting state in tri-layer \bbb (Bi2223) near optimal doping ($T_c$ = 110 K) on a single crystal via infrared reflectivity and spectroscopic ellipsometry. We report the temperature dependence of the low-frequency integrated spectral weight $W(\Omega_c)$ for different values of the cutoff energy $\Omega_c$. Two different model-independent analyses consistently show that for $\Omega_c$ = 1 eV, which is below the charge transfer gap, $W(\Omega_c)$ increases below $T_c$, implying the lowering of the kinetic energy of the holes. This is opposite to the BCS scenario, but it follows the same trend observed in the bi-layer compound \bb (Bi2212). The size of this effect is larger in Bi2223 than in Bi2212, approximately scaling with the critical temperature. In the normal state, the temperature dependence of $W(\Omega_c)$ is close to $T^2$ up to 300 K

Electromagnetic response of superconductors and optical sum rule

The interrelation between the condensation energy and the optical sum rules has been investigated. It has been shown that the so called 'partial' sum rule violation is related mainly to a temperature dependence of the relaxation rate rather than to the appearance of superconductivity itself. Moreover, we demonstrate that the experimental data on the temperature dependence of the optical sum rule can be explained rather well by an account of strong electron-phonon interaction.Comment: 16 pages, 1 figure. Submitted to Solid State Communication