Equivalency of Optimality Criteria of Markov Decision Process and Model Predictive Control

This paper shows that the optimal policy and value functions of a Markov Decision Process (MDP), either discounted or not, can be captured by a finite-horizon undiscounted Optimal Control Problem (OCP), even if based on an inexact model. This can be achieved by selecting a proper stage cost and terminal cost for the OCP. A very useful particular case of OCP is a Model Predictive Control (MPC) scheme where a deterministic (possibly nonlinear) model is used to limit the computational complexity. In practice, Reinforcement Learning algorithms can then be used to tune the parameterized MPC scheme. We verify the proposed theorems analytically in an LQR case and we investigate some other nonlinear examples in simulations.Comment: 11 pages and 13 figure

Therapeutic Effect of a Low-Level Laser on Acute Pain and Post-operative Mouth Opening After Closed Reduction of Mandibular-Condylar Fracture: Low-Level Laser Therapy in patients with closed reduction of mandibular-condylar fractur

Introduction: The aim of this study was to determine the therapeutic effect of low-level laser therapy on acute pain and the range of mouth opening after condylar closed-reduction surgery. The use of low-level lasers, especially to reduce inflammation and pain, has received more attention in recent years. The results of many studies performed in this field are contradictory, and the effectiveness of low-level lasers in the treatment of patients is still uncertain. Materials and Methods: This study was performed as a randomized, double-blinded clinical trial on 40 patients with condylar closed reduction surgery. Patients were randomly divided into two groups of 20 patients, including the placebo and intervention groups, in which the recent group received active low-level laser (100 mw, 2J/cm2, 20S/point, 14 extraoral points,7 days). The range of jaw movements after opening the intermaxillary-fixation was measured. Patients’ pain was measured using the visual analog scale (VAS). Data were analyzed using SPSS software version 21, the Chi-square test, and repeated measures ANOVA. Results: There was no significant difference between the study groups in terms of the range of jaw motion. Our results showed that at the end of the study, the mean pain score by VAS was 56.85 (SD = 1.387) in the intervention group and 60.95 (SD = 4.861) in the placebo group (P = 0.007). Conclusion: The results of this study indicated the effectiveness of low-level lasers in reducing acute pain in patients undergoing closed condylar surgery. Iranian Registry of Clinical Trials (IRCT20200520047519N1

Convex Neural Network-Based Cost Modifications for Learning Model Predictive Control

Developing model predictive control (MPC) schemes can be challenging for systems where an accurate model is not available, or too costly to develop. With the increasing availability of data and tools to treat them, learning-based MPC has of late attracted wide attention. It has recently been shown that adapting not only the MPC model, but also its cost function is conducive to achieving optimal closed-loop performance when an accurate model cannot be provided. In the learning context, this modification can be performed via parametrizing the MPC cost and adjusting the parameters via, e.g., reinforcement learning (RL). In this framework, simple cost parametrizations can be effective, but the underlying theory suggests that rich parametrizations in principle can be useful. In this paper, we propose such a cost parametrization using a class of neural networks (NNs) that preserves convexity. This choice avoids creating difficulties when solving the MPC problem via sensitivity-based solvers. In addition, this choice of cost parametrization ensures nominal stability of the resulting MPC scheme. Moreover, we detail how this choice can be applied to economic MPC problems where the cost function is generic and therefore does not necessarily fulfill any specific property

Experimental study on nanoparticles-assisted low-salinity water for enhanced oil recovery in asphaltenic oil reservoirs

The purpose of this research is to look into the augmentation of silica nanoparticles (NPs) with low salinity (LowSal) brine for EOR. A series of analyses, including oil/water interfacial tension (IFT) and rock wettability tests were undertaken to determine an optimal dispersion to flood into a porous carbonate core with a defined pore size distribution. At 60°C and 14.5 psi, the maximum drop (i.e., roughly 12.5 mN/m) in oil/water IFT by 0.3 wt% brine occurred, but when 0.08 wt% silica was added to the brine, the IFT reduced to 14.51 mN/m at 60°C and 14.5 psi. The wettability analysis revealed a significant reduction in contact angle, from 142° to 72° and 59°, using 0.04 and 0.08 wt% silica in LowSal brine, but the extent reduced by brine alone was insufficient. The results of rock pore size characterization were discussed in terms of the accomplishment of operating EOR in the porous medium in the presence of NPs. The addition of 0.08 wt% silica to the injected brine resulted in an additional oil recovery of 16.3% OOIP as well as a significant shift in the endpoints/cross-points of the oil/water relative permeability curves. The findings of this research might help improve oil recovery from asphaltenic oil reservoirs or, more environmentally friendly, remediate petroleum crude-oil polluted soil