Off-policy Q-learning: set-point design for optimizing dual-rate rougher flotation operational processes

Rougher flotation, composed of unit processes operating at a fast time scale and economic performance measurements known as operational indices measured at a slower time scale, is very basic and the first concentration stage for flotation plants. Optimizing operational process for rougher flotation circuits is extremely important due to high economic profit arising from the optimality of operational indices. This paper presents a novel off-policy Q-learning method to learn theoptimal solution to rougher flotation operational processes without the knowledge of dynamics of unit processes and operational indices. To this end, first, the optimal operational control (OOC) for dual-rate rougher flotationprocesses is formulated. Second, H∞ tracking control problem is developed to optimally prescribe the set-points for the rougher flotation processes. Then, a zero-sum game off-policy Q-learning algorithm is proposed to find theoptimal set-points by using measured data. Finally, simulation experiments are employed to show the effectiveness of the proposed method

Super-Twisting-Algorithm-Based Terminal Sliding Mode Control for a Bioreactor System

This study proposes a class of super-twisting-algorithm-based (STA-based) terminal sliding mode control (TSMC) for a bioreactor system with second-order type dynamics. TSMC not only can retain the advantages of conventional sliding mode control (CSMC), including easy implementation, robustness to disturbances, and fast response, but also can make the system states converge to the equivalent point in a finite amount of time after the system states intersect the sliding surface. The chattering phenomena in TSMC will originally exist on the sliding surface after the system states achieve the sliding surface and before the system states reach the equivalent point. However, by using the super twisting algorithm (STA), the chattering phenomena can be obviously reduced. The proposed method is also compared with two other methods: (1) CSMC without STA and (2) TSMC without STA. Finally, the control schemes are applied to the control of a bioreactor system to illustrate the effectiveness and applicability. Simulation results show that it can achieve better performance by using the proposed method

Optimal Control for Aperiodic Dual-Rate Systems With Time-Varying Delays

[EN] In this work, we consider a dual-rate scenario with slow input and fast output. Our objective is the maximization of the decay rate of the system through the suitable choice of the n-input signals between two measures (periodic sampling) and their times of application. The optimization algorithm is extended for time-varying delays in order to make possible its implementation in networked control systems. We provide experimental results in an air levitation system to verify the validity of the algorithm in a real plant.This work was supported in part by the Spanish Ministry of Economy and Competitiveness (MINECO) under the Projects DPI2012-31303 and DPI2014-55932-C2-2-R.Aranda-Escolástico, E.; Salt Llobregat, JJ.; Guinaldo, M.; Chacon, J.; Dormido, S. (2018). Optimal Control for Aperiodic Dual-Rate Systems With Time-Varying Delays. Sensors. 18(5):1-19. https://doi.org/10.3390/s18051491S119185Mansano, R., Godoy, E., & Porto, A. (2014). The Benefits of Soft Sensor and Multi-Rate Control for the Implementation of Wireless Networked Control Systems. Sensors, 14(12), 24441-24461. doi:10.3390/s141224441Shao, Q. M., & Cinar, A. (2015). System identification and distributed control for multi-rate sampled systems. Journal of Process Control, 34, 1-12. doi:10.1016/j.jprocont.2015.06.010Albertos, P., & Salt, J. (2011). Non-uniform sampled-data control of MIMO systems. Annual Reviews in Control, 35(1), 65-76. doi:10.1016/j.arcontrol.2011.03.004Cuenca, A., & Salt, J. (2012). RST controller design for a non-uniform multi-rate control system. Journal of Process Control, 22(10), 1865-1877. doi:10.1016/j.jprocont.2012.09.010Cuenca, Á., Ojha, U., Salt, J., & Chow, M.-Y. (2015). A non-uniform multi-rate control strategy for a Markov chain-driven Networked Control System. Information Sciences, 321, 31-47. doi:10.1016/j.ins.2015.05.035Kalman, R. E., & Bertram, J. E. (1959). General synthesis procedure for computer control of single-loop and multiloop linear systems (an optimal sampling system). Transactions of the American Institute of Electrical Engineers, Part II: Applications and Industry, 77(6), 602-609. doi:10.1109/tai.1959.6371508Khargonekar, P., Poolla, K., & Tannenbaum, A. (1985). Robust control of linear time-invariant plants using periodic compensation. IEEE Transactions on Automatic Control, 30(11), 1088-1096. doi:10.1109/tac.1985.1103841Bamieh, B., Pearson, J. B., Francis, B. A., & Tannenbaum, A. (1991). A lifting technique for linear periodic systems with applications to sampled-data control. Systems & Control Letters, 17(2), 79-88. doi:10.1016/0167-6911(91)90033-bLi, D., Shah, S. L., Chen, T., & Qi, K. Z. (2001). Application of dual-rate modeling to CCR octane quality inferential control. IFAC Proceedings Volumes, 34(25), 353-357. doi:10.1016/s1474-6670(17)33849-1Salt, J., & Albertos, P. (2005). Model-based multirate controllers design. IEEE Transactions on Control Systems Technology, 13(6), 988-997. doi:10.1109/tcst.2005.857410Nemani, M., Tsao, T.-C., & Hutchinson, S. (1994). Multi-Rate Analysis and Design of Visual Feedback Digital Servo-Control System. Journal of Dynamic Systems, Measurement, and Control, 116(1), 45-55. doi:10.1115/1.2900680Sim, T. P., Lim, K. B., & Hong, G. S. (2002). Multirate predictor control scheme for visual servo control. IEE Proceedings - Control Theory and Applications, 149(2), 117-124. doi:10.1049/ip-cta:20020238Xinghui Huang, Nagamune, R., & Horowitz, R. (2006). A comparison of multirate robust track-following control synthesis techniques for dual-stage and multisensing servo systems in hard disk drives. IEEE Transactions on Magnetics, 42(7), 1896-1904. doi:10.1109/tmag.2006.875353Wu, Y., Liu, Y., & Zhang, W. (2013). A Discrete-Time Chattering Free Sliding Mode Control with Multirate Sampling Method for Flight Simulator. Mathematical Problems in Engineering, 2013, 1-8. doi:10.1155/2013/865493Salt, J., & Tomizuka, M. (2014). Hard disk drive control by model based dual-rate controller. Computation saving by interlacing. Mechatronics, 24(6), 691-700. doi:10.1016/j.mechatronics.2013.12.003Salt, J., Casanova, V., Cuenca, A., & Pizá, R. (2013). Multirate control with incomplete information over Profibus-DP network. International Journal of Systems Science, 45(7), 1589-1605. doi:10.1080/00207721.2013.844286Liu, F., Gao, H., Qiu, J., Yin, S., Fan, J., & Chai, T. (2014). Networked Multirate Output Feedback Control for Setpoints Compensation and Its Application to Rougher Flotation Process. IEEE Transactions on Industrial Electronics, 61(1), 460-468. doi:10.1109/tie.2013.2240640Khargonekar, P. P., & Sivashankar, N. (1991). 2 optimal control for sampled-data systems. Systems & Control Letters, 17(6), 425-436. doi:10.1016/0167-6911(91)90082-pTornero, J., Albertos, P., & Salt, J. (2001). Periodic Optimal Control of Multirate Sampled Data Systems. IFAC Proceedings Volumes, 34(12), 195-200. doi:10.1016/s1474-6670(17)34084-3Kim, C. H., Park, H. J., Lee, J., Lee, H. W., & Lee, K. D. (2015). Multi-rate optimal controller design for electromagnetic suspension systems via linear matrix inequality optimization. Journal of Applied Physics, 117(17), 17B506. doi:10.1063/1.4906588LEE, J. H., GELORMINO, M. S., & MORARIH, M. (1992). Model predictive control of multi-rate sampled-data systems: a state-space approach. International Journal of Control, 55(1), 153-191. doi:10.1080/00207179208934231Mizumoto, I., Ikejiri, M., & Takagi, T. (2015). Stable Adaptive Predictive Control System Design via Adaptive Output Predictor for Multi-rate Sampled Systems∗∗This work was partially supported by KAKENHI, the Grant-in-Aid for Scientific Research (C) 25420444, from the Japan Society for the Promotion of Science (JSPS). IFAC-PapersOnLine, 48(8), 1039-1044. doi:10.1016/j.ifacol.2015.09.105Carpiuc, S., & Lazar, C. (2016). Real-Time Multi-Rate Predictive Cascade Speed Control of Synchronous Machines in Automotive Electrical Traction Drives. IEEE Transactions on Industrial Electronics, 1-1. doi:10.1109/tie.2016.2561881Roshany-Yamchi, S., Cychowski, M., Negenborn, R. R., De Schutter, B., Delaney, K., & Connell, J. (2013). Kalman Filter-Based Distributed Predictive Control of Large-Scale Multi-Rate Systems: Application to Power Networks. IEEE Transactions on Control Systems Technology, 21(1), 27-39. doi:10.1109/tcst.2011.2172444Donkers, M. C. F., Tabuada, P., & Heemels, W. P. M. H. (2012). Minimum attention control for linear systems. Discrete Event Dynamic Systems, 24(2), 199-218. doi:10.1007/s10626-012-0155-xQuevedo, D. E., Ma, W.-J., & Gupta, V. (2015). Anytime Control Using Input Sequences With Markovian Processor Availability. IEEE Transactions on Automatic Control, 60(2), 515-521. doi:10.1109/tac.2014.2335311Aranda Escolastico, E., Guinaldo, M., Cuenca, A., Salt, J., & Dormido, S. (2017). Anytime Optimal Control Strategy for Multi-Rate Systems. IEEE Access, 5, 2790-2797. doi:10.1109/access.2017.2671906Guinaldo, M., Sánchez, J., & Dormido, S. (2017). Control en red basado en eventos: de lo centralizado a lo distribuido. Revista Iberoamericana de Automática e Informática Industrial RIAI, 14(1), 16-30. doi:10.1016/j.riai.2016.09.007Van Loan, C. (1977). The Sensitivity of the Matrix Exponential. SIAM Journal on Numerical Analysis, 14(6), 971-981. doi:10.1137/0714065Hazan, E. (2016). Introduction to Online Convex Optimization. Foundations and Trends® in Optimization, 2(3-4), 157-325. doi:10.1561/2400000013Sala, A., Cuenca, Á., & Salt, J. (2009). A retunable PID multi-rate controller for a networked control system. Information Sciences, 179(14), 2390-2402. doi:10.1016/j.ins.2009.02.017Chacon, J., Saenz, J., Torre, L., Diaz, J., & Esquembre, F. (2017). Design of a Low-Cost Air Levitation System for Teaching Control Engineering. Sensors, 17(10), 2321. doi:10.3390/s1710232