On Weak Topology for Optimal Control of Switched Nonlinear Systems

Optimal control of switched systems is challenging due to the discrete nature of the switching control input. The embedding-based approach addresses this challenge by solving a corresponding relaxed optimal control problem with only continuous inputs, and then projecting the relaxed solution back to obtain the optimal switching solution of the original problem. This paper presents a novel idea that views the embedding-based approach as a change of topology over the optimization space, resulting in a general procedure to construct a switched optimal control algorithm with guaranteed convergence to a local optimizer. Our result provides a unified topology based framework for the analysis and design of various embedding-based algorithms in solving the switched optimal control problem and includes many existing methods as special cases

A decentralized control and optimization framework for autonomic performance management of web-server systems

Web-based services such as online banking and e-commerce are often hosted on distributed computing systems comprising heterogeneous and networked servers in a data-center setting. To operate such systems efficiently while satisfying stringent quality-of-service (QoS) requirements, multiple performance-related parameters must be dynamically tuned to track changing operating conditions. For example, the workload to be processed may be time varying and hardware/software resources may fail during system operation. To cope with their growing scale and complexity, such computing systems must become largely autonomic, capable of being managed with minimal human intervention.This study develops a distributed cooperative-control framework using concepts from optimal control theory and hybrid dynamical systems to adaptively manage the performance of computer clusters operating in dynamic and uncertain environments. As case studies, we focus on power management and dynamic resource provisioning problems in such clusters.First, we apply the control framework to minimize the power consumed by a server cluster under a time-varying workload. The overall power-management problem is decomposed into smaller sub-problems and solved in cooperative fashion by individual controllers on each server. This approach allows for the scalable control of large computing systems. The control framework also adapts to controller failures and allows for the dynamic addition and removal of controllers during system operation. We validate the proposed approach using a discrete-event simulator with real-world workload traces, and our results indicate that the controllers achieve a 55% reduction in power consumption when compared to an uncontrolled system in which each server operates at its maximum frequency at all times.We then develop a distributed resource provisioning framework to achieve di®erentiated QoS among multiple online services using concepts from hybrid control. We use a discrete hybrid automaton to model the operation of the computing cluster. The resource provisioning problem combining both QoS control and power management is then solved using a decentralized model predictive controller to maximize the operating profits generated by the cluster according to a specified service level agreement. Simulation results indicate that the controller generates 27% additional profit when compared to an uncontrolled system.Ph.D., Electrical Engineering -- Drexel University, 200

Energy and Service Life Management Strategy for a Two-Drive Multi-Speed Electric Vehicle

Regulations of zero emission passenger cars appear on the horizon, and battery electric vehicles (BEV) are the main solution from the current market. It has been a focus of both academia and industry to extend their range. One of the main approaches is to reduce their energy consumption. Recent studies have shown that the two-drive topology and the multi-speed topology help to do so. It is natural to combine both concepts and to design a two-drive multi-speed topology for BEVs. Due to its more than one degree of freedom, an online energy management strategy (EMS) controlling torque set points of both electric motors and target gear positions is necessary to exploit its potential for reducing total energy consumption in real-world applications. There are numerous studies on EMSs for BEVs and hybrid electric vehicles. The overwhelming majority of them shared the same assumption: shift processes are neglectable. Based on the shift duration statistics, the shift processes of the most common transmissions in today’s market are too long to be ignored for an EMS with an operation frequency of at least 1 Hz. How to develop an EMS that considers shift processes? Suppose that an EMS is developed. It controls the powertrain in favour of low energy consumption, and the parts and the components are loaded accordingly. Some parts might fatigue and fail much faster than others, not because of poor construction dimensioning, but because of excessive use. What can an EMS do to prevent such an extreme scenario? Furthermore, is there a general way to design EMSs for multi-drive BEVs? This thesis is initiated by developing an online EMS for a two-drive multi-speed BEV called “Speed4E”, and tends to address the questions raised earlier. A predictive EMS in a Model Predictive Control framework is developed. A hybrid system considering the shift processes is proposed. Based on it and the Hybrid Minimum Principle, a solver and its algorithms are developed. The Principle is chosen for its accuracy and low time complexity, the two most important attributes of an online EMS. Minimizing the instantaneous Hamiltonian in the Principle is mathematically analysed. Several Lemmas that reduce the time complexity considerably are produced. Compared to an EMS that minimizes instantaneous energy consumption and ignores shift processes, the predictive EMS reduces the energy consumption in the Worldwide Harmonized Light Vehicles Test Cycle (WLTC) by 0.26 % and the shift count by 63.41 %. The hybrid system, the predictive EMS and the mathematical analysis are, as far as the author knows, first of their kinds. A novel multi-criteria operation strategy (MCOS) considering powertrain service life is proposed. Thanks to the hybrid system, the influence of the shift processes on fatigue is included. The MCOS extends the powertrain service life by several times but sacrifices the energy consumption. A general multi-drive (at least two) multi-speed electric powertrain is proposed. Its hybrid system is formulated. The Principle is applied to produce the optimality condition. It is showcased, how to modify certain sets and sample space in the formulation to have the general model and problem represent certain electric powertrains. A unified framework to design EMS for the general multi-drive electric powertrain is proposed, where the algorithms developed for the predictive EMS can be applied

Nonsmooth dynamic optimization of systems with varying structure

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 2011.Cataloged from PDF version of thesis.Includes bibliographical references (p. 357-365).In this thesis, an open-loop numerical dynamic optimization method for a class of dynamic systems is developed. The structure of the governing equations of the systems under consideration change depending on the values of the states, parameters and the controls. Therefore, these systems are called systems with varying structure. Such systems occur frequently in the models of electric and hydraulic circuits, chemical processes, biological networks and machinery. As a result, the determination of parameters and controls resulting in the optimal performance of these systems has been an important research topic. Unlike dynamic optimization problems where the structure of the underlying system is constant, the dynamic optimization of systems with varying structure requires the determination of the optimal evolution of the system structure in time in addition to optimal parameters and controls. The underlying varying structure results in nonsmooth and discontinuous optimization problems. The nonsmooth single shooting method introduced in this thesis uses concepts from nonsmooth analysis and nonsmooth optimization to solve dynamic optimization problems involving systems with varying structure whose dynamics can be described by locally Lipschitz continuous ordinary or differential-algebraic equations. The method converts the infinitedimensional dynamic optimization problem into an nonlinear program by parameterizing the controls. Unlike the state of the art, the method does not enumerate possible structures explicitly in the optimization and it does not depend on the discretization of the dynamics. Instead, it uses a special integration algorithm to compute state trajectories and derivative information. As a result, the method produces more accurate solutions to problems where the underlying dynamics is highly nonlinear and/or stiff for less effort than the state of the art. The thesis develops substitutes for the gradient and the Jacobian of a function in case these quantities do not exist. These substitutes are set-valued maps and an elements of these maps need to be computed for optimization purposes. Differential equations are derived whose solutions furnish the necessary elements. These differential equations have discontinuities in time. A numerical method for their solution is proposed based on state event location algorithms that detects these discontinuities. Necessary conditions of optimality for nonlinear programs are derived using these substitutes and it is shown that nonsmooth optimization methods called bundle methods can be used to obtain solutions satisfying these necessary conditions. Case studies compare the method to the state of the art and investigate its complexity empirically.by Mehmet Yunt.Ph.D

Optimal Switching Of 1-dof Oscillating Systems

This paper considers the class of hybrid linear second-order oscillating systems, in which two parameters are free to be assigned in a finite set of values. The control task is to decide, at any time instant, the value of the parameters as a function of the system state vector, in order to minimize a quadratic functional over an infinite horizon. The problem lends itself to cope with a variety of important applications, in diverse engineering fields. In the paper a numerical algorithm to compute the optimal switching rule is presented. Then the algorithm is applied to a simplified model of a vehicle suspension system with the aim of minimizing the chassis acceleration (comfort-oriented control). © Springer-Verlag Berlin Heidelberg 2007.4416 LNCS118130Branicky, M.S., Multiple Lyapunov functions and other analysis tools for switched and hybrid systems (1998) IEEE Trans. Automat. Contr, 43, pp. 475-482Hespanha, J.P., Uniform stability of switched linear systems : Extensions of LaSalle's principle (2004) IEEE Trans. Automat. Contr, 49, pp. 470-482Hockerman-Frommer, J., Kulkarni, S.R., Ramadge, P.J., Controller switching based on output predictions errors (1998) IEEE Trans. Automat. Contr, 43, pp. 596-607Johansson, M., Rantzer, A., Computation of piecewise quadratic Lyapunov functions for hybrid systems (1998) IEEE Trans. Automat. Contr, 43, pp. 555-559Ye, H., Michel, A.N., Hou, L., Stability theory for hybrid dynamical systems (1998) IEEE Trans. Automat. Contr, 43, pp. 461-474DeCarlo, R.A., Branicky, M.S., Pettersson, S., Lennartson, B., Perspectives and results on the stability and stabilizability of hybrid systems (2000) Proceedings of the IEEE, 88 (7), pp. 1069-1082Liberzon, D., Morse, A.S., Basic problems in stability and design of switched systems (1999) IEEE Control Systems Magazine, 19, pp. 59-70Liberzon, D., (2003) Switching in Systems and Control, , BirkhauserGeromel, J.C., Colaneri, P., Stabilization of continuous-time switched linear systems (2006) SIAM J. Control Optim, , to appearSussmann, H., New theories of set-valued differentials and new versuions of the maximum principle of optimal control theory, in Nonlinear Control in the year 2000, A. Isidori, F. Lamnabhi-Lagarrige, W. Respondek Eds, Vol2, pp- 487-526, Springer verlag, 2001Egerstedt, M., Ogren, P., Shakernia, O., Ligeros, J., Toward optimal control of switched linear systems (2000) Proc. 39th IEEE CDC, pp. 587-592Riedinger, P., Iung, C., Kratz, F., An optimal control approach for hybrid systems (2003) European J. of Contr, 49 (5), pp. 449-458Xuping, X., Antsaklis, P.J., Results and Perspectives on Computational Methods for Optimal Control of Switched Systems (2003) Sixth International Workshop on Hybrid Systems Computation and Control (HSCC), , Prague, Czech RepublicS. Bengea and R. DeCarlo, Optimal control of switching system, Automatica, 41, no. 2, pp. 11.27, 2005Giua, A., Seatzu, C. and Van der Mee, C.M., Optimal control of autonomous linear systems switched with a preassigned finite sequence, in Proc. of the ISIC,pp. 144-149, 2001Bemporad, A., Corona, D., Giua, A. and Seatzu, C., Optimal state feedback quadratic regulation of linear hybrid automata, in Proc. of the IFAC ADHS, pp. 407-412, 2003Seatzu, C., Corona, D., Giua, A., bemporad, A., Optimal control of continuous time switched affine systems (2006) IEEE Trans. on Automatic Control, 51 (5), pp. 726-741Athans, M., Falb, P., (1966) Optimal Control: An Introduction to the Theory and its Applications, , New York: McGraw-HillSavaresi, S.M., Silani, E., Bittanti, S., Acceleration-Driven- Damper (ADD): An optimal control algorithm for confort-oriented semiactive suspensions (2006) Trans. of the ASME, 127, pp. 218-228Geromel, J.C., Colaneri, P., Bolzern, P., Dynamic output feedback control of switched linear systems (2006), submittedWilliams, R.A., Automotive active suspensions, part I: Basic principles (1997) IMechE, 211, pp. 415-42

Multimodal Control in Uncertain Environments using Reinforcement Learning and Gaussian Processes

[ES] El control de sistemas complejos puede ser realizado descomponiendo la tarea de control en una secuencia de modos de control, o simplemente modos. Cada modo implementa una ley de retroalimentación hasta que se activa una condición de terminación, en respuesta a la ocurrencia de un evento exógeno/endógeno que indica que la ejecución del modo debe finalizar. En este trabajo se presenta una propuesta novedosa para encontrar una política de conmutación óptima para resolver el problema de control optimizando alguna medida de costo/beneficio. Una política óptima implementa un programa de control multimodal óptimo, el cual consiste en un encadenamiento de modos de control. La propuesta realizada incluye el desarrollo y formulación de un algoritmo basado en la idea de la programación dinámica integrando procesos Gaussianos y aprendizaje Bayesiano activo. Mediante el enfoque propuesto es posible realizar un uso eficiente de los datos para mejorar la exploración de las soluciones sobre espacios de estados continuos. Un caso de estudio representativo es abordado para demostrar el desempeño del algoritmo propuesto.[EN] The control of complex systems can be done decomposing the control task into a sequence of control modes, or modes for short. Each mode implements a parameterized feedback law until a termination condition is activated in response to the occurrence of an exogenous/endogenous event, which indicates that the execution mode must end. This paper presents a novel approach to find an optimal switching policy to solve a control problem by optimizing some measure of cost/benefit. An optimal policy implements an optimal multimodal control program, consisting in a sequence of control modes. The proposal includes the development of an algorithm based on the idea of dynamic programming integrating Gaussian processes and Bayesian active learning. In addition, an efficient use of the data to improve the exploration of the continuous state spaces solutions can be achieved through this approach. A representative case study is discussed and analyzed to demonstrate the performance of the proposed algorithm.De Paula, M.; Ávila, LO.; Sánchez Reinoso, C.; Acosta, GG. (2015). Control Multimodal en Entornos Inciertos usando Aprendizaje por Refuerzos y Procesos Gaussianos. Revista Iberoamericana de Automática e Informática industrial. 12(4):385-396. https://doi.org/10.1016/j.riai.2015.09.004OJS385396124Abate, A., Prandini, M., Lygeros, J., & Sastry, S. (2008). Probabilistic reachability and safety for controlled discrete time stochastic hybrid systems. Automatica, 44(11), 2724-2734. doi:10.1016/j.automatica.2008.03.027Adamek, F., M Sobotka, O Stursberg. 2008. Stochastic optimal control for hybrid systems with uncertain discrete dynamics. Proceedings of the IEEE International Conference on Automation Science and Engineering, 23-28. Washington D.C.Åström, Karl Johan, Bo Bernhardsson. 2003. System with Lebesgue Sampling. Directions in Mathematical Systems Theory and Optimization, LNCIS 268. LNCIS. Springer-Verlag Berlin Heidelberg.Axelsson, H., Wardi, Y., Egerstedt, M., & Verriest, E. I. (2007). Gradient Descent Approach to Optimal Mode Scheduling in Hybrid Dynamical Systems. Journal of Optimization Theory and Applications, 136(2), 167-186. doi:10.1007/s10957-007-9305-yBarton, P. I., Lee, C. K., & Yunt, M. (2006). Optimization of hybrid systems. Computers & Chemical Engineering, 30(10-12), 1576-1589. doi:10.1016/j.compchemeng.2006.05.024Bemporad, A., & Di Cairano, S. (2011). Model-Predictive Control of Discrete Hybrid Stochastic Automata. IEEE Transactions on Automatic Control, 56(6), 1307-1321. doi:10.1109/tac.2010.2084810Bemporad, A., & Morari, M. (1999). Control of systems integrating logic, dynamics, and constraints. Automatica, 35(3), 407-427. doi:10.1016/s0005-1098(98)00178-2Bensoussan, A.,J. L. Menaldi. 2000. Stochastic hybrid control. Journal of Mathematical Analysis and Applications 249.Bertsekas, Dimitri P. 2000. Dynamic Programming and Optimal Control, Vol. I. 2nd ed. Athena Scientific.Blackmore, L., Ono, M., Bektassov, A., & Williams, B. C. (2010). A Probabilistic Particle-Control Approximation of Chance-Constrained Stochastic Predictive Control. IEEE Transactions on Robotics, 26(3), 502-517. doi:10.1109/tro.2010.2044948Borrelli, F., Baotić, M., Bemporad, A., & Morari, M. (2005). Dynamic programming for constrained optimal control of discrete-time linear hybrid systems. Automatica, 41(10), 1709-1721. doi:10.1016/j.automatica.2005.04.017Bryson, Jr Arthur E., Yu-Chi Ho. 1975. Applied optimal control: optimization, estimation and control. Revised. Taylor & Francis.Busoniu, Lucian, Robert Babuska, Bart De Schutter,Damien Ernst. 2010. Reinforcement learning and dynamic programming using function approximators. 1.a ed. CRC Press.Cassandras, Christos G., John Lygeros. 2007. Stochastic hybrid systems. Boca Raton: Taylor & Francis.Deisenroth, Marc Peter. 2010. Efficient Reinforcement Learning Using Gaussian Processes. KIT Scientific Publishing.Deisenroth, M. P., Rasmussen, C. E., & Peters, J. (2009). Gaussian process dynamic programming. Neurocomputing, 72(7-9), 1508-1524. doi:10.1016/j.neucom.2008.12.019Di Cairano, S., Bemporad, A., & Júlvez, J. (2009). Event-driven optimization-based control of hybrid systems with integral continuous-time dynamics. Automatica, 45(5), 1243-1251. doi:10.1016/j.automatica.2008.12.011Ding, X.-C., Wardi, Y., & Egerstedt, M. (2009). On-Line Optimization of Switched-Mode Dynamical Systems. IEEE Transactions on Automatic Control, 54(9), 2266-2271. doi:10.1109/tac.2009.2026864Egerstedt, M., Wardi, Y., & Axelsson, H. (2006). Transition-Time Optimization for Switched-Mode Dynamical Systems. IEEE Transactions on Automatic Control, 51(1), 110-115. doi:10.1109/tac.2005.861711Girard, Agathe. 2004. Approximate methods for propagation of uncertainty with gaussian process models. University of Glasgow.Kuss, M. 2006. Gaussian process models for robust regression, classification, and reinforcement learning. Technische Universite Darmstadt.Liberzon, Daniel. 2003. Switching in systems and control. Systems & Control: Foundations & Applications. Boston: Birkhäuser Boston Inc.Lincoln, B., & Rantzer, A. (2006). Relaxing Dynamic Programming. IEEE Transactions on Automatic Control, 51(8), 1249-1260. doi:10.1109/tac.2006.878720Lunze, J., & Lehmann, D. (2010). A state-feedback approach to event-based control. Automatica, 46(1), 211-215. doi:10.1016/j.automatica.2009.10.035Mehta, Tejas,Magnus Egerstedt. 2005. Learning multi-modal control programs. Hybrid Systems: Computation and Control, 466-479. Lecture Notes in Computer Science. Springer Berlin.Mehta, T. R., & Egerstedt, M. (2006). An optimal control approach to mode generation in hybrid systems. Nonlinear Analysis: Theory, Methods & Applications, 65(5), 963-983. doi:10.1016/j.na.2005.07.044Mehta, T. R., & Egerstedt, M. (2008). Multi-modal control using adaptive motion description languages. Automatica, 44(7), 1912-1917. doi:10.1016/j.automatica.2007.11.024Pajares Martin-Sanz, G., y De la Cruz Garcia J.M. 2010. Aprendizaje automático. Un enfoque práctico, Cap. 12, Aprendizaje por Refuerzos. RA-MA.Rantzer, A. (2006). Relaxed dynamic programming in switching systems. IEE Proceedings - Control Theory and Applications, 153(5), 567-574. doi:10.1049/ip-cta:20050094Rasmussen, Carl Edward,Christopher K. I. Williams. 2006. Gaussian processes for machine learning. MIT Press.Rosenstein, Michael T.,Andrew G. Barto. 2004. Supervised Actor-Critic Reinforcement Learning. Handbook of Learning and Approximate Dynamic Programming, 359-380. John Wiley & Sons, Inc.Song, C., & Li, P. (2010). Near optimal control for a class of stochastic hybrid systems. Automatica, 46(9), 1553-1557. doi:10.1016/j.automatica.2010.05.024Sutton, Richard S.,Andrew G. Barto. 1998. Reinforcement learning: An introduction. MIT Press.Verdinelli, I., & Kadane, J. B. (1992). Bayesian Designs for Maximizing Information and Outcome. Journal of the American Statistical Association, 87(418), 510-515. doi:10.1080/01621459.1992.10475233Xu, Xuping,Panos J. Antsaklis. 2003. Results and perspectives on computational methods for optimal control of switched systems. Proceedings of the 6th international conference on Hybrid systems: computation and control, 540-555. Springer-Verlag.Xu, Y.-K., & Cao, X.-R. (2011). Lebesgue-Sampling-Based Optimal Control Problems With Time Aggregation. IEEE Transactions on Automatic Control, 56(5), 1097-1109. doi:10.1109/tac.2010.207361