Stationary policies for the second moment stability in a class of stochastic systems

This paper presents a study on the uniform second moment stability for a class of stochastic control system. The main result states that the existence of the long-run average cost under a stationary policy is equivalent to the uniform second moment stability of the corresponding stochastic control system. To illustrate the result, a numerical example is developed to verify the uniform second moment stability of a simultaneous state-feedback control system

Almost periodic parameters for the second moment stability of linear stochastic systems

The technical note presents conditions to assure the uniform second moment stability for a class of linear time-varying stochastic systems. The system parameters are assumed to be almost periodic, a concept that is weaker than the periodic one. Under the existence of the long-run average cost associated with the stochastic system, we apply the almost periodicity to prove the desired stability result. An application illustrates the usefulness of the approach by implementing an almost periodic state-feedback strategy to control the velocity of a DC motor device

Second moment constraints and the control problem of Markov jump linear systems

This paper addresses the optimal solution for the regulator control problem of Markov jump linear systems subject to second moment constraints. We can characterize and obtain the solution explicitly using linear matrix inequalities techniques. The constraints are imposed on the second moment of both the system state and control vector, and the optimal solution is obtained in a computable form. To illustrate the usefulness of the approach, specially that for systems subject to abrupt variations and physical limitations, we present an application for one joint of the European Robotic Arm

On the numerical solution of the control problem of switched linear systems

This paper presents a method to compute an epsilon-optimal solution of the control problem of switched linear systems. A difficulty that emerges in the evalution of the optimal solution is that the cardinality of the solution set increases exponentially as long as the time-horizon increases linearly, which turns the problemintractable when the horizon is sufficiently large. We propose a numerical method to overcome such difficulty, in the sense that our approach allows the evalution of epsilon-optimal solutions with corresponding sets that do not increase exponentially

On the control of Markov jump linear systems with no mode observation: Application to a DC Motor device

This paper deals with the control problem of discrete-time Markov jump linear systems for the case in which the controller does not have access to the state of the Markov chain. A necessary optimal condition, which is nonlinear with respect to the optimizing variables, is introduced, and the corresponding solution is obtained through a variational convergent method. We illustrate the practical usefulness of the derived approach by applying it in the control speed of a real DC Motor device subject to abrupt power failures

Gradient-based optimization techniques for the design of static controllers for Markov jump linear systems with unobservable modes

The paper formulates the static control problem of Markov jump linear systems, assuming that the controller does not have access to the jump variable. We derive the expression of the gradient for the cost motivated by the evaluation of 10 gradient-based optimization techniques. The numerical efficiency of these techniques is verified by using the data obtained from practical experiments. The corresponding solution is used to design a scheme to control the velocity of a real-time DC motor device subject to abrupt power failures

Control of temperature to suppress the population of Rhyzopertha dominica (F.) (Coleoptera, Bostrichidae) in a grain silo prototype

This note presents some results from laboratory experiments that were conducted to characterize the influence of temperature in the mortality of adults of the insect known as lesser grain borer, Rhyzopertha dominica (F.). The insects were separated into strains and were appropriately immersed into a mass of wheat, and the infested wheat was stored in a silo bin of small dimensions with control of temperature. Our experiments indicates that the control of temperature can be used as a successful tool to increase the mortality of R. dominica in grain silos. The paper also describes the construction of the electrical device that implements the control of temperature in the proposed grain silo prototype

Average Optimal Stationary Policies: Convexity And Convergence Conditions In Linear Stochastic Control Systems

This paper provides a set of conditions for the existence of an optimal stationary policy in the long-run average cost control problem of linear stochastic systems. The main conditions are based on convexity of the cost by stage and convergence of trajectories. The discrete-time system is assumed to be linear with respect to the state but the controls take an abstract state-feedback structure, possibly a nonlinear one. An application is considered to illustrate the derived theory. ©2009 IEEE.33883393Bertsekas, D.P., Shreve, S.E., (1978) Stochastic Optimal Control: The Discrete Time Case, , Academic PressHernández-Lerma, O., Lasserre, J.B., (1996) Discrete-Time Markov Control Processes: Basic Optimality Criteria, , Springer-Verlag, New YorkAverage cost optimal policies for Markov control processes with Borel state space and unbounded costs (1990) Systems Control Lett., 15, pp. 349-356Arapostathis, A., Borkar, V.S., Fernández-Gaucherand, E., Ghosh, M.K., Marcus, S.I., Discrete-time controlled Markov processes with average cost criterion: A survey (1993) SIAM J. Control Optim., 31 (2), pp. 282-344Vargas, A.N., Do Val, J.B.R., On the existence of stationary optimal policies for the average-cost control problem of linear systems with abstract state-feedback Proc. 47th IEEE Conf. on Decision and Control, Cancun, Mexico, 2008, pp. 3682-3687A controllability condition for the existence of average optimal stationary policies of linear stochastic systems European Control Conference - ECC2009Anderson, B.D.O., Moore, J.B., (1979) Optimal Filtering, , Prentice-Hall, Englewood Cliffs, N.JBartle, R.G., (1964) The Elements of Real Analysis, , John Wiley & Sons, IncLewis, F.L., Syrmos, V.L., (1995) Optimal Control, , Wiley- Interscience, 2 edDavis, M.H.A., Vinter, R.B., (1985) Stochastic Modelling and Control, , Chapman and Hall, LondonKubrusly, C.S., On discrete stochastic bilinear systems stability (1986) J. Math. Anal. Appl., 113, pp. 36-58Knopp, K., (1956) Infinite Sequences and Series, , New York: Dover PublicationsWirth, F., Asymptotic behavior of the value functions of discrete-time discounted optimal control (2001) J. Optim. Theory Appl., 110 (1), pp. 183-210Rudin, W., (1987) Real and Complex Analysis, , McGraw-Hill Publishing Co.3rd editionCesari, L., (1983) Optimization-Theory and Applications. Problems with Ordinary Differential Equations, 17. , Springer Verlag. Applications of MathematicsGoldberg, R.R., (1964) Methods of Real Analysis, , Blaisdell Publishing Compan

Minimum Second Moment State For The Existence Of Average Optimal Stationary Policies In Linear Stochastic Systems

This note considers the long-run aver- age cost control problem for a class of discrete-time stochastic systems. The stochastic system is assumed to be linear with respect to the state but the controls possess a general structure, possibly a nonlinear one. The main contribution of this paper is to show that the existence of a minimal second moment system state implies the existence of an optimal stationary policy for the long-run average cost problem. A numerical example illustrates the derived result. © 2010 AACC.373377Syrmos, V., Abdallah, C., Dorato, P., Grigoriadis, K., Static output feedback- A survey (1997) Automatica, 33, pp. 125-137Vargas, A.N., Do Val, J.B.R., On the existence of stationary optimal policies for the average-cost control problem of linear systems with abstract state-feedback (2008) Proc. 47th IEEE Conf. on Decision and Control, pp. 3682-3687. , Cancun, MexicoA bounded cost condition for the existence of average optimal stationary policies of linear stochastic systems (2009) Proc. European Control Conference, pp. 38-42A controllability condition for the existence of average optimal stationary policies of linear stochastic systems (2009) Proc. European Control Conference, pp. 32-37Approximation of the optimal average cost for a class of linear stochastic control systems (2010) American Control Conference, , (submitted)Hernández-Lerma, O., Lasserre, J.B., (1996) Discrete-Time Markov Control Processes: Basic Optimality Criteria, , Springer-Verlag, New YorkAubin, J.-P., Optima and equilibria: An introduction to nonlinear analysis (1998) Graduate Texts in Mathematics, 140. , Springer-Verlag, seriesAnderson, B.D.O., Moore, J.B., (1979) Optimal Filtering, , Prentice-Hall, Englewood Cliffs, N.JWu, M.Y., A note on stability of linear time-varying systems (1974) IEEE Trans. Automat. Control, 19, pp. 162-162Bertsekas, D.P., Shreve, S.E., (1978) Stochastic Optimal Control: The Discrete Time Case, , Academic PressSchal, M., Average optimality in dynamic programming with general state space (1993) Math. Oper. Res., 18, pp. 163-17

Almost Periodic Parameters For The Second Moment Stability Of Linear Stochastic Systems

The technical note presents conditions to assure the uniform second moment stability for a class of linear time-varying stochastic systems. The system parameters are assumed to be almost periodic, a concept that is weaker than the periodic one. Under the existence of the long-run average cost associated with the stochastic system, we apply the almost periodicity to prove the desired stability result. An application illustrates the usefulness of the approach by implementing an almost periodic state-feedback strategy to control the velocity of a DC motor device. © 1963-2012 IEEE.59410721077Angelo, H.D., (1970) Linear Time-Varying Systems: Analysis and Synthesis, , Boston, MA: Allyn and BaconWillems, J.L., (1970) Stability Theory of Dynamical Systems, , Edinburgh, U. K.: Thomas Nelson and Sons LtdAnderson, B.D.O., Moore, J.B., Detectability and stabilizability of time-Varying discrete-Time linear systems (1981) SIAM Journal on Control and Optimization, 19 (1), pp. 20-32Halanay, A., Ionescu, V., (1994) Time-Varying Discrete Linear Systems, , Basel, Switzerland: Birkhäuser VerlagKwon, W.H., Pearson, A.E., On feedback stabilization of time-Varying discrete linear systems (1978) IEEE Transactions on Automatic Control, AC-23 (3), pp. 479-481Przyluski, K.M., Rolewicz, S., On stability of linear time-varying infinite-dimensional discrete-time systems (1984) Syst. Control Lett., 4 (5), pp. 307-315Kubrusly, C.S., Uniform stability for time-varying infinite-dimensional discrete linear systems (1988) IMA J. Math. Control Inform., 5 (4), pp. 269-283Lin, H., Antsaklis, P.J., Stability and stabilizability of switched linear systems: A survey of recent results (2009) IEEE Trans. Autom. Control, 54 (2), pp. 308-322. , FebChesi, G., Garulli, A., Tesi, A., Vicino, A., Series: Lecture Notes in control and information sciences (2009) Homogeneous Polynomial Forms for Robustness Analysis of Uncertain Systems, 390. , Berlin, Germany: SpringerKozin, F., A survey of stability of stochastic systems (1969) Automatica, 5, pp. 95-112Arnold, L., (1974) Stochastic Differential Equations: Theory and Applications, , New York: Wiley-InterscienceKushner, H.J., (1967) Stochastic Stability and Control, , New York: Academic PressKhasminskii, R.Z., (1980) Stochastic Stability of Differential Equations, , Alphen aan den Rijn, Germantown: Sijthoff & NoordhoffKats, I.Y., Martynyuk, A.A., (2002) Stability and Stabilization of Nonlinear Systems with Random Structures, , Abingdon, U. K.: Taylor & FrancisVargas, A.N., Do Val, J.B.R., Average cost and stability of time-varying linear systems (2010) IEEE Trans. Autom. Control, 55, pp. 714-720Conley, C., Miller, R., Asymptotic stability without uniform stability: Almost periodic coefficients (1965) J. Differential Equat., 1, pp. 333-336Fink, A., (1974) Almost Periodic Differential Equations, 377. , Berlin, Germany: Springer-Verlag Lecture Notes in MathematicsDiagana, T., Elaydi, S., Abdul-Aziz, Y., Population models in almost periodic environments (2007) J. Difference Equat. and Appl., 13 (4), pp. 239-260Hurd, H., Makagon, A., Miamee, A.G., On AR(1) models with periodic and almost periodic coefficients (2002) Stochastic Process. Appl., 100 (1-2), pp. 167-185Colonius, F., Wichtrey, T., Control systems with almost periodic excitations (2009) SIAM J. Control Optim., 48 (2), pp. 1055-1079Pinto, M., Robledo, G., Cauchy matrix for linear almost periodic systems and some consequences (2011) Nonlinear Anal., 74 (16), pp. 5426-5439Lii, K.-S., Rosenblatt, M., Estimation for almost periodic processes (2006) Annals of Statistics, 34 (3), pp. 1115-1139. , DOI 10.1214/009053606000000218Moore, J.B., Anderson, B.D.O., Coping with singular transition matrices in estimation and control stability theory (1980) International Journal of Control, 31 (3), pp. 571-586Van Willigenburg, L.G., De Koning, W., Linear systems theory revisited (2008) Automatica, 44 (7), pp. 1686-1696Corduneanu, C., (1989) Almost Periodic Functions, , 2nd ed. New York: Chelsea Publishing CompanyVesely, M., Construction of almost periodic sequences with given properties (2008) Electron. J. Diff. Equat., 1 (126), pp. 1-22Leonhard, W., (2001) Control of Electrical Drives, , 3rd ed. New York: Springer-VerlagRubaai, A., Kotaru, R., Online identification and control of a DC motor using learning adaptation of neural networks (2000) IEEE Trans. Ind. Appl., 36 (3), pp. 935-942Vargas, A.N., Costa, E.F., Do Val, J.B.R., On the control of markov jump linear systems with no mode observation: Application to a DC motor device (2013) Int. J. Robust Nonlin. Control, 23 (10), pp. 1136-2115Anderson, B.D.O., Moore, J.B., (1979) Optimal Filtering, , Englewood Cliffs, NJ: Prentice-Hal