Connections Between Adaptive Control and Optimization in Machine Learning

This paper demonstrates many immediate connections between adaptive control and optimization methods commonly employed in machine learning. Starting from common output error formulations, similarities in update law modifications are examined. Concepts in stability, performance, and learning, common to both fields are then discussed. Building on the similarities in update laws and common concepts, new intersections and opportunities for improved algorithm analysis are provided. In particular, a specific problem related to higher order learning is solved through insights obtained from these intersections.Comment: 18 page

Regression Filtration with Resetting to Provide Exponential Convergence of MRAC for Plants with Jump Change of Unknown Parameters

This paper proposes a new method to provide the exponential convergence of both the parameter and tracking errors of the composite adaptive control system without the requirement of the regressor persistent excitation (PE). Instead, the composite adaptation law obtained in this paper requires the regressor to be finitely exciting (FE) to guarantee the above-mentioned properties. Unlike known solutions, not only does it relax the PE requirement, but also it functions effectively under the condition of a jump change of the plant uncertainty parameters. To derive such an adaptation law, an integral filter of regressor with damping and resetting is proposed. It provides the required properties of the control system, and its output signal is bounded even when its input is subjected to noise and disturbances. A rigorous analytical proof of all mentioned properties of the developed adaptation law is presented. Such law is compared with the known composite ones relaxing the PE requirement. The wing-rock problem is used for the modeling of the developed composite MRAC system. The obtained results fully support the theoretical analysis and demonstrate the advantages of the proposed method.Comment: 12 pages, 3 figure