Connections Between Adaptive Control and Optimization in Machine Learning

This paper demonstrates many immediate connections between adaptive control and optimization methods commonly employed in machine learning. Starting from common output error formulations, similarities in update law modifications are examined. Concepts in stability, performance, and learning, common to both fields are then discussed. Building on the similarities in update laws and common concepts, new intersections and opportunities for improved algorithm analysis are provided. In particular, a specific problem related to higher order learning is solved through insights obtained from these intersections.Comment: 18 page

Understanding Complexity

{Excerpt} In development agencies, paradigms of linear causality condition much thinking and practice. They encourage command-and-control hierarchies, centralize decision making, and dampen creativity and innovation. Globalization demands that organizations see our turbulent world as a collection of evolving ecosystems. To survive and flourish they must then be adaptable and fleet footed. Notions of complexity offer a wealth of insights and guidance to 21st century organizations that strive to do so