On Unified Modeling, Canonical Duality-Triality Theory, Challenges and Breakthrough in Optimization

A unified model is addressed for general optimization problems in multi-scale complex systems. Based on necessary conditions and basic principles in physics, the canonical duality-triality theory is presented in a precise way to include traditional duality theories and popular methods as special applications. Two conjectures on NP-hardness are discussed, which should play important roles for correctly understanding and efficiently solving challenging real-world problems. Applications are illustrated for both nonconvex continuous optimization and mixed integer nonlinear programming. Misunderstandings and confusion on some basic concepts, such as objectivity, nonlinearity, Lagrangian, and Lagrange multiplier method are discussed and classified. Breakthrough from recent false challenges by C. Z\u{a}linescu and his co-workers are addressed. This paper will bridge a significant gap between optimization and multi-disciplinary fields of applied math and computational sciences.Comment: 28 pages, 2 figure

A Tour of Reinforcement Learning: The View from Continuous Control

This manuscript surveys reinforcement learning from the perspective of optimization and control with a focus on continuous control applications. It surveys the general formulation, terminology, and typical experimental implementations of reinforcement learning and reviews competing solution paradigms. In order to compare the relative merits of various techniques, this survey presents a case study of the Linear Quadratic Regulator (LQR) with unknown dynamics, perhaps the simplest and best-studied problem in optimal control. The manuscript describes how merging techniques from learning theory and control can provide non-asymptotic characterizations of LQR performance and shows that these characterizations tend to match experimental behavior. In turn, when revisiting more complex applications, many of the observed phenomena in LQR persist. In particular, theory and experiment demonstrate the role and importance of models and the cost of generality in reinforcement learning algorithms. This survey concludes with a discussion of some of the challenges in designing learning systems that safely and reliably interact with complex and uncertain environments and how tools from reinforcement learning and control might be combined to approach these challenges.Comment: minor revision with a few clarifying passages and corrected typo

A Review on Bilevel Optimization: From Classical to Evolutionary Approaches and Applications

Bilevel optimization is defined as a mathematical program, where an optimization problem contains another optimization problem as a constraint. These problems have received significant attention from the mathematical programming community. Only limited work exists on bilevel problems using evolutionary computation techniques; however, recently there has been an increasing interest due to the proliferation of practical applications and the potential of evolutionary algorithms in tackling these problems. This paper provides a comprehensive review on bilevel optimization from the basic principles to solution strategies; both classical and evolutionary. A number of potential application problems are also discussed. To offer the readers insights on the prominent developments in the field of bilevel optimization, we have performed an automated text-analysis of an extended list of papers published on bilevel optimization to date. This paper should motivate evolutionary computation researchers to pay more attention to this practical yet challenging area

Canonical Duality-Triality Theory: Bridge Between Nonconvex Analysis/Mechanics and Global Optimization in Complex Systems

Canonical duality-triality is a breakthrough methodological theory, which can be used not only for modeling complex systems within a unified framework, but also for solving a wide class of challenging problems from real-world applications. This paper presents a brief review on this theory, its philosophical origin, physics foundation, and mathematical statements in both finite and infinite dimensional spaces, with emphasizing on its role for bridging the gap between nonconvex analysis/mechanics and global optimization. Special attentions are paid on unified understanding the fundamental difficulties in large deformation mechanics, bifurcation/chaos in nonlinear science, and the NP-hard problems in global optimization, as well as the theorems, methods, and algorithms for solving these challenging problems. Misunderstandings and confusions on some basic concepts, such as objectivity, nonlinearity, Lagrangian, and generalized convexities are discussed and classified. Breakthrough from recent challenges and conceptual mistakes by M. Voisei, C. Zalinescu and his co-worker are addressed. Some open problems and future works in global optimization and nonconvex mechanics are proposed.Comment: 43 pages, 4 figures. appears in Mathematics and Mechanics of Solids, 201

A Bayesian Perspective of Statistical Machine Learning for Big Data

Statistical Machine Learning (SML) refers to a body of algorithms and methods by which computers are allowed to discover important features of input data sets which are often very large in size. The very task of feature discovery from data is essentially the meaning of the keyword `learning' in SML. Theoretical justifications for the effectiveness of the SML algorithms are underpinned by sound principles from different disciplines, such as Computer Science and Statistics. The theoretical underpinnings particularly justified by statistical inference methods are together termed as statistical learning theory. This paper provides a review of SML from a Bayesian decision theoretic point of view -- where we argue that many SML techniques are closely connected to making inference by using the so called Bayesian paradigm. We discuss many important SML techniques such as supervised and unsupervised learning, deep learning, online learning and Gaussian processes especially in the context of very large data sets where these are often employed. We present a dictionary which maps the key concepts of SML from Computer Science and Statistics. We illustrate the SML techniques with three moderately large data sets where we also discuss many practical implementation issues. Thus the review is especially targeted at statisticians and computer scientists who are aspiring to understand and apply SML for moderately large to big data sets.Comment: 26 pages, 3 figures, Review pape

Old and new challenges in Hadamard spaces

Hadamard spaces have traditionally played important roles in geometry and geometric group theory. More recently, they have additionally turned out to be a suitable framework for convex analysis, optimization and nonlinear probability theory. The attractiveness of these emerging subject fields stems, inter alia, from the fact that some of the new results have already found their applications both in mathematics and outside. Most remarkably, a gradient flow theorem in Hadamard spaces was used to attack a conjecture of Donaldson in Kahler geometry. Other areas of applications include metric geometry and minimization of submodular functions on modular lattices. There have been also applications into computational phylogenetics and imaging. We survey recent developments in Hadamard space analysis and optimization with the intention to advertise various open problems in the area. We also point out several fallacies in the existing proofs.Comment: 33 pages. Comments welcom

The Ontology of Knowledge Based Optimization

Optimization has been becoming a central of studies in mathematic and has many areas with different applications. However, many themes of optimization came from different area have not ties closing to origin concepts. This paper is to address some variants of optimization problems using ontology in order to building basic of knowledge about optimization, and then using it to enhance strategy to achieve knowledge based optimization.Comment: 20 pages, Proceedings of International/National Seminar Matematika dan Terapan (SiManTap), FMIPA Universitas Sumatera Utara, Medan: 11-3

Super-speeds with Zero-RAM: Next Generation Large-Scale Optimization in Your Laptop!

This article presents the novel breakthrough general purpose algorithm for large scale optimization problems. The novel algorithm is capable of achieving breakthrough speeds for very large-scale optimization on general purpose laptops and embedded systems. Application of the algorithm to the Griewank function was possible in up to 1 billion decision variables in double precision took only 64485 seconds (~18 hours) to solve, while consuming 7,630 MB (7.6 GB) or RAM on a single threaded laptop CPU. It shows that the algorithm is computationally and memory (space) linearly efficient, and can find the optimal or near-optimal solution in a fraction of the time and memory that many conventional algorithms require. It is envisaged that this will open up new possibilities of real-time large-scale problems on personal laptops and embedded systems.Comment: 7 pages, 4 figures, 7 table

Artificial Intelligence Paradigm for Customer Experience Management in Next-Generation Networks: Challenges and Perspectives

With advancements of next-generation programmable networks a traditional rule-based decision-making may not be able to adapt effectively to changing network and customer requirements and provide optimal customer experience. Customer experience management (CEM) components and implementation challenges with respect to operator, network, and business requirements must be understood to meet required demands. This paper gives an overview of CEM components and their design challenges. We elaborate on data analytics and artificial intelligence driven CEM and their functional differences. This overview provides a path toward autonomous CEM framework in next-generation networks and sets the groundwork for future enhancements.Comment: 9 pages, 5 figure

On the optimal control of some nonsmooth distributed parameter systems arising in mechanics

Variational inequalities are an important mathematical tool for modelling free boundary problems that arise in different application areas. Due to the intricate nonsmooth structure of the resulting models, their analysis and optimization is a difficult task that has drawn the attention of researchers for several decades. In this paper we focus on a class of variational inequalities, called of the second kind, with a twofold purpose. First, we aim at giving a glance at some of the most prominent applications of these types of variational inequalities in mechanics, and the related analytical and numerical difficulties. Second, we consider optimal control problems constrained by these variational inequalities and provide a thorough discussion on the existence of Lagrange multipliers and the different types of optimality systems that can be derived for the characterization of local minima. The article ends with a discussion of the main challenges and future perspectives of this important problem class