701 research outputs found
A Survey of Evolutionary Continuous Dynamic Optimization Over Two Decades:Part B
Many real-world optimization problems are dynamic. The field of dynamic optimization deals with such problems where the search space changes over time. In this two-part paper, we present a comprehensive survey of the research in evolutionary dynamic optimization for single-objective unconstrained continuous problems over the last two decades. In Part A of this survey, we propose a new taxonomy for the components of dynamic optimization algorithms, namely, convergence detection, change detection, explicit archiving, diversity control, and population division and management. In comparison to the existing taxonomies, the proposed taxonomy covers some additional important components, such as convergence detection and computational resource allocation. Moreover, we significantly expand and improve the classifications of diversity control and multi-population methods, which are under-represented in the existing taxonomies. We then provide detailed technical descriptions and analysis of different components according to the suggested taxonomy. Part B of this survey provides an indepth analysis of the most commonly used benchmark problems, performance analysis methods, static optimization algorithms used as the optimization components in the dynamic optimization algorithms, and dynamic real-world applications. Finally, several opportunities for future work are pointed out
A survey of evolutionary continuous dynamic optimization over two decades – part A
Many real-world optimization problems are dynamic. The field of dynamic optimization deals with such problems where the search space changes over time. In this two-part paper, we present a comprehensive survey of the research in evolutionary dynamic optimization for single-objective unconstrained continuous problems over the last two decades. In Part A of this survey, we propose a new taxonomy for the components of dynamic optimization algorithms, namely, convergence detection, change detection, explicit archiving, diversity control, and population division and management. In comparison to the existing taxonomies, the proposed taxonomy covers some additional important components, such as convergence detection and computational resource allocation. Moreover, we significantly expand and improve the classifications of diversity control and multi-population methods, which are under-represented in the existing taxonomies. We then provide detailed technical descriptions and analysis of different components according to the suggested taxonomy. Part B of this survey provides an indepth analysis of the most commonly used benchmark problems, performance analysis methods, static optimization algorithms used as the optimization components in the dynamic optimization algorithms, and dynamic real-world applications. Finally, several opportunities for future work are pointed out
Scaling Up Dynamic Optimization Problems: A Divide-and-Conquer Approach
Scalability is a crucial aspect of designing efficient algorithms. Despite their prevalence, large-scale dynamic optimization problems are not well-studied in the literature. This paper is concerned with designing benchmarks and frameworks for the study of large-scale dynamic optimization problems. We start by a formal analysis of the moving peaks benchmark and show its nonseparable nature irrespective of its number of peaks. We then propose a composite moving peaks benchmark suite with exploitable modularity covering a wide range of scalable partially separable functions suitable for the study of large-scale dynamic optimization problems. The benchmark exhibits modularity, heterogeneity, and imbalance features to resemble real-world problems. To deal with the intricacies of large-scale dynamic optimization problems, we propose a decomposition-based coevolutionary framework which breaks a large-scale dynamic optimization problem into a set of lower dimensional components. A novel aspect of the framework is its efficient bi-level resource allocation mechanism which controls the budget assignment to components and the populations responsible for tracking multiple moving optima. Based on a comprehensive empirical study on a wide range of large-scale dynamic optimization problems with up to 200 dimensions, we show the crucial role of problem decomposition and resource allocation in dealing with these problems. The experimental results clearly show the superiority of the proposed framework over three other approaches in solving large-scale dynamic optimization problems
Changes from Classical Statistics to Modern Statistics and Data Science
A coordinate system is a foundation for every quantitative science,
engineering, and medicine. Classical physics and statistics are based on the
Cartesian coordinate system. The classical probability and hypothesis testing
theory can only be applied to Euclidean data. However, modern data in the real
world are from natural language processing, mathematical formulas, social
networks, transportation and sensor networks, computer visions, automations,
and biomedical measurements. The Euclidean assumption is not appropriate for
non Euclidean data. This perspective addresses the urgent need to overcome
those fundamental limitations and encourages extensions of classical
probability theory and hypothesis testing , diffusion models and stochastic
differential equations from Euclidean space to non Euclidean space. Artificial
intelligence such as natural language processing, computer vision, graphical
neural networks, manifold regression and inference theory, manifold learning,
graph neural networks, compositional diffusion models for automatically
compositional generations of concepts and demystifying machine learning
systems, has been rapidly developed. Differential manifold theory is the
mathematic foundations of deep learning and data science as well. We urgently
need to shift the paradigm for data analysis from the classical Euclidean data
analysis to both Euclidean and non Euclidean data analysis and develop more and
more innovative methods for describing, estimating and inferring non Euclidean
geometries of modern real datasets. A general framework for integrated analysis
of both Euclidean and non Euclidean data, composite AI, decision intelligence
and edge AI provide powerful innovative ideas and strategies for fundamentally
advancing AI. We are expected to marry statistics with AI, develop a unified
theory of modern statistics and drive next generation of AI and data science.Comment: 37 page
Recommended from our members
A Survey on Nature-Inspired Medical Image Analysis: A Step Further in Biomedical Data Integration
Landscape-based Evolutionary Algorithms for Dynamic Optimization Problems
In real-world structured optimization problems, specific objective functions, decision variables, constraints, data and/or parameters may vary over time. These problems are generally recognized as dynamic optimization problems (DOPs).
Evolutionary computation (EC) is a stochastic global search approach that has been successfully used to find optimal or near-optimal solutions for a wide range of optimization problems. EC is conceptually simple and imposes no specific mathematical properties requirement, thus showing competitive performance in dealing with static optimization problems. However, EC encounters challenges in dynamic problems on adaptability and efficiency. For the employment of EC in DOPs, two key points should be considered: the nature of optimization problems to be solved and the class of algorithms to be designed, where the crucial element of the former is landscape analysis and the latter frequently leads to the type of the algorithm.
A new approach named Landscape Influenced Dynamic Optimization Algorithm (LIDOA) is proposed to incorporate landscape analysis information into the search process, where a landscape-based strategy is integrated with appropriately designed evolutionary algorithms. In LIDOA, the knowledge learned in each landscape is archived and re-utilized in the new environment. Several classical evolutionary algorithms, including genetic algorithm (GA), self-adaptive differential evolution algorithm (jDE) and covariance matrix adaptation evolution strategy (CMA-ES), are employed to examine the efficiency of LIDOA, and four landscape measures are considered. Experimental results showed the overall advantage of LIDOA.
LIDOA with a single landscape measure is then expanded to multiple landscape measures. Three multi-measure methods are designed that are able to achieve good performance on evolutionary algorithms with appropriately integrated multiple landscape measures. According to the experimental results, LIDOA with multi-measure methods also improves the performance of GA, jDE and CMA-ES.
The second key point in employing multiple evolutionary algorithms in DOPs is also studied. Three multi-algorithm methods are investigated based on jDE and GA, where an information sharing strategy and a self-adjusted parameter strategy are designed. Experimental results show that with an appropriate integration mechanism, all three multi-algorithm methods can obtain better performance over a single algorithm. Two key parameters in multi-algorithm methods are discussed. The similarity check strategy with multi-measure is also integrated with three multi-algorithm methods, and experimental results demonstrate the efficacy of both multi-algorithm methods and multi-measure strategies.
Furthermore, to show the applicability of the concept in other algorithms, it is tested on quantum-inspired evolutionary algorithms. The performance of LIDOA with quantum-inspired evolutionary algorithms shows that LIDOA and quantum operators are beneficial for jDE, GA and CMA-ES, though their contributions vary.
Finally, the proposed algorithms are applied to two practical problems (parameter estimation for frequency-modulated (FM) sound waves and spread spectrum radar polyphase code design). With appropriately selected landscape measure(s), LIDOA is able to improve the performance on both problems. When the complexity of the two applicable problems increases, the proposed hybrid framework with a multi-algorithm and multi-measure method is more reliable
Simulation Intelligence: Towards a New Generation of Scientific Methods
The original "Seven Motifs" set forth a roadmap of essential methods for the
field of scientific computing, where a motif is an algorithmic method that
captures a pattern of computation and data movement. We present the "Nine
Motifs of Simulation Intelligence", a roadmap for the development and
integration of the essential algorithms necessary for a merger of scientific
computing, scientific simulation, and artificial intelligence. We call this
merger simulation intelligence (SI), for short. We argue the motifs of
simulation intelligence are interconnected and interdependent, much like the
components within the layers of an operating system. Using this metaphor, we
explore the nature of each layer of the simulation intelligence operating
system stack (SI-stack) and the motifs therein: (1) Multi-physics and
multi-scale modeling; (2) Surrogate modeling and emulation; (3)
Simulation-based inference; (4) Causal modeling and inference; (5) Agent-based
modeling; (6) Probabilistic programming; (7) Differentiable programming; (8)
Open-ended optimization; (9) Machine programming. We believe coordinated
efforts between motifs offers immense opportunity to accelerate scientific
discovery, from solving inverse problems in synthetic biology and climate
science, to directing nuclear energy experiments and predicting emergent
behavior in socioeconomic settings. We elaborate on each layer of the SI-stack,
detailing the state-of-art methods, presenting examples to highlight challenges
and opportunities, and advocating for specific ways to advance the motifs and
the synergies from their combinations. Advancing and integrating these
technologies can enable a robust and efficient hypothesis-simulation-analysis
type of scientific method, which we introduce with several use-cases for
human-machine teaming and automated science
- …