234 research outputs found
Dependency structure matrix, genetic algorithms, and effective recombination
In many different fields, researchers are often confronted by problems arising from complex systems. Simple heuristics or even enumeration works quite well on small and easy problems; however, to efficiently solve large and difficult problems, proper decomposition is the key. In this paper, investigating and analyzing interactions between components of complex systems shed some light on problem decomposition. By recognizing three bare-bones interactions-modularity, hierarchy, and overlap, facet-wise models arc developed to dissect and inspect problem decomposition in the context of genetic algorithms. The proposed genetic algorithm design utilizes a matrix representation of an interaction graph to analyze and explicitly decompose the problem. The results from this paper should benefit research both technically and scientifically. Technically, this paper develops an automated dependency structure matrix clustering technique and utilizes it to design a model-building genetic algorithm that learns and delivers the problem structure. Scientifically, the explicit interaction model describes the problem structure very well and helps researchers gain important insights through the explicitness of the procedure.This work was sponsored by Taiwan National Science Council under grant NSC97-
2218-E-002-020-MY3, U.S. Air Force Office of Scientific Research, Air Force Material
Command, USAF, under grants FA9550-06-1-0370 and FA9550-06-1-0096, U.S. National
Science Foundation under CAREER grant ECS-0547013, ITR grant DMR-03-25939 at
Materials Computation Center, grant ISS-02-09199 at US National Center for Supercomputing Applications, UIUC, and the Portuguese Foundation for Science and Technology
under grants SFRH/BD/16980/2004 and PTDC/EIA/67776/2006
Particle swarm optimization with state-based adaptive velocity limit strategy
Velocity limit (VL) has been widely adopted in many variants of particle
swarm optimization (PSO) to prevent particles from searching outside the
solution space. Several adaptive VL strategies have been introduced with which
the performance of PSO can be improved. However, the existing adaptive VL
strategies simply adjust their VL based on iterations, leading to
unsatisfactory optimization results because of the incompatibility between VL
and the current searching state of particles. To deal with this problem, a
novel PSO variant with state-based adaptive velocity limit strategy (PSO-SAVL)
is proposed. In the proposed PSO-SAVL, VL is adaptively adjusted based on the
evolutionary state estimation (ESE) in which a high value of VL is set for
global searching state and a low value of VL is set for local searching state.
Besides that, limit handling strategies have been modified and adopted to
improve the capability of avoiding local optima. The good performance of
PSO-SAVL has been experimentally validated on a wide range of benchmark
functions with 50 dimensions. The satisfactory scalability of PSO-SAVL in
high-dimension and large-scale problems is also verified. Besides, the merits
of the strategies in PSO-SAVL are verified in experiments. Sensitivity analysis
for the relevant hyper-parameters in state-based adaptive VL strategy is
conducted, and insights in how to select these hyper-parameters are also
discussed.Comment: 33 pages, 8 figure
Genetic learning particle swarm optimization
Social learning in particle swarm optimization (PSO) helps collective efficiency, whereas individual reproduction in genetic algorithm (GA) facilitates global effectiveness. This observation recently leads to hybridizing PSO with GA for performance enhancement. However, existing work uses a mechanistic parallel superposition and research has shown that construction of superior exemplars in PSO is more effective. Hence, this paper first develops a new framework so as to organically hybridize PSO with another optimization technique for “learning.” This leads to a generalized “learning PSO” paradigm, the *L-PSO. The paradigm is composed of two cascading layers, the first for exemplar generation and the second for particle updates as per a normal PSO algorithm. Using genetic evolution to breed promising exemplars for PSO, a specific novel *L-PSO algorithm is proposed in the paper, termed genetic learning PSO (GL-PSO). In particular, genetic operators are used to generate exemplars from which particles learn and, in turn, historical search information of particles provides guidance to the evolution of the exemplars. By performing crossover, mutation, and selection on the historical information of particles, the constructed exemplars are not only well diversified, but also high qualified. Under such guidance, the global search ability and search efficiency of PSO are both enhanced. The proposed GL-PSO is tested on 42 benchmark functions widely adopted in the literature. Experimental results verify the effectiveness, efficiency, robustness, and scalability of the GL-PSO
Thermodynamic AI and the fluctuation frontier
Many Artificial Intelligence (AI) algorithms are inspired by physics and
employ stochastic fluctuations. We connect these physics-inspired AI algorithms
by unifying them under a single mathematical framework that we call
Thermodynamic AI. Seemingly disparate algorithmic classes can be described by
this framework, for example, (1) Generative diffusion models, (2) Bayesian
neural networks, (3) Monte Carlo sampling and (4) Simulated annealing. Such
Thermodynamic AI algorithms are currently run on digital hardware, ultimately
limiting their scalability and overall potential. Stochastic fluctuations
naturally occur in physical thermodynamic systems, and such fluctuations can be
viewed as a computational resource. Hence, we propose a novel computing
paradigm, where software and hardware become inseparable. Our algorithmic
unification allows us to identify a single full-stack paradigm, involving
Thermodynamic AI hardware, that could accelerate such algorithms. We contrast
Thermodynamic AI hardware with quantum computing where noise is a roadblock
rather than a resource. Thermodynamic AI hardware can be viewed as a novel form
of computing, since it uses a novel fundamental building block. We identify
stochastic bits (s-bits) and stochastic modes (s-modes) as the respective
building blocks for discrete and continuous Thermodynamic AI hardware. In
addition to these stochastic units, Thermodynamic AI hardware employs a
Maxwell's demon device that guides the system to produce non-trivial states. We
provide a few simple physical architectures for building these devices and we
develop a formalism for programming the hardware via gate sequences. We hope to
stimulate discussion around this new computing paradigm. Beyond acceleration,
we believe it will impact the design of both hardware and algorithms, while
also deepening our understanding of the connection between physics and
intelligence.Comment: 47 pages, 18 figures, Added relevant reference
Segment-based predominant learning swarm optimizer for large-scale optimization
Large-scale optimization has become a significant yet challenging area in evolutionary computation. To solve this problem, this paper proposes a novel segment-based predominant learning swarm optimizer (SPLSO) swarm optimizer through letting several predominant particles guide the learning of a particle. First, a segment-based learning strategy is proposed to randomly divide the whole dimensions into segments. During update, variables in different segments are evolved by learning from different exemplars while the ones in the same segment are evolved by the same exemplar. Second, to accelerate search speed and enhance search diversity, a predominant learning strategy is also proposed, which lets several predominant particles guide the update of a particle with each predominant particle responsible for one segment of dimensions. By combining these two learning strategies together, SPLSO evolves all dimensions simultaneously and possesses competitive exploration and exploitation abilities. Extensive experiments are conducted on two large-scale benchmark function sets to investigate the influence of each algorithmic component and comparisons with several state-of-the-art meta-heuristic algorithms dealing with large-scale problems demonstrate the competitive efficiency and effectiveness of the proposed optimizer. Further the scalability of the optimizer to solve problems with dimensionality up to 2000 is also verified
Adaptive multimodal continuous ant colony optimization
Seeking multiple optima simultaneously, which multimodal optimization aims at, has attracted increasing attention but remains challenging. Taking advantage of ant colony optimization algorithms in preserving high diversity, this paper intends to extend ant colony optimization algorithms to deal with multimodal optimization. First, combined with current niching methods, an adaptive multimodal continuous ant colony optimization algorithm is introduced. In this algorithm, an adaptive parameter adjustment is developed, which takes the difference among niches into consideration. Second, to accelerate convergence, a differential evolution mutation operator is alternatively utilized to build base vectors for ants to construct new solutions. Then, to enhance the exploitation, a local search scheme based on Gaussian distribution is self-adaptively performed around the seeds of niches. Together, the proposed algorithm affords a good balance between exploration and exploitation. Extensive experiments on 20 widely used benchmark multimodal functions are conducted to investigate the influence of each algorithmic component and results are compared with several state-of-the-art multimodal algorithms and winners of competitions on multimodal optimization. These comparisons demonstrate the competitive efficiency and effectiveness of the proposed algorithm, especially in dealing with complex problems with high numbers of local optima
A review of population-based metaheuristics for large-scale black-box global optimization: Part A
Scalability of optimization algorithms is a major challenge in coping with the ever growing size of optimization problems in a wide range of application areas from high-dimensional machine learning to complex large-scale engineering problems. The field of large-scale global optimization is concerned with improving the scalability of global optimization algorithms, particularly population-based metaheuristics. Such metaheuristics have been successfully applied to continuous, discrete, or combinatorial problems ranging from several thousand dimensions to billions of decision variables. In this two-part survey, we review recent studies in the field of large-scale black-box global optimization to help researchers and practitioners gain a bird’s-eye view of the field, learn about its major trends, and the state-of-the-art algorithms. Part of the series covers two major algorithmic approaches to large-scale global optimization: problem decomposition and memetic algorithms. Part of the series covers a range of other algorithmic approaches to large-scale global optimization, describes a wide range of problem areas, and finally touches upon the pitfalls and challenges of current research and identifies several potential areas for future research
PS-FW: A Hybrid Algorithm Based on Particle Swarm and Fireworks for Global Optimization
Particle swarm optimization (PSO) and fireworks algorithm (FWA) are two recently developed optimization methods which have been applied in various areas due to their simplicity and efficiency. However, when being applied to high-dimensional optimization problems, PSO algorithm may be trapped in the local optima owing to the lack of powerful global exploration capability, and fireworks algorithm is difficult to converge in some cases because of its relatively low local exploitation efficiency for noncore fireworks. In this paper, a hybrid algorithm called PS-FW is presented, in which the modified operators of FWA are embedded into the solving process of PSO. In the iteration process, the abandonment and supplement mechanism is adopted to balance the exploration and exploitation ability of PS-FW, and the modified explosion operator and the novel mutation operator are proposed to speed up the global convergence and to avoid prematurity. To verify the performance of the proposed PS-FW algorithm, 22 high-dimensional benchmark functions have been employed, and it is compared with PSO, FWA, stdPSO, CPSO, CLPSO, FIPS, Frankenstein, and ALWPSO algorithms. Results show that the PS-FW algorithm is an efficient, robust, and fast converging optimization method for solving global optimization problems
Multiple Relevant Feature Ensemble Selection Based on Multilayer Co-Evolutionary Consensus MapReduce
IEEE Although feature selection for large data has been intensively investigated in data mining, machine learning, and pattern recognition, the challenges are not just to invent new algorithms to handle noisy and uncertain large data in applications, but rather to link the multiple relevant feature sources, structured, or unstructured, to develop an effective feature reduction method. In this paper, we propose a multiple relevant feature ensemble selection (MRFES) algorithm based on multilayer co-evolutionary consensus MapReduce (MCCM). We construct an effective MCCM model to handle feature ensemble selection of large-scale datasets with multiple relevant feature sources, and explore the unified consistency aggregation between the local solutions and global dominance solutions achieved by the co-evolutionary memeplexes, which participate in the cooperative feature ensemble selection process. This model attempts to reach a mutual decision agreement among co-evolutionary memeplexes, which calls for the need for mechanisms to detect some noncooperative co-evolutionary behaviors and achieve better Nash equilibrium resolutions. Extensive experimental comparative studies substantiate the effectiveness of MRFES to solve large-scale dataset problems with the complex noise and multiple relevant feature sources on some well-known benchmark datasets. The algorithm can greatly facilitate the selection of relevant feature subsets coming from the original feature space with better accuracy, efficiency, and interpretability. Moreover, we apply MRFES to human cerebral cortex-based classification prediction. Such successful applications are expected to significantly scale up classification prediction for large-scale and complex brain data in terms of efficiency and feasibility
- …