Should Evolution Necessarily be Egolution?

In the article I study the evolutionary adaptivity of two simple population models, based on either altruistic or egoistic law of energy exchange. The computational experiments show the convincing advantage of the altruists, which brings us to a small discussion about genetic algorithms and extraterrestrial life

On the use of two reference points in decomposition based multiobjective evolutionary algorithms

Decomposition based multiobjective evolutionary algorithms approximate the Pareto front of a multiobjective optimization problem by optimizing a set of subproblems in a collaborative manner. Often, each subproblem is associated with a direction vector and a reference point. The settings of these parameters have a very critical impact on convergence and diversity of the algorithm. Some work has been done to study how to set and adjust direction vectors to enhance algorithm performance for particular problems. In contrast, little effort has been made to study how to use reference points for controlling diversity in decomposition based algorithms. In this paper, we first study the impact of the reference point setting on selection in decomposition based algorithms. To balance the diversity and convergence, a new variant of the multiobjective evolutionary algorithm based on decomposition with both the ideal point and the nadir point is then proposed. This new variant also employs an improved global replacement strategy for performance enhancement. Comparison of our proposed algorithm with some other state-of-the-art algorithms is conducted on a set of multiobjective test problems. Experimental results show that our proposed algorithm is promising

MOEA/D-Based Probabilistic PBI Approach for Risk-Based Optimal Operation of Hybrid Energy System with Intermittent Power Uncertainty

The stochastic nature of intermittent energy resources has brought significant challenges to the optimal operation of the hybrid energy systems. This article proposes a probabilistic multiobjective evolutionary algorithm based on decomposition (MOEA/D) method with two-step risk-based decision-making strategy to tackle this problem. A scenario-based technique is first utilized to generate a stochastic model of the hybrid energy system. Those scenarios divide the feasible domain into several regions. Then, based on the MOEA/D framework, a probabilistic penalty-based boundary intersection (PBI) with gradient descent differential evolution (GDDE) algorithm is proposed to search the optimal scheme from these regions under different uncertainty budgets. To ensure reliable and low risk operation of the hybrid energy system, the Markov inequality is employed to deduce a proper interval of the uncertainty budget. Further, a fuzzy grid technique is proposed to choose the best scheme for real-world applications. The experimental results confirm that the probabilistic adjustable parameters can properly control the uncertainty budget and lower the risk probability. Further, it is also shown that the proposed MOEA/D-GDDE can significantly enhance the optimization efficiency.National Natural Science Fund; National Natural Science Key Fund

Evolutionary Algorithms for Static and Dynamic Multiobjective Optimization

Many real-world optimization problems consist of a number of conflicting objectives that have to be optimized simultaneously. Due to the presence of multiple conflicting ob- jectives, there is no single solution that can optimize all the objectives. Therefore, the resulting multiobjective optimization problems (MOPs) resort to a set of trade-off op- timal solutions, called the Pareto set in the decision space and the Pareto front in the objective space. Traditional optimization methods can at best find one solution in a sin- gle run, thereby making them inefficient to solve MOPs. In contrast, evolutionary algo- rithms (EAs) are able to approximate multiple optimal solutions in a single run. This strength makes EAs good candidates for solving MOPs. Over the past several decades, there have been increasing research interests in developing EAs or improving their perfor- mance, resulting in a large number of contributions towards the applicability of EAs for MOPs. However, the performance of EAs depends largely on the properties of the MOPs in question, e.g., static/dynamic optimization environments, simple/complex Pareto front characteristics, and low/high dimensionality. Different problem properties may pose dis- tinct optimization difficulties to EAs. For example, dynamic (time-varying) MOPs are generally more challenging than static ones to EAs. Therefore, it is not trivial to further study EAs in order to make them widely applicable to MOPs with various optimization scenarios or problem properties. This thesis is devoted to exploring EAs’ ability to solve a variety of MOPs with dif- ferent problem characteristics, attempting to widen EAs’ applicability and enhance their general performance. To start with, decomposition-based EAs are enhanced by incorpo- rating two-phase search and niche-guided solution selection strategies so as to make them suitable for solving MOPs with complex Pareto fronts. Second, new scalarizing functions are proposed and their impacts on evolutionary multiobjective optimization are exten- sively studied. On the basis of the new scalarizing functions, an efficient decomposition- based EA is introduced to deal with a class of hard MOPs. Third, a diversity-first- and-convergence-second sorting method is suggested to handle possible drawbacks of convergence-first based sorting methods. The new sorting method is then combined with strength based fitness assignment, with the aid of reference directions, to optimize MOPs with an increase of objective dimensionality. After that, we study the field of dynamic multiobjective optimization where objective functions and constraints can change over time. A new set of test problems consisting of a wide range of dynamic characteristics is introduced at an attempt to standardize test environments in dynamic multiobjective optimization, thereby aiding fair algorithm comparison and deep performance analysis. Finally, a dynamic EA is developed to tackle dynamic MOPs by exploiting the advan- tages of both generational and steady-state algorithms. All the proposed approaches have been extensively examined against existing state-of-the-art methods, showing fairly good performance in a variety of test scenarios. The research work presented in the thesis is the output of initiative and novel attempts to tackle some challenging issues in evolutionary multiobjective optimization. This re- search has not only extended the applicability of some of the existing approaches, such as decomposition-based or Pareto-based algorithms, for complex or hard MOPs, but also contributed to moving forward research in the field of dynamic multiobjective optimiza- tion with novel ideas including new test suites and novel algorithm design

Gen M., Specification of genetic search directions in cellular multi-objective genetic algorithms

Abstract. When we try to implement a multi-objective genetic algorithm (MOGA) with variable weights for finding a set of Pareto optimal solutions, one difficulty lies in determining appropriate search directions for genetic search. In our MOGA, a weight value for each objective in a scalar fitness function was randomly specified. Based on the fitness function with the randomly specified weight values, a pair of parent solutions are selected for generating a new solution by genetic operations. In order to find a variety of Pareto optimal solutions of a multi-objective optimization problem, weight vectors should be distributed uniformly on the Pareto optimal surface. In this paper, we propose a proportional weight specification method for our MOGA and its variants. We apply the proposed weight specification method to our MOGA and a cellular MOGA for examining its effect on their search ability.

Multi-Objective Reinforcement Learning

The recent surge in artificial intelligence (AI) agents assisting us in daily tasks suggests that these agents possess the ability to comprehend key aspects of our environment, thereby facilitating better decision-making. Presently, this understanding is predominantly acquired through data-driven learning methods. Notably, reinforcement learning (RL) stands out as a natural framework for agents to acquire behaviors by interacting with their environment and learning from feedback. However, despite the effectiveness of RL in training agents to optimize a single objective, such as minimizing cost or maximizing performance, it overlooks the inherent complexity of decision-making in real-world scenarios where multiple objectives may need to be considered simultaneously. Indeed, an essential aspect that remains understudied is the human tendency to make compromises in various situations, influenced by values, circumstances, or mood. This limitation underscores the need for advancements in AI methodologies to address the nuanced trade-offs inherent to human decision-making. Thus, this work aims to explore the extension of RL principles into multi-objective settings, where agents can learn behaviors that balance competing objectives, thereby enabling more adaptable and personalized AI systems. In the first part of this thesis, we explore the domain of multi-objective reinforcement learning (MORL), a recent technique aimed at enabling AI agents to acquire diverse behaviors associated with different trade-offs from multiple feedback signals. While MORL is relatively recent, works in this field often rely on existing knowledge coming from older fields such as multi-objective optimization (MOO) and RL. Our initial contribution involves a comprehensive analysis of the relationships between RL, MOO, and MORL. This examination culminates in the development of a taxonomy for categorizing MORL algorithms, drawing on concepts derived from preceding fields. Building upon this foundational understanding, we proceed to investigate the feasibility of leveraging techniques from MOO and RL to enhance MORL methodologies. This exploration yields several contributions. Among these, we introduce the utilization of metaheuristics to address the exploration-exploitation dilemma in MORL. Additionally, we introduce a versatile framework rooted in the derived taxonomy, facilitating the creation of novel MORL algorithms based on techniques coming from MOO and RL. Furthermore, our efforts extend towards improving the scientific rigor and practical applicability of MORL in real-world scenarios. To this end, we introduce methods and a suite of open-source tools that have become the standard in MORL. Many real-world situations also involve collaboration among multiple agents to accomplish tasks efficiently. Therefore, the second part of this thesis transitions to settings involving multiple agents, leading to the nascent field of multi-objective multi-agent reinforcement learning (MOMARL). In this domain, as an initial contribution, we release a comprehensive set of open-source utilities aimed to accelerate and establish a robust foundation for research within this evolving domain. Furthermore, we perform an initial study exploring the transferability of knowledge and methodologies from both MORL and multi-agent RL to the MOMARL settings. Finally, we validate our approach in a real-world application. Specifically, we aim to automatically learn the coordination of multiple drones having different objectives, harnessing the MOMARL framework to orchestrate their actions effectively. This empirical validation serves as evidence of the viability and versatility of the proposed methodologies in addressing complex real-world challenges