Constrained Multi-objective Optimization with Deep Reinforcement Learning Assisted Operator Selection

Solving constrained multi-objective optimization problems with evolutionary algorithms has attracted considerable attention. Various constrained multi-objective optimization evolutionary algorithms (CMOEAs) have been developed with the use of different algorithmic strategies, evolutionary operators, and constraint-handling techniques. The performance of CMOEAs may be heavily dependent on the operators used, however, it is usually difficult to select suitable operators for the problem at hand. Hence, improving operator selection is promising and necessary for CMOEAs. This work proposes an online operator selection framework assisted by Deep Reinforcement Learning. The dynamics of the population, including convergence, diversity, and feasibility, are regarded as the state; the candidate operators are considered as actions; and the improvement of the population state is treated as the reward. By using a Q-Network to learn a policy to estimate the Q-values of all actions, the proposed approach can adaptively select an operator that maximizes the improvement of the population according to the current state and thereby improve the algorithmic performance. The framework is embedded into four popular CMOEAs and assessed on 42 benchmark problems. The experimental results reveal that the proposed Deep Reinforcement Learning-assisted operator selection significantly improves the performance of these CMOEAs and the resulting algorithm obtains better versatility compared to nine state-of-the-art CMOEAs

Metodologia para alocação de filtros para mitigação de distorção harmônica em sistemas de distribuição

Este trabalho propõe uma metodologia para definição dos locais de instalação de filtros passivos dentro de um sistema de distribuição no intuito de mitigar as distorções harmônicas. A alocação de filtros em um sistema elétrico requer atenção, pois, dependendo da escolha do local para instalação deles, existe a possibilidade de deterioração da Qualidade de Energia Elétrica (QEE) do sistema, que ocorre com o aumento de distorção harmônica em outros pontos devido à modificação do fluxo de corrente no sistema. A presente pesquisa propõe otimizar a instalação dos filtros em Sistemas Elétricos de Potência (SEP), utilizando a mínima quantidade dos mesmos, visando melhorar a QEE de forma sistêmica com o menor custo possível. A abordagem proposta nesta tese traz uma maneira não usual de alocação dos filtros passivos; onde estes são alocados de forma monofásica ao invés de trifásica. Assim, filtros sintonizados podem ser instalados em apenas uma fase em um determinado ramo do sistema. Devido ao comportamento intrínseco em muitos sistemas de distribuição, desequilibrado, optou-se por esta proposta nesta pesquisa. O procedimento realizado é baseado na interação de dois programas: a modelagem e simulação do SEP é feita no Alternative Transient Program (ATP), que simula o fluxo harmônico do sistema, e em conjunto com o Matrix Laboratory (MatLab©), são verificados os índices de QEE referentes a distorções harmônicas e então realizada a otimização de alocação de filtros. A solução é obtida através de um algoritmo de otimização multiobjetivo, o Non-dominated Sorting Genetic Algorithm II (NSGA II), que trabalha com o conceito de dominância. Como típico em problemas com multiobjetivo, os objetivos alvos deste problema são conflitantes, as duas funções objetivo do problema visam: diminuição dos indicadores de distorção harmônica e diminuição do custo dos filtros a serem alocados. A solução traz uma gama de opções dentro de uma Fronteira de Pareto, que são consideradas igualmente boas entre si. Este estudo mostrou que a instalação de filtros monofásicos em sistemas de distribuição com presença de ramos monofásicos ou bifásicos é uma boa forma de mitigação de distorções harmônicas sistêmica quando o sistema é poluído de forma dispersa, conseguindo, ainda, compensar a potência reativa do SEP. Com a inserção dos filtros especificamente em ramos monofásicos ou bifásicos em sistemas elétricos há diminuição no custo total dos filtros alocados para a resolução do problema de distorções harmônicas.This study proposes a methodology for passive filters allocation within an electric distribution system aiming to reduce system harmonic distortion. Allocation of filters in an electrical system requires attention; depending of the choice of filters allocation might harm Power Quality (PQ) increasing harmonic distortion at other nodes due to the modification of the harmonic current flow in the system. Thus, the present research aims to find the optimal filters installation places using the least amount of them, aiming to improve PQ systemically. The approach proposed in this thesis brings an unusual way of allocating passive filters; they are allocated in a single-phase mode instead of three-phase, as is the classic way. So, tuned filters can be installed in one phase only on one branch of the system. Due to the characteristic of the distribution systems, unbalanced loads and lines, it was decided to follow this path in this research, in order to verify the effectiveness. The methodology is based on the interaction of two programs: the modeling and simulation of the electric power system in the Alternative Transient Program (ATP), which simulates the harmonic flow of the system and with the Matrix Laboratory (MatLab©) the PQ indexes for harmonic distortions are verified and then the optimization of filter allocation is carried out. The solution is executed through a multiobjective optimization algorithm, the Non-dominated Sorting Genetic Algorithm II (NSGA II), which uses the dominance concept. As typical in multiobjective problems, the objectives are conflicting, the objective functions for the problem are: minimize the number of node/phases that exceed harmonic distortion limits and minimize the cost of the filters to be allocated. The solution brings a range of options within a Pareto frontier, which are considered equally good with each other. This study showed that the installation of single-phase tuned filters in distribution systems with the presence of single-phase or two-phase circuits is a good way of mitigating harmonic distortions systemic when the system is polluted by harmonics in dispersed mode, furthermore getting improving the reactive power of the system. With the insertion of the filters specifically in single-phase and two-phase circuits, it will to decrease the investment of the total filters allocated to solve the problem of harmonic distortions

Constrained Multi-Objective Optimization With Deep Reinforcement Learning Assisted Operator Selection

Ming F, Gong W, Wang L, Jin Y. Constrained Multi-Objective Optimization With Deep Reinforcement Learning Assisted Operator Selection. IEEE/CAA Journal of Automatica Sinica. 2024;11(4):919-931.Solving constrained multi-objective optimization problems with evolutionary algorithms has attracted considerable attention. Various constrained multi-objective optimization evolutionary algorithms (CMOEAs) have been developed with the use of different algorithmic strategies, evolutionary operators, and constraint-handling techniques. The performance of CMOEAs may be heavily dependent on the operators used, however, it is usually difficult to select suitable operators for the problem at hand. Hence, improving operator selection is promising and necessary for CMOEAs. This work proposes an online operator selection framework assisted by Deep Reinforcement Learning. The dynamics of the population, including convergence, diversity, and feasibility, are regarded as the state; the candidate operators are considered as actions; and the improvement of the population state is treated as the reward. By using a Q-network to learn a policy to estimate the Q-values of all actions, the proposed approach can adaptively select an operator that maximizes the improvement of the population according to the current state and thereby improve the algorithmic performance. The framework is embedded into four popular CMOEAs and assessed on 42 benchmark problems. The experimental results reveal that the proposed Deep Reinforcement Learning-assisted operator selection significantly improves the performance of these CMOEAs and the resulting algorithm obtains better versatility compared to nine state-of-the-art CMOEAs