Application of computational intelligence to explore and analyze system architecture and design alternatives

Systems Engineering involves the development or improvement of a system or process from effective need to a final value-added solution. Rapid advances in technology have led to development of sophisticated and complex sensor-enabled, remote, and highly networked cyber-technical systems. These complex modern systems present several challenges for systems engineers including: increased complexity associated with integration and emergent behavior, multiple and competing design metrics, and an expansive design parameter solution space. This research extends the existing knowledge base on multi-objective system design through the creation of a framework to explore and analyze system design alternatives employing computational intelligence. The first research contribution is a hybrid fuzzy-EA model that facilitates the exploration and analysis of possible SoS configurations. The second contribution is a hybrid neural network-EA in which the EA explores, analyzes, and evolves the neural network architecture and weights. The third contribution is a multi-objective EA that examines potential installation (i.e. system) infrastructure repair strategies. The final contribution is the introduction of a hierarchical multi-objective evolutionary algorithm (MOEA) framework with a feedback mechanism to evolve and simultaneously evaluate competing subsystem and system level performance objectives. Systems architects and engineers can utilize the frameworks and approaches developed in this research to more efficiently explore and analyze complex system design alternatives --Abstract, page iv

Multi-Objective Evolutionary Neural Network to Predict Graduation Success at the United States Military Academy

This paper presents an evolutionary neural network approach to classify student graduation status based upon selected academic, demographic, and other indicators. A pareto-based, multi-objective evolutionary algorithm utilizing the Strength Pareto Evolutionary Algorithm (SPEA2) fitness evaluation scheme simultaneously evolves connection weights and identifies the neural network topology using network complexity and classification accuracy as objective functions. A combined vector-matrix representation scheme and differential evolution recombination operators are employed. The model is trained, tested, and validated using 5100 student samples with data compiled from admissions records and institutional research databases. The inputs to the evolutionary neural network model are used to classify students as: graduates, late graduates, or non-graduates. Results of the hybrid method show higher mean classification rates (88%) than the current methodology (80%) with a potential savings of $130M. Additionally, the proposed method is more efficient in that a less complex neural network topology is identified by the algorithm

Evolutionary learning and global search for multi-optimal PID tuning rules

With the advances in microprocessor technology, control systems are widely seen not only in industry but now also in household appliances and consumer electronics. Among all control schemes developed so far, Proportional plus Integral plus Derivative (PID) control is the most widely adopted in practice. Today, more than 90% of industrial controllers have a built-in PID function. Their wide applications have stimulated and sustained the research and development of PID tuning techniques, patents, software packages and hardware modules. Due to parameter interaction and format variation, tuning a PID controller is not as straightforward as one would have anticipated. Therefore, designing speedy tuning rules should greatly reduce the burden on new installation and ‘time-to-market’ and should also enhance the competitive advantages of the PID system under offer. A multi-objective evolutionary algorithm (MOEA) would be an ideal candidate to conduct the learning and search for multi-objective PID tuning rules. A simple to implement MOEA, termed s-MOEA, is devised and compared with MOEAs developed elsewhere. Extensive study and analysis are performed on metrics for evaluating MOEA performance, so as to help with this comparison and development. As a result, a novel visualisation technique, termed “Distance and Distribution” (DD)” chart, is developed to overcome some of the limitations of existing metrics and visualisation techniques. The DD chart allows a user to view the comparison of multiple sets of high order non-dominated solutions in a two-dimensional space. The capability of DD chart is shown in the comparison process and it is shown to be a useful tool for gathering more in-depth information of an MOEA which is not possible in existing empirical studies. Truly multi-objective global PID tuning rules are then evolved as a result of interfacing the s-MOEA with closed-loop simulations under practical constraints. It takes into account multiple, and often conflicting, objectives such as steady-state accuracy and transient responsiveness against stability and overshoots, as well as tracking performance against load disturbance rejection. These evolved rules are compared against other tuning rules both offline on a set of well-recognised PID benchmark test systems and online on three laboratory systems of different dynamics and transport delays. The results show that the rules significantly outperform all existing tuning rules, with multi-criterion optimality. This is made possible as the evolved rules can cover a delay to time constant ratio from zero to infinity based on first-order plus delay plant models. For second-order plus delay plant models, they can also cover all possible dynamics found in practice

A Pareto Based Multi-Objective Evolutionary Algorithm Approach to Military Installation Rail Infrastructure Investment

Decision making for military railyard infrastructure is an inherently multi-objective problem, balancing cost versus capability. In this research, a Pareto-based Multi-Objective Evolutionary Algorithm is compared to a military rail inventory and decision support tool (RAILER). The problem is formulated as a multi-objective evolutionary algorithm in which the overall railyard condition is increased while decreasing cost to repair and maintain. A prioritization scheme for track maintenance is introduced that takes into account the volume of materials transported over the track and each rail segment’s primary purpose. Available repair options include repairing current 90 gauge rail, upgrade of rail segments to 115 gauge rail, and the swapping of rail removed during the upgrade. The proposed Multi-Objective Evolutionary Algorithm approach provides several advantages to the RAILER approach. The MOEA methodology allows decision makers to incorporate additional repair options beyond the current repair or do nothing options. It was found that many of the solutions identified by the evolutionary algorithm were both lower cost and provide a higher overall condition that those generated by DoD’s rail inventory and decision support system, RAILER. Additionally, the MOEA methodology generates lower cost, higher capability solutions when reduced sets of repair options are considered. The collection of non-dominated solutions provided by this technique gives decision makers increased flexibility and the ability to evaluate whether an additional cost repair solution is worth the increase in facility rail condition

What can we learn from multi-objective meta-optimization of Evolutionary Algorithms in continuous domains?

Properly configuring Evolutionary Algorithms (EAs) is a challenging task made difficult by many different details that affect EAs' performance, such as the properties of the fitness function, time and computational constraints, and many others. EAs' meta-optimization methods, in which a metaheuristic is used to tune the parameters of another (lower-level) metaheuristic which optimizes a given target function, most often rely on the optimization of a single property of the lower-level method. In this paper, we show that by using a multi-objective genetic algorithm to tune an EA, it is possible not only to find good parameter sets considering more objectives at the same time but also to derive generalizable results which can provide guidelines for designing EA-based applications. In particular, we present a general framework for multi-objective meta-optimization, to show that "going multi-objective" allows one to generate configurations that, besides optimally fitting an EA to a given problem, also perform well on previously unseen ones

Identification of nonlinear processes based on Wiener-Hammerstein models and heuristic optimization.

[ES] En muchos campos de la ingeniería los modelos matemáticos son utilizados para describir el comportamiento de los sistemas, procesos o fenómenos. Hoy en día, existen varias técnicas o métodos que pueden ser usadas para obtener estos modelos. Debido a su versatilidad y simplicidad, a menudo se prefieren los métodos de identificación de sistemas. Por lo general, estos métodos requieren la definición de una estructura y la estimación computacional de los parámetros que la componen utilizando un conjunto de procedimientos y mediciones de las señales de entrada y salida del sistema. En el contexto de la identificación de sistemas no lineales, un desafío importante es la selección de la estructura. En el caso de que el sistema a identificar presente una no linealidad de tipo estático, los modelos orientados a bloques, pueden ser útiles para definir adecuadamente una estructura. Sin embargo, el diseñador puede enfrentarse a cierto grado de incertidumbre al seleccionar el modelo orientado a bloques adecuado en concordancia con el sistema real. Además de este inconveniente, se debe tener en cuenta que la estimación de algunos modelos orientados a bloques no es sencilla, como es el caso de los modelos de Wiener-Hammerstein que consisten en un bloque NL en medio de dos subsistemas LTI. La presencia de dos subsistemas LTI en los modelos de Wiener-Hammerstein es lo que principalmente dificulta su estimación. Generalmente, el procedimiento de identificación comienza con la estimación de la dinámica lineal, y el principal desafío es dividir esta dinámica entre los dos bloques LTI. Por lo general, esto implica una alta interacción del usuario para desarrollar varios procedimientos, y el modelo final estimado depende principalmente de estas etapas previas. El objetivo de esta tesis es contribuir a la identificación de los modelos de Wiener-Hammerstein. Esta contribución se basa en la presentación de dos nuevos algoritmos para atender aspectos específicos que no han sido abordados en la identificación de este tipo de modelos. El primer algoritmo, denominado WH-EA, permite estimar todos los parámetros de un modelo de Wiener-Hammerstein con un solo procedimiento a partir de un modelo dinámico lineal. Con WH-EA, una buena estimación no depende de procedimientos intermedios ya que el algoritmo evolutivo simultáneamente busca la mejor distribución de la dinámica, ajusta con precisión la ubicación de los polos y los ceros y captura la no linealidad estática. Otra ventaja importante de este algoritmo es que bajo consideraciones específicas y utilizando una señal de excitación adecuada, es posible crear un enfoque unificado que permite también la identificación de los modelos de Wiener y Hammerstein, que son casos particulares del modelo de Wiener-Hammerstein cuando uno de sus bloques LTI carece de dinámica. Lo interesante de este enfoque unificado es que con un mismo algoritmo es posible identificar los modelos de Wiener, Hammerstein y Wiener-Hammerstein sin que el usuario especifique de antemano el tipo de estructura a identificar. El segundo algoritmo llamado WH-MOEA, permite abordar el problema de identificación como un Problema de Optimización Multiobjetivo (MOOP). Sobre la base de este algoritmo se presenta un nuevo enfoque para la identificación de los modelos de Wiener-Hammerstein considerando un compromiso entre la precisión alcanzada y la complejidad del modelo. Con este enfoque es posible comparar varios modelos con diferentes prestaciones incluyendo como un objetivo de identificación el número de parámetros que puede tener el modelo estimado. El aporte de este enfoque se sustenta en el hecho de que en muchos problemas de ingeniería los requisitos de diseño y las preferencias del usuario no siempre apuntan a la precisión del modelo como un único objetivo, sino que muchas veces la complejidad es también un factor predominante en la toma de decisiones.[CA] En molts camps de l'enginyeria els models matemàtics són utilitzats per a descriure el comportament dels sistemes, processos o fenòmens. Hui dia, existeixen diverses tècniques o mètodes que poden ser usades per a obtindre aquests models. A causa de la seua versatilitat i simplicitat, sovint es prefereixen els mètodes d'identificació de sistemes. En general, aquests mètodes requereixen la definició d'una estructura i l'estimació computacional dels paràmetres que la componen utilitzant un conjunt de procediments i mesuraments dels senyals d'entrada i eixida del sistema. En el context de la identificació de sistemes no lineals, un desafiament important és la selecció de l'estructura. En el cas que el sistema a identificar presente una no linealitat de tipus estàtic, els models orientats a blocs, poden ser útils per a definir adequadament una estructura. No obstant això, el dissenyador pot enfrontar-se a cert grau d'incertesa en seleccionar el model orientat a blocs adequat en concordança amb el sistema real. A més d'aquest inconvenient, s'ha de tindre en compte que l'estimació d'alguns models orientats a blocs no és senzilla, com és el cas dels models de Wiener-Hammerstein que consisteixen en un bloc NL enmig de dos subsistemes LTI. La presència de dos subsistemes LTI en els models de Wiener-Hammerstein és el que principalment dificulta la seua estimació. Generalment, el procediment d'identificació comença amb l'estimació de la dinàmica lineal, i el principal desafiament és dividir aquesta dinàmica entre els dos blocs LTI. En general, això implica una alta interacció de l'usuari per a desenvolupar diversos procediments, i el model final estimat depén principalment d'aquestes etapes prèvies. L'objectiu d'aquesta tesi és contribuir a la identificació dels models de Wiener-Hammerstein. Aquesta contribució es basa en la presentació de dos nous algorismes per a atendre aspectes específics que no han sigut adreçats en la identificació d'aquesta mena de models. El primer algorisme, denominat WH-EA (Algorisme Evolutiu per a la identificació de sistemes de Wiener-Hammerstein), permet estimar tots els paràmetres d'un model de Wiener-Hammerstein amb un sol procediment a partir d'un model dinàmic lineal. Amb WH-EA, una bona estimació no depén de procediments intermedis ja que l'algorisme evolutiu simultàniament busca la millor distribució de la dinàmica, afina la ubicació dels pols i els zeros i captura la no linealitat estàtica. Un altre avantatge important d'aquest algorisme és que sota consideracions específiques i utilitzant un senyal d'excitació adequada, és possible crear un enfocament unificat que permet també la identificació dels models de Wiener i Hammerstein, que són casos particulars del model de Wiener-Hammerstein quan un dels seus blocs LTI manca de dinàmica. L'interessant d'aquest enfocament unificat és que amb un mateix algorisme és possible identificar els models de Wiener, Hammerstein i Wiener-Hammerstein sense que l'usuari especifique per endavant el tipus d'estructura a identificar. El segon algorisme anomenat WH-MOEA (Algorisme evolutiu multi-objectiu per a la identificació de models de Wiener-Hammerstein), permet abordar el problema d'identificació com un Problema d'Optimització Multiobjectiu (MOOP). Sobre la base d'aquest algorisme es presenta un nou enfocament per a la identificació dels models de Wiener-Hammerstein considerant un compromís entre la precisió aconseguida i la complexitat del model. Amb aquest enfocament és possible comparar diversos models amb diferents prestacions incloent com un objectiu d'identificació el nombre de paràmetres que pot tindre el model estimat. L'aportació d'aquest enfocament se sustenta en el fet que en molts problemes d'enginyeria els requisits de disseny i les preferències de l'usuari no sempre apunten a la precisió del model com un únic objectiu, sinó que moltes vegades la complexitat és també un factor predominant en la presa de decisions.[EN] In several engineering fields, mathematical models are used to describe the behaviour of systems, processes or phenomena. Nowadays, there are several techniques or methods for obtaining mathematical models. Because of their versatility and simplicity, system identification methods are often preferred. Generally, systems identification methods require defining a structure and estimating computationally the parameters that make it up, using a set of procedures y measurements of the system's input and output signals. In the context of nonlinear system identification, a significant challenge is the structure selection. In the case that the system to be identified presents a static type of nonlinearity, block-oriented models can be useful to define a suitable structure. However, the designer may face a certain degree of uncertainty when selecting the block-oriented model in accordance with the real system. In addition to this inconvenience, the estimation of some block-oriented models is not an easy task, as is the case with the Wiener-Hammerstein models consisting of a NL block in the middle of two LTI subsystems. The presence of two LTI subsystems in the Wiener-Hammerstein models is what mainly makes their estimation difficult. Generally, the identification procedure begins with the estimation of the linear dynamics, and the main challenge is to split this dynamic between the two LTI block. Usually, this implies a high user interaction to develop several procedures, and the final model estimated mostly depends on these previous stages. The aim of this thesis is to contribute to the identification of the Wiener-Hammerstein models. This contribution is based on the presentation of two new algorithms to address specific aspects that have not been addressed in the identification of this type of model. The first algorithm, called WH-EA (An Evolutionary Algorithm for Wiener-Hammerstein System Identification), allows estimating all the parameters of a Wiener-Hammerstein model with a single procedure from a linear dynamic model. With WH-EA, a good estimate does not depend on intermediate procedures since the evolutionary algorithm looks for the best dynamic division, while the locations of the poles and zeros are fine-tuned, and nonlinearity is captured simultaneously. Another significant advantage of this algorithm is that under specific considerations and using a suitable excitation signal; it is possible to create a unified approach that also allows the identification of Wiener and Hammerstein models which are particular cases of the Wiener-Hammerstein model when one of its LTI blocks lacks dynamics. What is interesting about this unified approach is that with the same algorithm, it is possible to identify Wiener, Hammerstein, and Wiener-Hammerstein models without the user specifying in advance the type of structure to be identified. The second algorithm called WH-MOEA (Multi-objective Evolutionary Algorithm for Wiener-Hammerstein identification), allows to address the identification problem as a Multi-Objective Optimisation Problem (MOOP). Based on this algorithm, a new approach for the identification of Wiener-Hammerstein models is presented considering a compromise between the accuracy achieved and the model complexity. With this approach, it is possible to compare several models with different performances, including as an identification target the number of parameters that the estimated model may have. The contribution of this approach is based on the fact that in many engineering problems the design requirements and user's preferences do not always point to the accuracy of the model as a single objective, but many times the complexity is also a predominant factor in decision-making.Zambrano Abad, JC. (2021). Identification of nonlinear processes based on Wiener-Hammerstein models and heuristic optimization [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/171739TESI

Meta-heuristic algorithms in car engine design: a literature survey

Meta-heuristic algorithms are often inspired by natural phenomena, including the evolution of species in Darwinian natural selection theory, ant behaviors in biology, flock behaviors of some birds, and annealing in metallurgy. Due to their great potential in solving difficult optimization problems, meta-heuristic algorithms have found their way into automobile engine design. There are different optimization problems arising in different areas of car engine management including calibration, control system, fault diagnosis, and modeling. In this paper we review the state-of-the-art applications of different meta-heuristic algorithms in engine management systems. The review covers a wide range of research, including the application of meta-heuristic algorithms in engine calibration, optimizing engine control systems, engine fault diagnosis, and optimizing different parts of engines and modeling. The meta-heuristic algorithms reviewed in this paper include evolutionary algorithms, evolution strategy, evolutionary programming, genetic programming, differential evolution, estimation of distribution algorithm, ant colony optimization, particle swarm optimization, memetic algorithms, and artificial immune system

An Approach for the Generation of Multi-Objective Algorithms Applied to the Integration and Test Order Problem

Multi-Objective Evolutionary Algorithms (MOEAs) have been successfully applied to solve hard real software engineering problems. However, to choose and design a MOEA is considered a difficult task, since there are several parameters and components to be configured. These aspects directly impact the generated solutions and the performance of MOEAs. In this sense, this paper proposes an approach for the automatic generation of MOEAs applied to the Integration and Test Order (ITO) problem. Such a problem refers to the generation of optimal sequences of units for integration testing. The approach includes a set of parameters and components of different MOEAs, and is implemented with two design algorithms: Grammatical Evolution (GE) and Iterated Racing (irace). Evaluation results are presented, comparing the MOEAs generated by both design algorithms. Furthermore, the generated MOEAs are compared to two well-known MOEAs used in the literature to solve the ITO problem. Results show that the MOEAs generated with GE and irace perform similarly, and both outperform traditional MOEAs. The approach can reduce efforts spent to design and configure MOEAs, and serves as basis for implementing solutions to other software engineering problems

A comparative study of multi objective optimization algorithms for a cellular automata model

Cellular Automata (CA) models can represent dynamic systems which are discrete in space and time that reflects the effect of intrinsic parameters where individual events are considered to occur from randomness. A CA model of two agents' chemical kinetics has been optimized earlier using NSGA-II based on Evolutionary Algorithm (EA). But the stochastic nature of the CA model along with its high sensitivity on the model parameters requires extensive investigation using different optimization algorithms. For this purpose, in the current study, four more recently developed and popular optimization algorithms based on EA, called NSGA-IIr, NSGA-IIa, AbYSS and MOEA/D, have been considered for investigation based on various performance measuring parameters. The study also compares the performances of the algorithms for different computational efforts with an objective to minimize the required number of objective function evaluations. Simulation results and Friedman rank statistical test show NSGA-IIa and NSGA-IIr as the best choices to optimize the CA stochastic model across any number of objective function evaluations. Though the choice of optimization algorithm does not change with function evaluations, higher function evaluations improve the pseudo-pareto front for the CA optimization problem. Such results will facilitate the use of stochastic CA models to represent complex (bio)-chemical networks