Adapting Swarm Intelligence For The Self-Assembly And Optimization Of Networks

While self-assembly is a fairly active area of research in swarm intelligence and robotics, relatively little attention has been paid to the issues surrounding the construction of network structures. Here, methods developed previously for modeling and controlling the collective movements of groups of agents are extended to serve as the basis for self-assembly or "growth" of networks, using neural networks as a concrete application to evaluate this novel approach. One of the central innovations incorporated into the model presented here is having network connections arise as persistent "trails" left behind moving agents, trails that are reminiscent of pheromone deposits made by agents in ant colony optimization models. The resulting network connections are thus essentially a record of agent movements. The model's effectiveness is demonstrated by using it to produce two large networks that support subsequent learning of topographic and feature maps. Improvements produced by the incorporation of collective movements are also examined through computational experiments. These results indicate that methods for directing collective movements can be extended to support and facilitate network self-assembly. Additionally, the traditional self-assembly problem is extended to include the generation of network structures based on optimality criteria, rather than on target structures that are specified a priori. It is demonstrated that endowing the network components involved in the self-assembly process with the ability to engage in collective movements can be an effective means of generating computationally optimal network structures. This is confirmed on a number of challenging test problems from the domains of trajectory generation, time-series forecasting, and control. Further, this extension of the model is used to illuminate an important relationship between particle swarm optimization, which usually occurs in high dimensional abstract spaces, and self-assembly, which is normally grounded in real and simulated 2D and 3D physical spaces

Metaheuristic design of feedforward neural networks: a review of two decades of research

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era

Artificial Neurogenesis: An Introduction and Selective Review

International audienceIn this introduction and review—like in the book which follows—we explore the hypothesis that adaptive growth is a means of producing brain-like machines. The emulation of neural development can incorporate desirable characteristics of natural neural systems into engineered designs. The introduction begins with a review of neural development and neural models. Next, artificial development— the use of a developmentally-inspired stage in engineering design—is introduced. Several strategies for performing this " meta-design " for artificial neural systems are reviewed. This work is divided into three main categories: bio-inspired representations ; developmental systems; and epigenetic simulations. Several specific network biases and their benefits to neural network design are identified in these contexts. In particular, several recent studies show a strong synergy, sometimes interchange-ability, between developmental and epigenetic processes—a topic that has remained largely under-explored in the literature

Neural Networks: Training and Application to Nonlinear System Identification and Control

This dissertation investigates training neural networks for system identification and classification. The research contains two main contributions as follow:1. Reducing number of hidden layer nodes using a feedforward componentThis research reduces the number of hidden layer nodes and training time of neural networks to make them more suited to online identification and control applications by adding a parallel feedforward component. Implementing the feedforward component with a wavelet neural network and an echo state network provides good models for nonlinear systems.The wavelet neural network with feedforward component along with model predictive controller can reliably identify and control a seismically isolated structure during earthquake. The network model provides the predictions for model predictive control. Simulations of a 5-story seismically isolated structure with conventional lead-rubber bearings showed significant reductions of all response amplitudes for both near-field (pulse) and far-field ground motions, including reduced deformations along with corresponding reduction in acceleration response. The controller effectively regulated the apparent stiffness at the isolation level. The approach is also applied to the online identification and control of an unmanned vehicle. Lyapunov theory is used to prove the stability of the wavelet neural network and the model predictive controller. 2. Training neural networks using trajectory based optimization approachesTraining neural networks is a nonlinear non-convex optimization problem to determine the weights of the neural network. Traditional training algorithms can be inefficient and can get trapped in local minima. Two global optimization approaches are adapted to train neural networks and avoid the local minima problem. Lyapunov theory is used to prove the stability of the proposed methodology and its convergence in the presence of measurement errors. The first approach transforms the constraint satisfaction problem into unconstrained optimization. The constraints define a quotient gradient system (QGS) whose stable equilibrium points are local minima of the unconstrained optimization. The QGS is integrated to determine local minima and the local minimum with the best generalization performance is chosen as the optimal solution. The second approach uses the QGS together with a projected gradient system (PGS). The PGS is a nonlinear dynamical system, defined based on the optimization problem that searches the components of the feasible region for solutions. Lyapunov theory is used to prove the stability of PGS and QGS and their stability under presence of measurement noise

Artificial Intelligence for Resilience in Smart Grid Operations

Today, the electric power grid is transforming into a highly interconnected network of advanced technologies, equipment, and controls to enable a smarter grid. The growing complexity of smart grid requires resilient operation and control. Power system resilience is defined as the ability to harden the system against and quickly recover from high-impact, low-frequency events. The introduction of two-way flows of information and electricity in the smart grid raises concerns of cyber-physical attacks. Proliferated penetration of renewable energy sources such as solar photovoltaic (PV) and wind power introduce challenges due to the high variability and uncertainty in generation. Unintentional disruptions and power system component outages have become a threat to real-time power system operations. Recent extreme weather events and natural disasters such as hurricanes, storms, and wildfires demonstrate the importance of resilience in the power system. It is essential to find solutions to overcome these challenges in maintaining resilience in smart grid. In this dissertation, artificial intelligence (AI) based approaches have been developed to enhance resilience in smart grid. Methods for optimal automatic generation control (AGC) have been developed for multi-area multi-machine power systems. Reliable AI models have been developed for predicting solar irradiance, PV power generation, and power system frequencies. The proposed short-horizon AI prediction models ranging from few seconds to a minute plus, outperform the state-of-art persistence models. The AI prediction models have been applied to provide situational intelligence for power system operations. An enhanced tie-line bias control in a multi-area power system for variable and uncertain environments has been developed with predicted PV power and bus frequencies. A distributed and parallel security-constrained optimal power flow (SCOPF) algorithm has been developed to overcome the challenges in solving SCOPF problem for large power networks. The methods have been developed and tested on an experimental laboratory platform consisting of real-time digital simulators, hardware/software phasor measurement units, and a real-time weather station

Evolutionary optimization of sparsely connected and time-lagged neural networks for time series forecasting

Time Series Forecasting (TSF) is an important tool to support decision mak- ing (e.g., planning production resources). Artificial Neural Networks (ANN) are innate candidates for TSF due to advantages such as nonlinear learn- ing and noise tolerance. However, the search for the best model is a complex task that highly affects the forecasting performance. In this work, we propose two novel Evolutionary Artificial Neural Networks (EANN) approaches for TSF based on an Estimation Distribution Algorithm (EDA) search engine. The first new approach consist of Sparsely connected Evolutionary ANN (SEANN), which evolves more flexible ANN structures to perform multi-step ahead forecasts. The second one, consists of an automatic Time lag feature selection EANN (TEANN) approach that evolves not only ANN parameters (e.g., input and hidden nodes, training parameters) but also which set of time lags are fed into the forecasting model. Several experiments were held, using a set of six time series, from different real-world domains. Also, two error metrics (i.e., Mean Squared Error and Symmetric Mean Absolute Per- centage Error) were analyzed. The two EANN approaches were compared against a base EANN (with no ANN structure or time lag optimization) and four other methods (Autoregressive Integrated Moving Average method, Random Forest, Echo State Network and Support Vector Machine). Overall, the proposed SEANN and TEANN methods obtained the best forecasting results. Moreover, they favor simpler neural network models, thus requiring less computational effort when compared with the base EANN.The research reported here has been supported by the Spanish Ministry of Science and Innovation under project TRA2010-21371-C03-03 and FCT - Fundacao para a Ciencia e Tecnologia within the Project Scope PEst- OE/EEI/UI0319/2014. The authors want to thank specially Martin Stepnicka and Lenka Vavrickova for all their help. The authors also want to thank Ramon Sagarna for introducing the subject of EDA

Odhad emocí a duševní koncentrace pomocí technik Deep Learningu

The purpose of this work is to evaluate the brain waves of humans with deep learn- ing methods and evolutionary computation techniques, and to verify the performance of applied techniques. In this thesis, we apply well–known metaheuristics and Artificial Neural Networks for classifying human mental activities using electroencephalographic signals. We developed a Brain–Computer Interface system that is able to process elec- troencephalographic signals and classify mental concentration versus relaxation. The system is able to automatically extract and learn representation of the given data. Based on scientific protocols we designed the Brain–Computer Interface experiments and we created an original and relevant data for the industrial and academic community. Our experimental data is available to the scientific community. In the experiments we used an electroencephalographic based device for collecting brain information form the subjects during specific activities. The collected data represents brain waves of subjects who was stimulated by writing tasks. Furthermore, we selected the best combination of the input features (brain waves information) using the following two metaheuristic techniques: Simulated Annealing and Geometric Particle Swarm Optimization. We applied a specific type of Artificial Neural Network, named Echo State Network, for solving the mapping between brain information and subject activities. The results indicate that it is possible to estimate the human con- centration using few electroencephalographic signals. In addition, the proposed system is developed with a fast and robust learning technique that can be easily adapted accord- ing to each subject. Moreover, this approach does not require powerful computational resources. As a consequence, the proposed system can be used in environments which are computationally limited and/or where the computational time is an important issue.Cílem práce je ohodnocení lidských mozkových vln s využitím metod hlubokého učení (deep learning) a evolučních výpočetních technik a pro ověření výkonu aplikovaných technik. V diplomové práci jsou využity dobře známé metaheuristiky a umělé neuronové sítě pro klasifikaci lidských mentálních aktivit za použití elektroencefalografických signálů. Bylo vyvinuto rozhraní mozek-počítač, které je schopno zpracovat elektroencefalografické signály a klasifikovat mentální soustředění v porovnání s relaxací. Systém je schopen automaticky extrahovat a naučit se reprezentaci daných dat. Na základě vědeckých protokolů byl navržen experiment pro rozhraní mozek-počítač a byla vytvořena původní a relevantní data pro průmyslovou a akademickou komunitu. Vygenerovaná pokusná data jsou přístupné pro vědeckou komunitu. V rámci experimentů bylo využito zařízení založené na encefalografii pro sběr mozkových signálů subjektu během specifických aktivit. Nasbíraná data reprezentují mozkové vlny subjektu, který byl stimulován psaním úloh. Dále byla vybrána nejlepší kombinace vstupních vlastností (informace o mozkové vlně) s využitím následujících dvou metaheuristických metod: simulovaného žíhání a geometrické optimalizace hejnem částic. Umělá neuronová síť, která se nazývá Echo State síť, byla aplikována pro řešení mapování mezi informacemi z mozku a aktivitami subjektu. Výsledky ukazují, že je možné odhadnout lidskou aktivitu pomocí několika encefalografických signálů. Kromě toho, navrhovaný systém je vyvinut s využitím rychlých a robustních učících technik, které mohou být jednoduše přizpůsobeny podle jednotlivých subjektů. Tento přístup navíc nevyžaduje výkonné výpočetní prostředky. V důsledku toho může být systém využit v prostředí, které jsou výpočetně omezeny a/nebo v případech, kdy výpočetní čas je důležitým hlediskem.460 - Katedra informatikyvýborn