A Study on Rotation Invariance in Differential Evolution

The file attached to this record is the author's final peer reviewed version. The Publisher's final version can be found by following the DOI link.Epistasis is the correlation between the variables of a function and is a challenge often posed by real-world optimisation problems. Synthetic benchmark problems simulate a highly epistatic problem by performing a so-called problem's rotation. Mutation in Differential Evolution (DE) is inherently rotational invariant since it simultaneously perturbs all the variables. On the other hand, crossover, albeit fundamental for achieving a good performance, retains some of the variables, thus being inadequate to tackle highly epistatic problems. This article proposes an extensive study on rotational invariant crossovers in DE. We propose an analysis of the literature, a taxonomy of the proposed method and an experimental setup where each problem is addressed in both its non-rotated and rotated version. Our experimental study includes $280$ problems over five different levels of dimensionality and nine algorithms. Numerical results show that 1) for a fixed quota of transferred design variables, the exponential crossover displays a better performance, on both rotated and non-rotated problems, in high dimensions while the binomial crossover seems to be preferable in low dimensions; 2) the rotational invariant mutation DE/current-to-rand is not competitive with standard DE implementations throughout the entire set of experiments we have presented; 3) DE crossovers that perform a change of coordinates to distribute the moves over the components of the offspring offer high-performance results on some problems. However, on average the standard DE/rand/1/exp appears to achieve the best performance on both rotated and non-rotated testbeds

The SOS Platform: Designing, Tuning and Statistically Benchmarking Optimisation Algorithms

open access articleWe present Stochastic Optimisation Software (SOS), a Java platform facilitating the algorithmic design process and the evaluation of metaheuristic optimisation algorithms. SOS reduces the burden of coding miscellaneous methods for dealing with several bothersome and time-demanding tasks such as parameter tuning, implementation of comparison algorithms and testbed problems, collecting and processing data to display results, measuring algorithmic overhead, etc. SOS provides numerous off-the-shelf methods including: (1) customised implementations of statistical tests, such as the Wilcoxon rank-sum test and the Holm–Bonferroni procedure, for comparing the performances of optimisation algorithms and automatically generating result tables in PDF and formats; (2) the implementation of an original advanced statistical routine for accurately comparing couples of stochastic optimisation algorithms; (3) the implementation of a novel testbed suite for continuous optimisation, derived from the IEEE CEC 2014 benchmark, allowing for controlled activation of the rotation on each testbed function. Moreover, we briefly comment on the current state of the literature in stochastic optimisation and highlight similarities shared by modern metaheuristics inspired by nature. We argue that the vast majority of these algorithms are simply a reformulation of the same methods and that metaheuristics for optimisation should be simply treated as stochastic processes with less emphasis on the inspiring metaphor behind them

The SOS Platform: Designing, Tuning and Statistically Benchmarking Optimisation Algorithms

We present Stochastic Optimisation Software (SOS), a Java platform facilitating the algorithmic design process and the evaluation of metaheuristic optimisation algorithms. SOS reduces the burden of coding miscellaneous methods for dealing with several bothersome and time-demanding tasks such as parameter tuning, implementation of comparison algorithms and testbed problems, collecting and processing data to display results, measuring algorithmic overhead, etc. SOS provides numerous off-the-shelf methods including: (1) customised implementations of statistical tests, such as the Wilcoxon rank-sum test and the Holm–Bonferroni procedure, for comparing the performances of optimisation algorithms and automatically generating result tables in PDF and LATEX formats; (2) the implementation of an original advanced statistical routine for accurately comparing couples of stochastic optimisation algorithms; (3) the implementation of a novel testbed suite for continuous optimisation, derived from the IEEE CEC 2014 benchmark, allowing for controlled activation of the rotation on each testbed function. Moreover, we briefly comment on the current state of the literature in stochastic optimisation and highlight similarities shared by modern metaheuristics inspired by nature. We argue that the vast majority of these algorithms are simply a reformulation of the same methods and that metaheuristics for optimisation should be simply treated as stochastic processes with less emphasis on the inspiring metaphor behind them

Rotationally invariant techniques for handling parameter interactions in evolutionary multi-objective optimization

In traditional optimization approaches the interaction of parameters associated with a problem is not a significant issue, but in the domain of Evolutionary Multi-Objective Optimization (EMOO) traditional genetic algorithm approaches have difficulties in optimizing problems with parameter interactions. Parameter interactions can be introduced when the search space is rotated. Genetic algorithms are referred to as being not rotationally invariant because their behavior changes depending on the orientation of the search space. Many empirical studies in single and multi-objective evolutionary optimization are done with respect to test problems which do not have parameter interactions. Such studies provide a favorably biased indication of genetic algorithm performance. This motivates the first aspect of our work; the improvement of the testing of EMOO algorithms with respect to the aforementioned difficulties that genetic algorithms experience in the presence of parameter interactions. To this end, we examine how EMOO algorithms can be assessed when problems are subject to an arbitrarily uniform degree of parameter interactions. We establish a theoretical basis for parameter interactions and how they can be measured. Furthermore, we ask the question of what difficulties a multi-objective genetic algorithm experiences on optimization problems exhibiting parameter interactions. We also ask how these difficulties can be overcome in order to efficiently find the Pareto-optimal front on such problems. Existing multi-objective test problems in the literature typically introduce parameter interactions by altering the fitness landscape, which is undesirable. We propose a new suite of test problems that exhibit parameter interactions through a rotation of the decision space, without altering the fitness landscape. In addition, we compare the performance of a number of recombination operators on these test problems. The second aspect of this work is concerned with developing an efficient multi-objective optimization algorithm which works well on problems with parameter interactions. We investigate how an evolutionary algorithm can be made more efficient on multi-objective problems with parameter interactions by developing four novel rotationally invariant differential evolution approaches. We also ask whether the proposed approaches are competitive in comparison with a state-of-the-art EMOO algorithm. We propose several differential evolution approaches incorporating directional information from the multi-objective search space in order to accelerate and direct the search. Experimental results indicate that dramatic improvements in efficiency can be achieved by directing the search towards points which are more dominant and more diverse. We also address the important issue of diversity loss in rotationally invariant vector-wise differential evolution. Being able to generate diverse solutions is critically important in order to avoid stagnation. In order to address this issue, one of the directed approaches that we examine incorporates a novel sampling scheme around better individuals in the search space. This variant is able to perform exceptionally well on the test problems with much less computational cost and scales to very high decision space dimensions even in the presence of parameter interactions

Symmetry-Adapted Machine Learning for Information Security

Symmetry-adapted machine learning has shown encouraging ability to mitigate the security risks in information and communication technology (ICT) systems. It is a subset of artificial intelligence (AI) that relies on the principles of processing future events by learning past events or historical data. The autonomous nature of symmetry-adapted machine learning supports effective data processing and analysis for security detection in ICT systems without the interference of human authorities. Many industries are developing machine-learning-adapted solutions to support security for smart hardware, distributed computing, and the cloud. In our Special Issue book, we focus on the deployment of symmetry-adapted machine learning for information security in various application areas. This security approach can support effective methods to handle the dynamic nature of security attacks by extraction and analysis of data to identify hidden patterns of data. The main topics of this Issue include malware classification, an intrusion detection system, image watermarking, color image watermarking, battlefield target aggregation behavior recognition model, IP camera, Internet of Things (IoT) security, service function chain, indoor positioning system, and crypto-analysis

Measurements, Models, Systems and Design

531 s.

Development of Novel Independent Component Analysis Techniques and their Applications

Real world problems very often provide minimum information regarding their causes. This is mainly due to the system complexities and noninvasive techniques employed by scientists and engineers to study such systems. Signal and image processing techniques used for analyzing such systems essentially tend to be blind. Earlier, training signal based techniques were used extensively for such analyses. But many times either these training signals are not practicable to be availed by the analyzer or become burden on the system itself. Hence blind signal/image processing techniques are becoming predominant in modern real time systems. In fact, blind signal processing has become a very important topic of research and development in many areas, especially biomedical engineering, medical imaging, speech enhancement, remote sensing, communication systems, exploration seismology, geophysics, econometrics, data mining, sensor networks etc. Blind Signal Processing has three major areas: Blind Signal Separation and Extraction, Independent Component Analysis (ICA) and Multichannel Blind Deconvolution and Equalization. ICA technique has also been typically applied to the other two areas mentioned above. Hence ICA research with its wide range of applications is quite interesting and has been taken up as the central domain of the present work