Learning Opposites with Evolving Rules

The idea of opposition-based learning was introduced 10 years ago. Since then a noteworthy group of researchers has used some notions of oppositeness to improve existing optimization and learning algorithms. Among others, evolutionary algorithms, reinforcement agents, and neural networks have been reportedly extended into their opposition-based version to become faster and/or more accurate. However, most works still use a simple notion of opposites, namely linear (or type- I) opposition, that for each $x\in[a,b]$ assigns its opposite as $\breve{x}_I=a+b-x$. This, of course, is a very naive estimate of the actual or true (non-linear) opposite $\breve{x}_{II}$, which has been called type-II opposite in literature. In absence of any knowledge about a function $y=f(\mathbf{x})$ that we need to approximate, there seems to be no alternative to the naivety of type-I opposition if one intents to utilize oppositional concepts. But the question is if we can receive some level of accuracy increase and time savings by using the naive opposite estimate $\breve{x}_I$ according to all reports in literature, what would we be able to gain, in terms of even higher accuracies and more reduction in computational complexity, if we would generate and employ true opposites? This work introduces an approach to approximate type-II opposites using evolving fuzzy rules when we first perform opposition mining. We show with multiple examples that learning true opposites is possible when we mine the opposites from the training data to subsequently approximate $\breve{x}_{II}=f(\mathbf{x},y)$.Comment: Accepted for publication in The 2015 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2015), August 2-5, 2015, Istanbul, Turke

Opposition-Based Differential Evolution

Evolutionary algorithms (EAs) are well-established techniques to approach those problems which for the classical optimization methods are difficult to solve. Tackling problems with mixed-type of variables, many local optima, undifferentiable or non-analytical functions are some examples to highlight the outstanding capabilities of the evolutionary algorithms. Among the various kinds of evolutionary algorithms, differential evolution (DE) is well known for its effectiveness and robustness. Many comparative studies confirm that the DE outperforms many other optimizers. Finding more accurate solution(s), in a shorter period of time for complex black-box problems, is still the main goal of all evolutionary algorithms. The opposition concept, on the other hand, has a very old history in philosophy, set theory, politics, sociology, and physics. But, there has not been any opposition-based contribution to optimization. In this thesis, firstly, the opposition-based optimization (OBO) is constituted. Secondly, its advantages are formally supported by establishing mathematical proofs. Thirdly, the opposition-based acceleration schemes, including opposition-based population initialization and generation jumping, are proposed. Fourthly, DE is selected as a parent algorithm to verify the acceleration effects of proposed schemes. Finally, a comprehensive set of well-known complex benchmark functions is employed to experimentally compare and analyze the algorithms. Results confirm that opposition-based DE (ODE) performs better than its parent (DE), in terms of both convergence speed and solution quality. The main claim of this thesis is not defeating DE, its numerous versions, or other optimizers, but to introduce a new notion into nonlinear continuous optimization via innovative metaheuristics, namely the notion of opposition. Although, ODE has been compared with six other optimizers and outperforms them overall. Furthermore, both presented experimental and mathematical results conform with each other and demonstrate that opposite points are more beneficial than pure random points for black-box problems; this fundamental knowledge can serve to accelerate other machine learning approaches as well (such as reinforcement learning and neural networks). And perhaps in future, it could replace the pure randomness with random-opposition model when there is no a priori knowledge about the solution/problem. Although, all conducted experiments utilize DE as a parent algorithm, the proposed schemes are defined at the population level and, hence, have an inherent potential to be utilized for acceleration of other DE extensions or even other population-based algorithms, such as genetic algorithms (GAs). Like many other newly introduced concepts, ODE and the proposed opposition-based schemes still require further studies to fully unravel their benefits, weaknesses, and limitations

Learning Opposites Using Neural Networks

Many research works have successfully extended algorithms such as evolutionary algorithms, reinforcement agents and neural networks using "opposition-based learning" (OBL). Two types of the "opposites" have been defined in the literature, namely \textit{type-I} and \textit{type-II}. The former are linear in nature and applicable to the variable space, hence easy to calculate. On the other hand, type-II opposites capture the "oppositeness" in the output space. In fact, type-I opposites are considered a special case of type-II opposites where inputs and outputs have a linear relationship. However, in many real-world problems, inputs and outputs do in fact exhibit a nonlinear relationship. Therefore, type-II opposites are expected to be better in capturing the sense of "opposition" in terms of the input-output relation. In the absence of any knowledge about the problem at hand, there seems to be no intuitive way to calculate the type-II opposites. In this paper, we introduce an approach to learn type-II opposites from the given inputs and their outputs using the artificial neural networks (ANNs). We first perform \emph{opposition mining} on the sample data, and then use the mined data to learn the relationship between input $x$ and its opposite $\breve{x}$. We have validated our algorithm using various benchmark functions to compare it against an evolving fuzzy inference approach that has been recently introduced. The results show the better performance of a neural approach to learn the opposites. This will create new possibilities for integrating oppositional schemes within existing algorithms promising a potential increase in convergence speed and/or accuracy.Comment: To appear in proceedings of the 23rd International Conference on Pattern Recognition (ICPR 2016), Cancun, Mexico, December 201