Modified Cellular Simultaneous Recurrent Networks with Cellular Particle Swarm Optimization

A cellular simultaneous recurrent network (CSRN) [1-11] is a neural network architecture that uses conventional simultaneous recurrent networks (SRNs), or cells in a cellular structure. The cellular structure adds complexity, so the training of CSRNs is far more challenging than that of conventional SRNs. Computer Go serves as an excellent test bed for CSRNs because of its clear-cut objective. For the training data, we developed an accurate theoretical foundation and game tree for the 2x2 game board. The conventional CSRN architecture suffers from the multi-valued function problem; our modified CSRN architecture overcomes the problem by employing ternary coding of the Go board\u27s representation and a normalized input dimension reduction. We demonstrate a 2x2 game tree trained with the proposed CSRN architecture and the proposed cellular particle swarm optimization

Spatial-temporal reasoning applications of computational intelligence in the game of Go and computer networks

Spatial-temporal reasoning is the ability to reason with spatial images or information about space over time. In this dissertation, computational intelligence techniques are applied to computer Go and computer network applications. Among four experiments, the first three are related to the game of Go, and the last one concerns the routing problem in computer networks. The first experiment represents the first training of a modified cellular simultaneous recurrent network (CSRN) trained with cellular particle swarm optimization (PSO). Another contribution is the development of a comprehensive theoretical study of a 2x2 Go research platform with a certified 5 dan Go expert. The proposed architecture successfully trains a 2x2 game tree. The contribution of the second experiment is the development of a computational intelligence algorithm calledcollective cooperative learning (CCL). CCL learns the group size of Go stones on a Go board with zero knowledge by communicating only with the immediate neighbors. An analysis determines the lower bound of a design parameter that guarantees a solution. The contribution of the third experiment is the proposal of a unified system architecture for a Go robot. A prototype Go robot is implemented for the first time in the literature. The last experiment tackles a disruption-tolerant routing problem for a network suffering from link disruption. This experiment represents the first time that the disruption-tolerant routing problem has been formulated with a Markov Decision Process. In addition, the packet delivery rate has been improved under a range of link disruption levels via a reinforcement learning approach --Abstract, page iv

Engine Data Classification with Simultaneous Recurrent Network using a Hybrid PSO-EA Algorithm

We applied an architecture which automates the design of simultaneous recurrent network (SRN) using a new evolutionary learning algorithm. This new evolutionary learning algorithm is based on a hybrid of particle swarm optimization (PSO) and evolutionary algorithm (EA). By combining the searching abilities of these two global optimization methods, the evolution of individuals is no longer restricted to be in the same generation, and better performed individuals may produce offspring to replace those with poor performance. The novel algorithm is then applied to the simultaneous recurrent network for the engine data classification. The experimental results show that our approach gives solid performance in categorizing the nonlinear car engine data

Cellular Simultanous Recurrent Networks for Image Processing

Artificial neural networks are inspired by the abilities of humans and animals to learn and adapt. Feed-forward networks are both fast and powerful, and are particularly useful for statistical pattern recognition. These networks are inspired by portions of the brain such as the visual cortex. However, feed-forward networks have been shown inadequate for complex applications such as long-term optimization, reinforced learning and image processing. Cellular Neural Networks (CNNs) are a type of recurrent network which have been used extensively for image processing. CNNs have shown limited success solving problems which involve topological relationships. Such problems include geometric transformations such as affine transformation and image registration. The Cellular Simultaneous Recurrent Network (CSRN) has been exploited to solve the 2D maze traversal problem, which is a long-term optimization problem with similar topological relations. From its inception, it has been speculated that the CSRN may have important implications in image processing. However, to date, very little work has been done to study CSRNs for image processing tasks. In this work, we investigate CSRNs for image processing. We propose a novel, generalized architecture for the CSRN suitable for generic image processing tasks. This architecture includes the use of sub-image processing which greatly improves the efficacy of CSRNs for image processing. We demonstrate the application of the CSRN with this generalized architecture across a variety of image processing problems including pixel level transformations, filtering, and geometric transformations. Results are evaluated and compared with standard MATLAB® functions. To better understand the inner workings of the CSRN we investigate the use of various CSRN cores including: 1) the original Generalized Multi-Layered Perceptron (GMLP) core used by Pang and Werbos to solve the 2D maze traversal problem, 2) the Elman Simultaneous Recurrent Network (ESRN), and 3) a novel ESRN core with multi-layered feedback. We compare the functionality of these cores in image processing applications. Further, we introduce the application of the unscented Kalman filter (UKF) for training of the CSRN. Results are compared with the standard Extended Kalman Filter (EKF) training method of CSRN. Finally, implications of current findings and proposed research directions are presented

Adaptive dynamic programming with eligibility traces and complexity reduction of high-dimensional systems

This dissertation investigates the application of a variety of computational intelligence techniques, particularly clustering and adaptive dynamic programming (ADP) designs especially heuristic dynamic programming (HDP) and dual heuristic programming (DHP). Moreover, a one-step temporal-difference (TD(0)) and n-step TD (TD(λ)) with their gradients are utilized as learning algorithms to train and online-adapt the families of ADP. The dissertation is organized into seven papers. The first paper demonstrates the robustness of model order reduction (MOR) for simulating complex dynamical systems. Agglomerative hierarchical clustering based on performance evaluation is introduced for MOR. This method computes the reduced order denominator of the transfer function by clustering system poles in a hierarchical dendrogram. Several numerical examples of reducing techniques are taken from the literature to compare with our work. In the second paper, a HDP is combined with the Dyna algorithm for path planning. The third paper uses DHP with an eligibility trace parameter (λ) to track a reference trajectory under uncertainties for a nonholonomic mobile robot by using a first-order Sugeno fuzzy neural network structure for the critic and actor networks. In the fourth and fifth papers, a stability analysis for a model-free action-dependent HDP(λ) is demonstrated with batch- and online-implementation learning, respectively. The sixth work combines two different gradient prediction levels of critic networks. In this work, we provide a convergence proofs. The seventh paper develops a two-hybrid recurrent fuzzy neural network structures for both critic and actor networks. They use a novel n-step gradient temporal-difference (gradient of TD(λ)) of an advanced ADP algorithm called value-gradient learning (VGL(λ)), and convergence proofs are given. Furthermore, the seventh paper is the first to combine the single network adaptive critic with VGL(λ). --Abstract, page iv

Training Winner-Take-All Simultaneous Recurrent Neural Networks

The winner-take-all (WTA) network is useful in database management, very large scale integration (VLSI) design, and digital processing. The synthesis procedure of WTA on single-layer fully connected architecture with sigmoid transfer function is still not fully explored. We discuss the use of simultaneous recurrent networks (SRNs) trained by Kalman filter algorithms for the task of finding the maximum among N numbers. The simulation demonstrates the effectiveness of our training approach under conditions of a shared-weight SRN architecture. A more general SRN also succeeds in solving a real classification application on car engine data

Evolutionary Algorithms, Markov Decision Processes, Adaptive Critic Designs, and Clustering: Commonalities, Hybridization and Performance

We briefly review and compare the mathematical formulation of Markov decision processes (MDP) and evolutionary algorithms (EA). In so doing, we observe that the adaptive critic design (ACD) approach to MDP can be viewed as a special form of EA. This leads us to pose pertinent questions about possible expansions of the methodology of ACD. This expansive view of EA is not limited to ACD. We discuss how it is possible to consider the powerful chained Lin Kernighan (chained LK) algorithm for the traveling salesman problem (TSP) as a degenerate case of EA. Finally, we review some recent TSP results, using clustering to divide-and-conquer, that provide superior speed and scalability

Path planning for general mazes

Path planning is used in, but not limited to robotics, telemetry, aerospace, and medical applications. The goal of the path planning is to identify a route from an origination point to a destination point while avoiding obstacles. This path might not always be the shortest in distance as time, terrain, speed limits, and many other factors can affect the optimality of the path. However, in this thesis, the length, computational time, and the smoothness of the path are the only constraints that will be considered with the length of the path being the most important. There are a variety of algorithms that can be used for path planning but Ant Colony Optimization (ACO), Neural Network, and A* will be the only algorithms explored in this thesis. The problem of solving general mazes has been greatly researched, but the contributions of this thesis extended Ant Colony Optimization to path planning for mazes, created a new landscape for the Neural Network to use, and added a bird\u27s eye view to the A* Algorithm. The Ant Colony Optimization that was used in this thesis was able to discover a path to the goal, but it was jagged and required a larger computational time compared to the Neural Network and A* algorithm discussed in this thesis. The Hopfield-type neural network used in this thesis propagated energy to create a landscape and used gradient decent to find the shortest path in terms of distance, but this thesis modified how the landscape was created to prevent the neural network from getting trapped in local minimas. The last contribution was applying a bird\u27s eye view to the A* algorithm to learn more about the environment which helped to create shorter and smoother paths