Deep Cellular Recurrent Neural Architecture for Efficient Multidimensional Time-Series Data Processing

Efficient processing of time series data is a fundamental yet challenging problem in pattern recognition. Though recent developments in machine learning and deep learning have enabled remarkable improvements in processing large scale datasets in many application domains, most are designed and regulated to handle inputs that are static in time. Many real-world data, such as in biomedical, surveillance and security, financial, manufacturing and engineering applications, are rarely static in time, and demand models able to recognize patterns in both space and time. Current machine learning (ML) and deep learning (DL) models adapted for time series processing tend to grow in complexity and size to accommodate the additional dimensionality of time. Specifically, the biologically inspired learning based models known as artificial neural networks that have shown extraordinary success in pattern recognition, tend to grow prohibitively large and cumbersome in the presence of large scale multi-dimensional time series biomedical data such as EEG. Consequently, this work aims to develop representative ML and DL models for robust and efficient large scale time series processing. First, we design a novel ML pipeline with efficient feature engineering to process a large scale multi-channel scalp EEG dataset for automated detection of epileptic seizures. With the use of a sophisticated yet computationally efficient time-frequency analysis technique known as harmonic wavelet packet transform and an efficient self-similarity computation based on fractal dimension, we achieve state-of-the-art performance for automated seizure detection in EEG data. Subsequently, we investigate the development of a novel efficient deep recurrent learning model for large scale time series processing. For this, we first study the functionality and training of a biologically inspired neural network architecture known as cellular simultaneous recurrent neural network (CSRN). We obtain a generalization of this network for multiple topological image processing tasks and investigate the learning efficacy of the complex cellular architecture using several state-of-the-art training methods. Finally, we develop a novel deep cellular recurrent neural network (CDRNN) architecture based on the biologically inspired distributed processing used in CSRN for processing time series data. The proposed DCRNN leverages the cellular recurrent architecture to promote extensive weight sharing and efficient, individualized, synchronous processing of multi-source time series data. Experiments on a large scale multi-channel scalp EEG, and a machine fault detection dataset show that the proposed DCRNN offers state-of-the-art recognition performance while using substantially fewer trainable recurrent units

Cellular Simultanous Recurrent Networks for Image Processing

Artificial neural networks are inspired by the abilities of humans and animals to learn and adapt. Feed-forward networks are both fast and powerful, and are particularly useful for statistical pattern recognition. These networks are inspired by portions of the brain such as the visual cortex. However, feed-forward networks have been shown inadequate for complex applications such as long-term optimization, reinforced learning and image processing. Cellular Neural Networks (CNNs) are a type of recurrent network which have been used extensively for image processing. CNNs have shown limited success solving problems which involve topological relationships. Such problems include geometric transformations such as affine transformation and image registration. The Cellular Simultaneous Recurrent Network (CSRN) has been exploited to solve the 2D maze traversal problem, which is a long-term optimization problem with similar topological relations. From its inception, it has been speculated that the CSRN may have important implications in image processing. However, to date, very little work has been done to study CSRNs for image processing tasks. In this work, we investigate CSRNs for image processing. We propose a novel, generalized architecture for the CSRN suitable for generic image processing tasks. This architecture includes the use of sub-image processing which greatly improves the efficacy of CSRNs for image processing. We demonstrate the application of the CSRN with this generalized architecture across a variety of image processing problems including pixel level transformations, filtering, and geometric transformations. Results are evaluated and compared with standard MATLAB® functions. To better understand the inner workings of the CSRN we investigate the use of various CSRN cores including: 1) the original Generalized Multi-Layered Perceptron (GMLP) core used by Pang and Werbos to solve the 2D maze traversal problem, 2) the Elman Simultaneous Recurrent Network (ESRN), and 3) a novel ESRN core with multi-layered feedback. We compare the functionality of these cores in image processing applications. Further, we introduce the application of the unscented Kalman filter (UKF) for training of the CSRN. Results are compared with the standard Extended Kalman Filter (EKF) training method of CSRN. Finally, implications of current findings and proposed research directions are presented

Modified Cellular Simultaneous Recurrent Networks with Cellular Particle Swarm Optimization

A cellular simultaneous recurrent network (CSRN) [1-11] is a neural network architecture that uses conventional simultaneous recurrent networks (SRNs), or cells in a cellular structure. The cellular structure adds complexity, so the training of CSRNs is far more challenging than that of conventional SRNs. Computer Go serves as an excellent test bed for CSRNs because of its clear-cut objective. For the training data, we developed an accurate theoretical foundation and game tree for the 2x2 game board. The conventional CSRN architecture suffers from the multi-valued function problem; our modified CSRN architecture overcomes the problem by employing ternary coding of the Go board\u27s representation and a normalized input dimension reduction. We demonstrate a 2x2 game tree trained with the proposed CSRN architecture and the proposed cellular particle swarm optimization

Recurrent Neural Networks and Matrix Methods for Cognitive Radio Spectrum Prediction and Security

In this work, machine learning tools, including recurrent neural networks (RNNs), matrix completion, and non-negative matrix factorization (NMF), are used for cognitive radio problems. Specifically addressed are a missing data problem and a blind signal separation problem. A specialized RNN called Cellular Simultaneous Recurrent Network (CSRN), typically used in image processing applications, has been modified. The CRSN performs well for spatial spectrum prediction of radio signals with missing data. An algorithm called soft-impute for matrix completion used together with an RNN performs well for missing data problems in the radio spectrum time-frequency domain. Estimating missing spectrum data can improve cognitive radio efficiency. An NMF method called tuning pruning is used for blind source separation of radio signals in simulation. An NMF optimization technique using a geometric constraint is proposed to limit the solution space of blind signal separation. Both NMF methods are promising in addressing a security problem known as spectrum sensing data falsification attack

Spatial-temporal reasoning applications of computational intelligence in the game of Go and computer networks

Spatial-temporal reasoning is the ability to reason with spatial images or information about space over time. In this dissertation, computational intelligence techniques are applied to computer Go and computer network applications. Among four experiments, the first three are related to the game of Go, and the last one concerns the routing problem in computer networks. The first experiment represents the first training of a modified cellular simultaneous recurrent network (CSRN) trained with cellular particle swarm optimization (PSO). Another contribution is the development of a comprehensive theoretical study of a 2x2 Go research platform with a certified 5 dan Go expert. The proposed architecture successfully trains a 2x2 game tree. The contribution of the second experiment is the development of a computational intelligence algorithm calledcollective cooperative learning (CCL). CCL learns the group size of Go stones on a Go board with zero knowledge by communicating only with the immediate neighbors. An analysis determines the lower bound of a design parameter that guarantees a solution. The contribution of the third experiment is the proposal of a unified system architecture for a Go robot. A prototype Go robot is implemented for the first time in the literature. The last experiment tackles a disruption-tolerant routing problem for a network suffering from link disruption. This experiment represents the first time that the disruption-tolerant routing problem has been formulated with a Markov Decision Process. In addition, the packet delivery rate has been improved under a range of link disruption levels via a reinforcement learning approach --Abstract, page iv

Abstracts on Radio Direction Finding (1899 - 1995)

The files on this record represent the various databases that originally composed the CD-ROM issue of "Abstracts on Radio Direction Finding" database, which is now part of the Dudley Knox Library's Abstracts and Selected Full Text Documents on Radio Direction Finding (1899 - 1995) Collection. (See Calhoun record https://calhoun.nps.edu/handle/10945/57364 for further information on this collection and the bibliography). Due to issues of technological obsolescence preventing current and future audiences from accessing the bibliography, DKL exported and converted into the three files on this record the various databases contained in the CD-ROM. The contents of these files are: 1) RDFA_CompleteBibliography_xls.zip [RDFA_CompleteBibliography.xls: Metadata for the complete bibliography, in Excel 97-2003 Workbook format; RDFA_Glossary.xls: Glossary of terms, in Excel 97-2003 Workbookformat; RDFA_Biographies.xls: Biographies of leading figures, in Excel 97-2003 Workbook format]; 2) RDFA_CompleteBibliography_csv.zip [RDFA_CompleteBibliography.TXT: Metadata for the complete bibliography, in CSV format; RDFA_Glossary.TXT: Glossary of terms, in CSV format; RDFA_Biographies.TXT: Biographies of leading figures, in CSV format]; 3) RDFA_CompleteBibliography.pdf: A human readable display of the bibliographic data, as a means of double-checking any possible deviations due to conversion