Porting concepts from DNNs back to GMMs

Deep neural networks (DNNs) have been shown to outperform Gaussian Mixture Models (GMM) on a variety of speech recognition benchmarks. In this paper we analyze the differences between the DNN and GMM modeling techniques and port the best ideas from the DNN-based modeling to a GMM-based system. By going both deep (multiple layers) and wide (multiple parallel sub-models) and by sharing model parameters, we are able to close the gap between the two modeling techniques on the TIMIT database. Since the 'deep' GMMs retain the maximum-likelihood trained Gaussians as first layer, advanced techniques such as speaker adaptation and model-based noise robustness can be readily incorporated. Regardless of their similarities, the DNNs and the deep GMMs still show a sufficient amount of complementarity to allow effective system combination

Deep learning for time series classification: a review

Time Series Classification (TSC) is an important and challenging problem in data mining. With the increase of time series data availability, hundreds of TSC algorithms have been proposed. Among these methods, only a few have considered Deep Neural Networks (DNNs) to perform this task. This is surprising as deep learning has seen very successful applications in the last years. DNNs have indeed revolutionized the field of computer vision especially with the advent of novel deeper architectures such as Residual and Convolutional Neural Networks. Apart from images, sequential data such as text and audio can also be processed with DNNs to reach state-of-the-art performance for document classification and speech recognition. In this article, we study the current state-of-the-art performance of deep learning algorithms for TSC by presenting an empirical study of the most recent DNN architectures for TSC. We give an overview of the most successful deep learning applications in various time series domains under a unified taxonomy of DNNs for TSC. We also provide an open source deep learning framework to the TSC community where we implemented each of the compared approaches and evaluated them on a univariate TSC benchmark (the UCR/UEA archive) and 12 multivariate time series datasets. By training 8,730 deep learning models on 97 time series datasets, we propose the most exhaustive study of DNNs for TSC to date.Comment: Accepted at Data Mining and Knowledge Discover

Surrogate Model Optimisation for PWR Fuel Management

Pressurised Water Reactor (PWR) fuel management is an operational problem for nuclear operators, requiring solutions on a regular basis throughout the life of the plant. A variety of conflicting factors and changing goals mean that fuel loading pattern design problems are multiobjective and, by design, have many input variables. This causes a combinatorial explosion, known as the ‘curse of dimensionality’, which makes these complex problems difficult to investigate. In this thesis, the method of surrogate model optimisation is adapted to PWR loading pattern generation. Surrogate models are developed based around three approaches: deep learning methods (convolutional neural networks and multi-layer perceptrons), the fission matrix and simulated quantum annealing. The models are used to predict core parameters of reactors in simplified optimisation scenarios for a microcore, a small modular reactor, and a ‘standard’ PWR. The experiments with deep learning models show that competitive results can be obtained for training sets using a much lower number of simulations than direct optimisation. Fission matrix experiments demonstrate the method to predict core parameters for the first time, with interesting preliminary results. Novel experiments using simulated quantum annealing demonstrate the technique is able to generate loading patterns by following heuristic rules and is suitable for application to custom optimisation hardware. The principal contribution of this work is to show that surrogate model optimisation can be used to augment fuel loading pattern optimisation, generating competitive results and providing enormous computational cost reduction and thus permitting more investigation within a given computational budget. These methods can also make use of new computational hardware such as neural chips and quantum annealers. The promising methods developed in this thesis thus provide candidate implementations that can bring the benefits of these innovations to the sphere of nuclear engineering

Study and Observation of the Variations of Accuracies for Handwritten Digits Recognition with Various Hidden Layers and Epochs using Neural Network Algorithm

In recent days, Artificial Neural Network (ANN) can be applied to a vast majority of fields including business, medicine, engineering, etc. The most popular areas where ANN is employed nowadays are pattern and sequence recognition, novelty detection, character recognition, regression analysis, speech recognition, image compression, stock market prediction, Electronic nose, security, loan applications, data processing, robotics, and control. The benefits associated with its broad applications leads to increasing popularity of ANN in the era of 21st Century. ANN confers many benefits such as organic learning, nonlinear data processing, fault tolerance, and self-repairing compared to other conventional approaches. The primary objective of this paper is to analyze the influence of the hidden layers of a neural network over the overall performance of the network. To demonstrate this influence, we applied neural network with different layers on the MNIST dataset. Also, another goal is to observe the variations of accuracies of ANN for different numbers of hidden layers and epochs and to compare and contrast among them.Comment: To be published in the 4th IEEE International Conference on Electrical Engineering and Information & Communication Technology (iCEEiCT 2018