Parallelized Interactive Machine Learning on Autonomous Vehicles

Deep reinforcement learning (deep RL) has achieved superior performance in complex sequential tasks by learning directly from image input. A deep neural network is used as a function approximator and requires no specific state information. However, one drawback of using only images as input is that this approach requires a prohibitively large amount of training time and data for the model to learn the state feature representation and approach reasonable performance. This is not feasible in real-world applications, especially when the data are expansive and training phase could introduce disasters that affect human safety. In this work, we use a human demonstration approach to speed up training for learning features and use the resulting pre-trained model to replace the neural network in the deep RL Deep Q-Network (DQN), followed by human interaction to further refine the model. We empirically evaluate our approach by using only a human demonstration model and modified DQN with human demonstration model included in the Microsoft AirSim car simulator. Our results show that (1) pre-training with human demonstration in a supervised learning approach is better and much faster at discovering features than DQN alone, (2) initializing the DQN with a pre-trained model provides a significant improvement in training time and performance even with limited human demonstration, and (3) providing the ability for humans to supply suggestions during DQN training can speed up the network's convergence on an optimal policy, as well as allow it to learn more complex policies that are harder to discover by random exploration.Comment: 6 pages, NAECON 2018 - IEEE National Aerospace and Electronics Conferenc

Simultaneous Model Selection and Optimization through Parameter-free Stochastic Learning

Stochastic gradient descent algorithms for training linear and kernel predictors are gaining more and more importance, thanks to their scalability. While various methods have been proposed to speed up their convergence, the model selection phase is often ignored. In fact, in theoretical works most of the time assumptions are made, for example, on the prior knowledge of the norm of the optimal solution, while in the practical world validation methods remain the only viable approach. In this paper, we propose a new kernel-based stochastic gradient descent algorithm that performs model selection while training, with no parameters to tune, nor any form of cross-validation. The algorithm builds on recent advancement in online learning theory for unconstrained settings, to estimate over time the right regularization in a data-dependent way. Optimal rates of convergence are proved under standard smoothness assumptions on the target function, using the range space of the fractional integral operator associated with the kernel

Fast k-means algorithm clustering

k-means has recently been recognized as one of the best algorithms for clustering unsupervised data. Since k-means depends mainly on distance calculation between all data points and the centers, the time cost will be high when the size of the dataset is large (for example more than 500millions of points). We propose a two stage algorithm to reduce the time cost of distance calculation for huge datasets. The first stage is a fast distance calculation using only a small portion of the data to produce the best possible location of the centers. The second stage is a slow distance calculation in which the initial centers used are taken from the first stage. The fast and slow stages represent the speed of the movement of the centers. In the slow stage, the whole dataset can be used to get the exact location of the centers. The time cost of the distance calculation for the fast stage is very low due to the small size of the training data chosen. The time cost of the distance calculation for the slow stage is also minimized due to small number of iterations. Different initial locations of the clusters have been used during the test of the proposed algorithms. For large datasets, experiments show that the 2-stage clustering method achieves better speed-up (1-9 times).Comment: 16 pages, Wimo2011; International Journal of Computer Networks & Communications (IJCNC) Vol.3, No.4, July 201

Efficient Training and Evaluation of Recurrent Neural Network Language Models for Automatic Speech Recognition

© 2014 IEEE. Recurrent neural network language models (RNNLMs) are becoming increasingly popular for a range of applications including automatic speech recognition. An important issue that limits their possible application areas is the computational cost incurred in training and evaluation. This paper describes a series of new efficiency improving approaches that allows RNNLMs to be more efficiently trained on graphics processing units (GPUs) and evaluated on CPUs. First, a modified RNNLM architecture with a nonclass-based, full output layer structure (F-RNNLM) is proposed. This modified architecture facilitates a novel spliced sentence bunch mode parallelization of F-RNNLM training using large quantities of data on a GPU. Second, two efficient RNNLM training criteria based on variance regularization and noise contrastive estimation are explored to specifically reduce the computation associated with the RNNLM output layer softmax normalisation term. Finally, a pipelined training algorithm utilizing multiple GPUs is also used to further improve the training speed. Initially, RNNLMs were trained on a moderate dataset with 20M words from a large vocabulary conversational telephone speech recognition task. The training time of RNNLM is reduced by up to a factor of 53 on a single GPU over the standard CPU-based RNNLM toolkit. A 56 times speed up in test time evaluation on a CPU was obtained over the baseline F-RNNLMs. Consistent improvements in both recognition accuracy and perplexity were also obtained over C-RNNLMs. Experiments on Google's one billion corpus also reveals that the training of RNNLM scales well

BAMBI: blind accelerated multimodal Bayesian inference

In this paper we present an algorithm for rapid Bayesian analysis that combines the benefits of nested sampling and artificial neural networks. The blind accelerated multimodal Bayesian inference (BAMBI) algorithm implements the MultiNest package for nested sampling as well as the training of an artificial neural network (NN) to learn the likelihood function. In the case of computationally expensive likelihoods, this allows the substitution of a much more rapid approximation in order to increase significantly the speed of the analysis. We begin by demonstrating, with a few toy examples, the ability of a NN to learn complicated likelihood surfaces. BAMBI's ability to decrease running time for Bayesian inference is then demonstrated in the context of estimating cosmological parameters from Wilkinson Microwave Anisotropy Probe and other observations. We show that valuable speed increases are achieved in addition to obtaining NNs trained on the likelihood functions for the different model and data combinations. These NNs can then be used for an even faster follow-up analysis using the same likelihood and different priors. This is a fully general algorithm that can be applied, without any pre-processing, to other problems with computationally expensive likelihood functions.Comment: 12 pages, 8 tables, 17 figures; accepted by MNRAS; v2 to reflect minor changes in published versio

Fast modelling of gas reservoirs using non-intrusive reduced order modelling and machine learning

This work focussed on developing approximate methods for rapidly estimating gas field production performance. Proper orthogonal decomposition (POD) - Radial basis function (RBF) and POD-Autoencoder (AE) Non Intrusive Reduced Order Models (NIROMs) were considered. The accuracy and speed of both NIROMs were evaluated for modelling different aspects of gas field modelling including reservoirs with time-varying and mixed production controls, reservoirs with and without aquifer pressure support, and for wells that were (or not ) shut-in during production lifecycle. These NIROMs were applied to predicting the performance of four gas reservoir models: a homogeneous synthetic model; a heterogeneous gas field with 3 wells and structures similar to the Norne Field; a water coning model in radian grid; and a sector model of a real gas field provided by Woodside Petroleum. The POD-RBF and POD-AE NIROMs were trained using the simulation solutions from a commercial reservoir simulator (ECLIPSE): grid distributions of pressure and saturations as well as time series production data such as production rates, cumulative productions and pressures. Different cases were run based on typical input parameters usually used in field performance studies. The simulation solutions were then standardised to zero mean and reduced into hyperspace using POD. In most cases, the optimum number of POD basis functions (99.9% energy criterion) of the solutions (training data) were used to reduce the training data into a lower-dimensional hyperspace space. The reduced training data and their corresponding parameter values were combined to form sample and response arrays based on a cause and effect pattern. RBF or AE was then used to interpolate the weighting coefficients that represented the dynamics of the gas reservoir as captured within the reduced training data. These weighting coefficients were used to propagate the prediction of new unseen simulation cases for the duration of predictions. The simulation results from either or both NIROMs was then compared against the simulation solution of the same cases in ECLIPSE. It was found that the POD-RBF is a better predictive tool for gas field modelling. It is faster, more accurate and consistent than the POD-AE, giving satisfactory predictions with up to 99% accuracy and 2 orders of magnitude speed-up. No single POD-AE is sufficient for predicting different production scenarios, besides, the process of arriving at a suitable POD-AE involves finetuning several hyper-parameters by trial and error, which may be more burdensome for practising petroleum engineers. The accuracy of NIROM’s prediction of production variable is generally improved by using more than the optimal number of POD-basis functions, while predictions of grid distributed properties are satisfactorily predicted with the optimal number of POD-basis functions. NIROM’s accuracy is dependent on whether the range of parameters of the prediction, their duration and specific production scenarios (such as having mixed production controls or aquifer pressure support) reflect those of the training cases. Overall, the number of training runs, the size of the reservoir model as well as the number of time intervals at which simulation output data is required all affect the speed of training both NIROMs for prediction. Other contributions of this work include showing that the linear RBF is the most suitable RBF for gas field modelling; developing a novel normalisation approach for time-varying parameters; and applying NIROMs to seasonally varying production scenarios with mixed production controls. This work is the first time that the POD-AE has been developed and evaluated for petroleum field development planning.Open Acces

Becoming a guitar hero: does it alter multisensory processing skills?

Three groups of novice gamers were trained for 10 hours using the music-genre game Rock Band©: one group played the game normally, another played using visual cues only, and a third simply listened to music. Pre- and post-test eye-tracking data was collected using a focused attention task in which participants quickly shifted their gaze toward a visual target; on some trials a to-be-ignored auditory tone was also presented. Past research has shown the tone to speed-up saccadic response time (SRT). We hypothesized that training on a music-genre video game would boost this intersensory facilitation effect, defined as the difference between SRTs on unimodal only trials minus SRTs on bimodal trials. There was an overall SRT decrease from pre- to post-test, but, more critically, the magnitude of the facilitation effect was not disproportionally enhanced in the full Rock Band© training group, relative to the controls. Future research avenues are considered.Master of Arts (M.A.) in Psycholog