Improving time efficiency of feedforward neural network learning

Feedforward neural networks have been widely studied and used in many applications in science and engineering. The training of this type of networks is mainly undertaken using the well-known backpropagation based learning algorithms. One major problem with this type of algorithms is the slow training convergence speed, which hinders their applications. In order to improve the training convergence speed of this type of algorithms, many researchers have developed different improvements and enhancements. However, the slow convergence problem has not been fully addressed. This thesis makes several contributions by proposing new backpropagation learning algorithms based on the terminal attractor concept to improve the existing backpropagation learning algorithms such as the gradient descent and Levenberg-Marquardt algorithms. These new algorithms enable fast convergence both at a distance from and in a close range of the ideal weights. In particular, a new fast convergence mechanism is proposed which is based on the fast terminal attractor concept. Comprehensive simulation studies are undertaken to demonstrate the effectiveness of the proposed backpropagataion algorithms with terminal attractors. Finally, three practical application cases of time series forecasting, character recognition and image interpolation are chosen to show the practicality and usefulness of the proposed learning algorithms with comprehensive comparative studies with existing algorithms

Change-based population coding

One standard interpretation of networks of cortical neurons is that they form dynamical attractors. Computations such as stimulus estimation are performed by mapping inputs to points on the networks’ attractive manifolds. These points represent population codes for the stimulus values. However, this standard interpretation is hard to reconcile with the observation that the firing rates of such neurons constantly change following presentation of stimuli. Furthermore, these population codes are not robust to both dynamical noise and synaptic noise and learning the corresponding weight matrices has never been demonstrated which seriously limits the extent of their application. In this thesis, we address this problem in the context of an invariant discrimination task. We suggest an alternative view, in which computations that are performed over the course of the transient evolution of a recurrently-connected network are read out by monitoring the change in a readily computed statistic of the activity of the network. Such changes can be inherently invariant to irrelevant dimensions of variability in the input, a critical capacity for many tasks. We illustrate these ideas using a well-studied visual hyperacuity task, in which the computation is required to be invariant to the overall retinal location of the input. We show a class of networks based on a wide variety of recurrent interactions that perform nearly as well as an ideal observer for the task, and are robust to significant levels of noise. We also show that this way of performing computations is fast, accurate, readily learnable and robust to various forms of noise

Model-based reinforcement learning and navigation in animals and machines

For decades, neuroscientists and psychologists have observed that animal performance on spatial navigation tasks suggests an internal learned map of the environment. More recently, map-based (or model-based) reinforcement learning has become a highly active research area in machine learning. With a learned model of their environment, both animals and artificial agents can generalize between tasks and learn rapidly. In this thesis, I present approaches for developing efficient model--based behaviour in machines and explaining model--based behaviour in animals. From a neuroscience perspective, I focus on the hippocampus, believed to be a major substrate of model-based behaviour in the brain. I consider how hippocampal connectivity enable path--finding between different locations in an environment. The model describes how environments with boundaries and barriers can be represented in recurrent neural networks (i.e. attractor networks), and how the transient activity in these networks, after being stimulated with a goal location, could be used for determining a path to the goal. I also propose how the connectivity of these map--like networks can be learned from the spatial firing patterns observed in the input pathway to the hippocampus (i.e. grid cells and border cells). From a machine learning perspective, I describe a reinforcement learning model that integrates model-based methods and "episodic control", an approach to reinforcement learning based on episodic memory. According to episodic control, the agent learns how to act in the environment by storing snapshot-like memories of its observations, then comparing its current observations to similar snapshot memories where it took an action that resulted in high reward. In our approach, the agent augments these real-world memories with episodes simulated offline using a learned model of the environment. These ``simulated memories'' allow the agent to adapt faster when the reward locations change. Next, I describe Variational State Tabulation (VaST), a model--based method for learning quickly with continuous and high-dimensional observations (like those found in 3D navigation tasks). The VaST agent learns to map its observations to a limited number of discrete abstract states, and build a transition model over those abstract states. The long--term values of different actions in each state are updated continuously and efficiently in the background as the agent explores the environment. I show how the VaST agent can learn faster than other state-of-the-art algorithms, even changing its policy after a single new experience, and how it can respond quickly to changing rewards in complex 3D environments. The models I present allow the agent to rapidly adapt to changing goals and rewards, a key component of intelligence. They use a combination of features attributed to model-based and episodic controllers, suggesting that the division between the two fields is not strict. I therefore also consider the consequences of these findings on theories of model-based learning, episodic control and hippocampal function

樹状突起ニューロン計算および差分進化アルゴリズムに関する研究

富山大学・富理工博甲第118号・陳瑋・2017/03/23富山大学201

A Recurrent Log-Linearized Gaussian Mixture Network

Context in time series is one of the most useful andinteresting characteristics for machine learning. In some cases, thedynamic characteristic would be the only basis for achieving a possibleclassification. A novel neural network, which is named “a recurrentlog-linearized Gaussian mixture network (R-LLGMN)," isproposed in this paper for classification of time series. The structureof this network is based on a hidden Markov model (HMM),which has been well developed in the area of speech recognition.R-LLGMN can as well be interpreted as an extension of a probabilisticneural network using a log-linearized Gaussian mixturemodel, in which recurrent connections have been incorporated tomake temporal information in use. Some simulation experimentsare carried out to compare R-LLGMN with the traditional estimatorof HMM as classifiers, and finally, pattern classification experimentsfor EEG signals are conducted. It is indicated from theseexperiments that R-LLGMN can successfully classify not only artificialdata but real biological data such as EEG signals

Oil reservoir properties estimation using neural networks

This paper investigates the applicability as well as the accuracy of artificial neural networks for estimating specific parameters that describe reservoir properties based on seismic data. This approach relies on JPL`s adjoint operators general purpose neural network code to determine the best suited architecture. The authors believe that results presented in this work demonstrate that artificial neural networks produce surprisingly accurate estimates of the reservoir parameters