A Study of Deep Learning Robustness Against Computation Failures

For many types of integrated circuits, accepting larger failure rates in computations can be used to improve energy efficiency. We study the performance of faulty implementations of certain deep neural networks based on pessimistic and optimistic models of the effect of hardware faults. After identifying the impact of hyperparameters such as the number of layers on robustness, we study the ability of the network to compensate for computational failures through an increase of the network size. We show that some networks can achieve equivalent performance under faulty implementations, and quantify the required increase in computational complexity

Learning long-range spatial dependencies with horizontal gated-recurrent units

Progress in deep learning has spawned great successes in many engineering applications. As a prime example, convolutional neural networks, a type of feedforward neural networks, are now approaching -- and sometimes even surpassing -- human accuracy on a variety of visual recognition tasks. Here, however, we show that these neural networks and their recent extensions struggle in recognition tasks where co-dependent visual features must be detected over long spatial ranges. We introduce the horizontal gated-recurrent unit (hGRU) to learn intrinsic horizontal connections -- both within and across feature columns. We demonstrate that a single hGRU layer matches or outperforms all tested feedforward hierarchical baselines including state-of-the-art architectures which have orders of magnitude more free parameters. We further discuss the biological plausibility of the hGRU in comparison to anatomical data from the visual cortex as well as human behavioral data on a classic contour detection task.Comment: Published at NeurIPS 2018 https://papers.nips.cc/paper/7300-learning-long-range-spatial-dependencies-with-horizontal-gated-recurrent-unit

Applications of recurrent neural networks in batch reactors. Part I: NARMA modelling of the dynamic behaviour of the heat transfer fluid

This paper is focused on the development of nonlinear models, using artificial neural networks, able to provide appropriate predictions when acting as process simulators. The dynamic behaviour of the heat transfer fluid temperature in a jacketed chemical reactor has been selected as a case study. Different structures of NARMA (Non-linear ARMA) models have been studied. The experimental results have allowed to carry out a comparison between the different neural approaches and a first-principles model. The best neural results are obtained using a parallel model structure based on a recurrent neural network architecture, which guarantees better dynamic approximations than currently employed neural models. The results suggest that parallel models built up with recurrent networks can be seen as an alternative to phenomenological models for simulating the dynamic behaviour of the heating/cooling circuits which change from batch installation to installation.Publicad