Volumetric efficiency modelling of internal combustion engines based on a novel adaptive learning algorithm of artificial neural networks

[EN] Air mass flow determination is one of the main variables on the control of internal combustion engines. Effectiveness of intake air systems is evaluated through the volumetric efficiency coefficient. Intake air systems characterization by means of physical models needs either significant amount of input data or notable calculation times. Because of these drawbacks, empirical approaches are often used by means of black-box models based on Artificial Neural Networks. As alternative to the standard gradient descendent method an adaptive learning algorithm is developed based on the increase of hidden layer weight update speed. The results presented in this paper show that the proposed adaptive learning method performs with higher learning speed, reduced computational resources and lower network complexities. A parametric study of several Multiple Layer Perceptron (MLP) networks is carried out with the variation of the number of epochs, number of hidden neurons, momentum coefficient and learning algorithm. The training and validation data are obtained from steady state tests carried out in an automotive turbocharged diesel engine. (C) 2017 Elsevier Ltd. All rights reserved.Authors want to acknowledge the "Apoyo para la investigacion y Desarrollo (PAID)", grant for doctoral studies (FPI S1 2015 2512), of Universitat Politecnica de Valencia.Luján, JM.; Climent, H.; García-Cuevas González, LM.; Moratal-Martínez, AA. (2017). Volumetric efficiency modelling of internal combustion engines based on a novel adaptive learning algorithm of artificial neural networks. Applied Thermal Engineering. 123:625-634. https://doi.org/10.1016/j.applthermaleng.2017.05.087S62563412

Extending Memory for Language Modelling

Breakthroughs in deep learning and memory networks have made major advances in natural language understanding. Language is sequential and information carried through the sequence can be captured through memory networks. Learning the sequence is one of the key aspects in learning the language. However, memory networks are not capable of holding infinitely long sequences in their memories and are limited by various constraints such as the vanishing or exploding gradient problem. Therefore, natural language understanding models are affected when presented with long sequential text. We introduce Long Term Memory network (LTM) to learn from infinitely long sequences. LTM gives priority to the current inputs to allow it to have a high impact. Language modeling is an important factor in natural language understanding. LTM was tested in language modeling, which requires long term memory. LTM is tested on Penn Tree bank dataset, Google Billion Word dataset and WikiText-2 dataset. We compare LTM with other language models which require long term memory

Handbook of Computational Intelligence in Manufacturing and Production Management

Artificial intelligence (AI) is simply a way of providing a computer or a machine to think intelligently like human beings. Since human intelligence is a complex abstraction, scientists have only recently began to understand and make certain assumptions on how people think and to apply these assumptions in order to design AI programs. It is a vast knowledge base discipline that covers reasoning, machine learning, planning, intelligent search, and perception building. Traditional AI had the limitations to meet the increasing demand of search, optimization, and machine learning in the areas of large, biological, and commercial database information systems and management of factory automation for different industries such as power, automobile, aerospace, and chemical plants. The drawbacks of classical AI became more pronounced due to successive failures of the decade long Japanese project on fifth generation computing machines. The limitation of traditional AI gave rise to development of new computational methods in various applications of engineering and management problems. As a result, these computational techniques emerged as a new discipline called computational intelligence (CI)

Collapse warning system using LSTM neural networks for construction disaster prevention in extreme wind weather

Strong wind during extreme weather conditions (e.g., strong winds during typhoons) is one of the natural factors that cause the collapse of frame-type scaffolds used in façade work. This study developed an alert system for use in determining whether the scaffold structure could withstand the stress of the wind force. Conceptually, the scaffolds collapsed by the warning system developed in the study contains three modules. The first module involves the establishment of wind velocity prediction models. This study employed various deep learning and machine learning techniques, namely deep neural networks, long short-term memory neural networks, support vector regressions, random forest, and k-nearest neighbors. Then, the second module contains the analysis of wind force on the scaffolds. The third module involves the development of the scaffold collapse evaluation approach. The study area was Taichung City, Taiwan. This study collected meteorological data from the ground stations from 2012 to 2019. Results revealed that the system successfully predicted the possible collapse time for scaffolds within 1 to 6 h, and effectively issued a warning time. Overall, the warning system can provide practical warning information related to the destruction of scaffolds to construction teams in need of the information to reduce the damage risk

A temporal continual learning framework for investment decisions

Temporal continual learning (TCL) is introduced in this thesis as an extension of continual learning (CL). While traditional CL has been applied to sequential tasks, extending CL to TCL aims to allow machines to accumulate specific knowledge of temporal states, to address concept drift (CD) problems. This approach is shown to hold considerable benefits in domains where non-stationary time-series are used for decision-making, particularly in finance. A TCL framework called continual learning augmentation (CLA) is introduced, to drive long-term decision making in complex, multivariate, temporal problems. Moreover, CLA uses an external memory structure to store learner parameters from particular past temporal states for recall in the future. The contributions of this work are fourfold: First, a temporal, state-based, external memory structure is developed. Second, this is used to memory augment well-understood base-learners, such as LSTM, feed-forward neural networks (FFNN) and linear regression. Third, a remember-gate, based on residual change, learns in an open-world fashion to define different states for which learner-parameters are stored along with a contextual reference of the state. Fourthly, a memory recall gate is developed, based on various time-series similarity approaches, which can compare the current input space with the contextual references stored in memory, recalling the most appropriate learner parameters for use in the current period. In testing, CLA is found to improve the performance of LSTM, FFNN, and linear regression learners applied to a complex, real-world finance task: stock selection in international and emerging equities investing. Several different similarity approaches are tested in CLA's remember-gate, with dynamic time warping (DTW) outperforming simple Euclidean distance (ED), while auto encoder (AE) distance is found to both mitigate the resource overheads of DTW and provide better performance. A hybrid approach is also introduced, warp-AE, which performs well. In addition, a visualisation is introduced to allow CLA to be interpreted by domain experts in terms of which memory did what and when. A complex application is used to test TCL and a five-point statistical testing framework is introduced. This thesis elucidates the research of the last five years regarding TCL. Keywords: Continual learning, time-series, memory, neural network

Learning long-term dependencies in segmented-memory recurrent neural networks with backpropagation of error

In general, recurrent neural networks have difficulties in learning long-term dependencies. The segmented-memory recurrent neural network (SMRNN) architecture together with the extended real-time recurrent learning (eRTRL) algorithm was proposed to circumvent this problem. Due to its computational complexity eRTRL becomes impractical with increasing network size. Therefore, we introduce the less complex extended backpropagation through time (eBPTT) for SMRNN together with a layer-local unsupervised pre-training procedure. A comparison on the information latching problem showed that eRTRL is better able to handle the latching of information over longer periods of time, even though eBPTT guaranteed a better generalisation when training was successful. Further, pre-training significantly improved the ability to learn long-term dependencies with eBPTT. Therefore, the proposed eBPTT algorithm is suited for tasks that require big networks where eRTRL is impractical. The pre-training procedure itself is independent of the supervised learning algorithm and can improve learning in SMRNN in general