Advances in forecasting with neural networks? Empirical evidence from the NN3 competition on time series prediction

This paper reports the results of the NN3 competition, which is a replication of the M3 competition with an extension of the competition towards neural network (NN) and computational intelligence (CI) methods, in order to assess what progress has been made in the 10 years since the M3 competition. Two masked subsets of the M3 monthly industry data, containing 111 and 11 empirical time series respectively, were chosen, controlling for multiple data conditions of time series length (short/long), data patterns (seasonal/non-seasonal) and forecasting horizons (short/medium/long). The relative forecasting accuracy was assessed using the metrics from the M3, together with later extensions of scaled measures, and non-parametric statistical tests. The NN3 competition attracted 59 submissions from NN, CI and statistics, making it the largest CI competition on time series data. Its main findings include: (a) only one NN outperformed the damped trend using the sMAPE, but more contenders outperformed the AutomatANN of the M3; (b) ensembles of CI approaches performed very well, better than combinations of statistical methods; (c) a novel, complex statistical method outperformed all statistical and Cl benchmarks; and (d) for the most difficult subset of short and seasonal series, a methodology employing echo state neural networks outperformed all others. The NN3 results highlight the ability of NN to handle complex data, including short and seasonal time series, beyond prior expectations, and thus identify multiple avenues for future research. (C) 2011 International Institute of Forecasters. Published by Elsevier B.V. All rights reserved

Metaheuristic design of feedforward neural networks: a review of two decades of research

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era

Brief Study of Identification System Using Regression Neural Network Based on Delay Tap Structures

this investigation aims to provide an overview and investigations of non-linear identification system using neural network approach. Nowadays, a lot of neural network approach was done to provide a satisfying identification system. Backpropagation scheme as the mainstream approach for developing identification system has several limitations such as training data, computation time, architecture, optimization technique for weight value update, and many others. Regression Neural Network which is found by specht on 1990 contains more advantages compare with backpropagation scheme. With the improvement of computation time, architecture, and robustness of this model and provided 90% of effectiveness, promising a good prospective to develop non-linear identification system. For future works, it can be implemented in neural network predictive model control system and another control scheme based on identification system approach

Medical imaging analysis with artificial neural networks

Given that neural networks have been widely reported in the research community of medical imaging, we provide a focused literature survey on recent neural network developments in computer-aided diagnosis, medical image segmentation and edge detection towards visual content analysis, and medical image registration for its pre-processing and post-processing, with the aims of increasing awareness of how neural networks can be applied to these areas and to provide a foundation for further research and practical development. Representative techniques and algorithms are explained in detail to provide inspiring examples illustrating: (i) how a known neural network with fixed structure and training procedure could be applied to resolve a medical imaging problem; (ii) how medical images could be analysed, processed, and characterised by neural networks; and (iii) how neural networks could be expanded further to resolve problems relevant to medical imaging. In the concluding section, a highlight of comparisons among many neural network applications is included to provide a global view on computational intelligence with neural networks in medical imaging

AI Solutions for MDS: Artificial Intelligence Techniques for Misuse Detection and Localisation in Telecommunication Environments

This report considers the application of Articial Intelligence (AI) techniques to the problem of misuse detection and misuse localisation within telecommunications environments. A broad survey of techniques is provided, that covers inter alia rule based systems, model-based systems, case based reasoning, pattern matching, clustering and feature extraction, articial neural networks, genetic algorithms, arti cial immune systems, agent based systems, data mining and a variety of hybrid approaches. The report then considers the central issue of event correlation, that is at the heart of many misuse detection and localisation systems. The notion of being able to infer misuse by the correlation of individual temporally distributed events within a multiple data stream environment is explored, and a range of techniques, covering model based approaches, `programmed' AI and machine learning paradigms. It is found that, in general, correlation is best achieved via rule based approaches, but that these suffer from a number of drawbacks, such as the difculty of developing and maintaining an appropriate knowledge base, and the lack of ability to generalise from known misuses to new unseen misuses. Two distinct approaches are evident. One attempts to encode knowledge of known misuses, typically within rules, and use this to screen events. This approach cannot generally detect misuses for which it has not been programmed, i.e. it is prone to issuing false negatives. The other attempts to `learn' the features of event patterns that constitute normal behaviour, and, by observing patterns that do not match expected behaviour, detect when a misuse has occurred. This approach is prone to issuing false positives, i.e. inferring misuse from innocent patterns of behaviour that the system was not trained to recognise. Contemporary approaches are seen to favour hybridisation, often combining detection or localisation mechanisms for both abnormal and normal behaviour, the former to capture known cases of misuse, the latter to capture unknown cases. In some systems, these mechanisms even work together to update each other to increase detection rates and lower false positive rates. It is concluded that hybridisation offers the most promising future direction, but that a rule or state based component is likely to remain, being the most natural approach to the correlation of complex events. The challenge, then, is to mitigate the weaknesses of canonical programmed systems such that learning, generalisation and adaptation are more readily facilitated

Automatic programming methodologies for electronic hardware fault monitoring

This paper presents three variants of Genetic Programming (GP) approaches for intelligent online performance monitoring of electronic circuits and systems. Reliability modeling of electronic circuits can be best performed by the Stressor - susceptibility interaction model. A circuit or a system is considered to be failed once the stressor has exceeded the susceptibility limits. For on-line prediction, validated stressor vectors may be obtained by direct measurements or sensors, which after pre-processing and standardization are fed into the GP models. Empirical results are compared with artificial neural networks trained using backpropagation algorithm and classification and regression trees. The performance of the proposed method is evaluated by comparing the experiment results with the actual failure model values. The developed model reveals that GP could play an important role for future fault monitoring systems.This research was supported by the International Joint Research Grant of the IITA (Institute of Information Technology Assessment) foreign professor invitation program of the MIC (Ministry of Information and Communication), Korea

Smart Sensor Monitoring in Machining of Difficult-to-cut Materials

The research activities presented in this thesis are focused on the development of smart sensor monitoring procedures applied to diverse machining processes with particular reference to the machining of difficult-to-cut materials. This work will describe the whole smart sensor monitoring procedure starting from the configuration of the multiple sensor monitoring system for each specific application and proceeding with the methodologies for sensor signal detection and analysis aimed at the extraction of signal features to feed to intelligent decision-making systems based on artificial neural networks. The final aim is to perform tool condition monitoring in advanced machining processes in terms of tool wear diagnosis and forecast, in the perspective of zero defect manufacturing and green technologies. The work has been addressed within the framework of the national MIUR PON research project CAPRI, acronym for “Carrello per atterraggio con attuazione intelligente” (Landing Gear with Intelligent Actuation), and the research project STEP FAR, acronym for “Sviluppo di materiali e Tecnologie Ecocompatibili, di Processi di Foratura, taglio e di Assemblaggio Robotizzato” (Development of eco-compatible materials and technologies for robotised drilling and assembly processes). Both projects are sponsored by DAC, the Campania Technological Aerospace District, and involve two aerospace industries, Magnaghi Aeronautica S.p.A. and Leonardo S.p.A., respectively. Due to the industrial framework in which the projects were developed and taking advantage of the support from the industrial partners, the project activities have been carried out with the aim to contribute to the scientific research in the field of machining process monitoring as well as to promote the industrial applicability of the results. The thesis was structured in order to illustrate all the methodologies, the experimental tests and the results obtained from the research activities. It begins with an introduction to “Sensor monitoring of machining processes” (Chapter 2) with particular attention to the main sensor monitoring applications and the types of sensors which are employed in machining. The key methods for advanced sensor signal processing, including the implementation of sensor fusion technology, are discussed in details as they represent the basic input for cognitive decision-making systems construction. The chapter finally presents a brief discussion on cloud-based manufacturing which will represent one of the future developments of this research work. Chapters 3 and 4 illustrate the case studies of machining process sensor monitoring investigated in the research work. Within the CAPRI project, the feasibility of the dry turning process of Ti6Al4V alloy (Chapter 3) was studied with particular attention to the optimization of the machining parameters avoiding the use of coolant fluids. Since very rapid tool wear is experienced during dry machining of Titanium alloys, the multiple sensor monitoring system was used in order to develop a methodology based on a smart system for on line tool wear detection in terms of maximum flank wear land. Within the STEP FAR project, the drilling process of carbon fibre reinforced (CFRP) composite materials was studied using diverse experimental set-ups. Regarding the tools, three different types of drill bit were employed, including traditional as well as innovative geometry ones. Concerning the investigated materials, two different types of stack configurations were employed, namely CFRP/CFRP stacks and hybrid Al/CFRP stacks. Consequently, the machining parameters for each experimental campaign were varied, and also the methods for signal analysis were changed to verify the performance of the different methodologies. Finally, for each case different neural network configurations were investigated for cognitive-based decision making. First of all, the applicability of the system was tested in order to perform tool wear diagnosis and forecast. Then, the discussion proceeds with a further aim of the research work, which is the reduction of the number of selected sensor signal features, in order to improve the performance of the cognitive decision-making system, simplify modelling and facilitate the implementation of these methodologies in a cloud manufacturing approach to tool condition monitoring. Sensor fusion methodologies were applied to the extracted and selected sensor signal features in the perspective of feature reduction with the purpose to implement these procedures for big data analytics within the Industry 4.0 framework. In conclusion, the positive impact of the proposed tool condition monitoring methodologies based on multiple sensor signal acquisition and processing is illustrated, with particular reference to the reliable assessment of tool state in order to avoid too early or too late cutting tool substitution that negatively affect machining time and cost