Testing for Structural Breaks in Nonlinear Dynamic Models Using Artificial Neural Network Approximations

In this paper we suggest a number of statistical tests based on neural network models, that are designed to be powerful against structural breaks in otherwise stationary time series processes while allowing for a variety of nonlinear specifications for the dynamic model underlying them. It is clear that in the presence of nonlinearity standard tests of structural breaks for linear models may not have the expected performance under the null hypothesis of no breaks because the model is misspecified. We therefore proceed by approximating the conditional expectation of the dependent variable through a neural network. Then, the residual from this approximation is tested using standard residual based structural break tests. We investigate the asymptoptic behaviour of residual based structural break tests in nonlinear regression models. Monte Carlo evidence suggests that the new tests are powerful against a variety of structural breaks while allowing for stationary nonlinearities.Nonlinearity, Structural breaks, Neural networks

Prediction of residual stress in precision milling of AISI H13 steel

Surface integrity describes the attributes of a surface and it influences the functional performance of a work piece significantly. Residual stress is one of the major characterization parameters of surface integrity. Non-favorable residual stresses on a machined surface can reduce the fatigue life and performance of the machined part. It therefore requires a prediction model for residual stress in order to establish machining strategy to obtain favorable residual stress for prolonged fatigue life. Hardened tool steels have been widely used to make molds and dies by precision milling in aerospace and automotive industries. Knowledge of the relationship between residual stress on the machined molds and machining conditions is very important for process control. In this work, a prediction model for residual stress was developed by using a model-based approach on an Artificial Neural Network. This model is expected to predict the residual stress based on cutting parameters such as cutting speed, feed rate, depth of cut and tool lead angle. Several precision milling trials were carried out using a central composite design method. The networks have been trained and validated by experimental results. The performance of a feed forward neural network model with backpropagation was assessed and compared with a radial basis function network model by criterion of least mean squared error. Furthermore, the neural network prediction model was supported by the finite element simulation of the milling process to understand the formation mechanism of the residual stress in the machined surface. It was found, that the predicted values by the neural network model matched well with the experimental results. The radial basis function network showed better results than the feed forward network and was therefore chosen to take forward in the analysis. The feed rate was in this case the most influential factor, because it contributes significantly to heat and deformation on the work piece. The model could be used to optimize machining processes to obtain machining strategy for generating favorable residual stress and increasing fatigue life performance of the machined parts

Dynamical Isometry is Achieved in Residual Networks in a Universal Way for any Activation Function

We demonstrate that in residual neural networks (ResNets) dynamical isometry is achievable irrespectively of the activation function used. We do that by deriving, with the help of Free Probability and Random Matrix Theories, a universal formula for the spectral density of the input-output Jacobian at initialization, in the large network width and depth limit. The resulting singular value spectrum depends on a single parameter, which we calculate for a variety of popular activation functions, by analyzing the signal propagation in the artificial neural network. We corroborate our results with numerical simulations of both random matrices and ResNets applied to the CIFAR-10 classification problem. Moreover, we study the consequence of this universal behavior for the initial and late phases of the learning processes. We conclude by drawing attention to the simple fact, that initialization acts as a confounding factor between the choice of activation function and the rate of learning. We propose that in ResNets this can be resolved based on our results, by ensuring the same level of dynamical isometry at initialization

Metallurgical productions fault detection method based on RESLSTM-CNN model

Timely detection of abnormal working conditions and accurate diagnosis of abnormal working conditions are of great research significance to ensure the safe and stable operation of metallurgical production processes and to avoid losses caused by faults. In this paper, it propose a residual long and short-term memory network and convolutional neural network (RESLSTM-CNN) model for fault detection in metallurgical production processes bearing fault detection with an accuracy of 98,92 %

Estimating position & velocity in 3D space from monocular video sequences using a deep neural network

This work describes a regression model based on Convolutional Neural Networks (CNN) and Long-Short Term Memory (LSTM) networks for tracking objects from monocular video sequences. The target application being pursued is Vision-Based Sensor Substitution (VBSS). In particular, the tool-tip position and velocity in 3D space of a pair of surgical robotic instruments (SRI) are estimated for three surgical tasks, namely suturing, needle-passing and knot-tying. The CNN extracts features from individual video frames and the LSTM network processes these features over time and continuously outputs a 12-dimensional vector with the estimated position and velocity values. A series of analyses and experiments are carried out in the regression model to reveal the benefits and drawbacks of different design choices. First, the impact of the loss function is investigated by adequately weighing the Root Mean Squared Error (RMSE) and Gradient Difference Loss (GDL), using the VGG16 neural network for feature extraction. Second, this analysis is extended to a Residual Neural Network designed for feature extraction, which has fewer parameters than the VGG16 model, resulting in a reduction of ~96.44 % in the neural network size. Third, the impact of the number of time steps used to model the temporal information processed by the LSTM network is investigated. Finally, the capability of the regression model to generalize to the data related to "unseen" surgical tasks (unavailable in the training set) is evaluated. The aforesaid analyses are experimentally validated on the public dataset JIGSAWS. These analyses provide some guidelines for the design of a regression model in the context of VBSS, specifically when the objective is to estimate a set of 1D time series signals from video sequences.Peer ReviewedPostprint (author's final draft

A comparative study of image processing thresholding algorithms on residual oxide scale detection in stainless steel production lines

The present work is intended for residual oxide scale detection and classification through the application of image processing techniques. This is a defect that can remain in the surface of stainless steel coils after an incomplete pickling process in a production line. From a previous detailed study over reflectance of residual oxide defect, we present a comparative study of algorithms for image segmentation based on thresholding methods. In particular, two computational models based on multi-linear regression and neural networks will be proposed. A system based on conventional area camera with a special lighting was installed and fully integrated in an annealing and pickling line for model testing purposes. Finally, model approaches will be compared and evaluated their performance..Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech