An Adaptive Retraining Method for the Exchange Rate Forecasting

The paper advances an original artificial intelligence-based mechanism for specific economic predictions. The time series under discussion are non-stationary; therefore the distribution of the time series changes over time. The algorithm establishes how a viable structure of an artificial neural network (ANN) at a previous moment of time could be retrained in an efficient manner, in order to support modifications in a complex input-output function of financial forecasting. A "remembering process" for the former knowledge achieved in the previous learning phase is used to enhance the accuracy of the predictions. The results show that the first training (which includes the searching phase for the optimal architecture) always takes a relatively long time, but then the system can be very easily retrained, as there are no changes in the structure. The advantage of the retraining procedure is that some relevant aspects are preserved (remembered) not only from the immediate previous training phase, but also from the previous but one phase, and so on. A kind of slow forgetting process also occurs; thus it is much easier for the ANN to remember specific aspects of the previous training instead of the first training. The experiments reveal the high importance of the retraining phase as an upgrading/updating process and the effect of ignoring it, as well. There has been a decrease in the test error when successive retraining phases were performed.Neural Networks, Exchange Rate, Adaptive Retraining, Delay Vectors, Iterative Simulation

Artificial Neural Network Pruning to Extract Knowledge

Artificial Neural Networks (NN) are widely used for solving complex problems from medical diagnostics to face recognition. Despite notable successes, the main disadvantages of NN are also well known: the risk of overfitting, lack of explainability (inability to extract algorithms from trained NN), and high consumption of computing resources. Determining the appropriate specific NN structure for each problem can help overcome these difficulties: Too poor NN cannot be successfully trained, but too rich NN gives unexplainable results and may have a high chance of overfitting. Reducing precision of NN parameters simplifies the implementation of these NN, saves computing resources, and makes the NN skills more transparent. This paper lists the basic NN simplification problems and controlled pruning procedures to solve these problems. All the described pruning procedures can be implemented in one framework. The developed procedures, in particular, find the optimal structure of NN for each task, measure the influence of each input signal and NN parameter, and provide a detailed verbal description of the algorithms and skills of NN. The described methods are illustrated by a simple example: the generation of explicit algorithms for predicting the results of the US presidential election.Comment: IJCNN 202

Towards Robust Neural Networks via Random Self-ensemble

Recent studies have revealed the vulnerability of deep neural networks: A small adversarial perturbation that is imperceptible to human can easily make a well-trained deep neural network misclassify. This makes it unsafe to apply neural networks in security-critical applications. In this paper, we propose a new defense algorithm called Random Self-Ensemble (RSE) by combining two important concepts: {\bf randomness} and {\bf ensemble}. To protect a targeted model, RSE adds random noise layers to the neural network to prevent the strong gradient-based attacks, and ensembles the prediction over random noises to stabilize the performance. We show that our algorithm is equivalent to ensemble an infinite number of noisy models $f_\epsilon$ without any additional memory overhead, and the proposed training procedure based on noisy stochastic gradient descent can ensure the ensemble model has a good predictive capability. Our algorithm significantly outperforms previous defense techniques on real data sets. For instance, on CIFAR-10 with VGG network (which has 92\% accuracy without any attack), under the strong C\&W attack within a certain distortion tolerance, the accuracy of unprotected model drops to less than 10\%, the best previous defense technique has $48\%$ accuracy, while our method still has $86\%$ prediction accuracy under the same level of attack. Finally, our method is simple and easy to integrate into any neural network.Comment: ECCV 2018 camera read