An Adaptive Retraining Method for the Exchange Rate Forecasting

The paper advances an original artificial intelligence-based mechanism for specific economic predictions. The time series under discussion are non-stationary; therefore the distribution of the time series changes over time. The algorithm establishes how a viable structure of an artificial neural network (ANN) at a previous moment of time could be retrained in an efficient manner, in order to support modifications in a complex input-output function of financial forecasting. A "remembering process" for the former knowledge achieved in the previous learning phase is used to enhance the accuracy of the predictions. The results show that the first training (which includes the searching phase for the optimal architecture) always takes a relatively long time, but then the system can be very easily retrained, as there are no changes in the structure. The advantage of the retraining procedure is that some relevant aspects are preserved (remembered) not only from the immediate previous training phase, but also from the previous but one phase, and so on. A kind of slow forgetting process also occurs; thus it is much easier for the ANN to remember specific aspects of the previous training instead of the first training. The experiments reveal the high importance of the retraining phase as an upgrading/updating process and the effect of ignoring it, as well. There has been a decrease in the test error when successive retraining phases were performed.Neural Networks, Exchange Rate, Adaptive Retraining, Delay Vectors, Iterative Simulation

A Neuro-Classification Model for Socio-Technical Systems

This paper presents an original classifier model based on an artificial neural network (ANN) architecture that is able to learn a specific human behavior and can be used in different socio-economic systems. After a training process, the system can identify and classify a human subject using a list of parameters. The model can be further used to analyze and build a safe socio-technical system (STS). A new technique is applied to find an optimal architecture of the neural network. The system shows a good accuracy of the classifications even for a relatively small amount of training data. Starting from a previous result on adaptive forecasting, the model is enhanced by using the retraining technique for an enlarged data set.artificial neural network, training process, classification, socio-technical system

Towards Robust Neural Networks via Random Self-ensemble

Recent studies have revealed the vulnerability of deep neural networks: A small adversarial perturbation that is imperceptible to human can easily make a well-trained deep neural network misclassify. This makes it unsafe to apply neural networks in security-critical applications. In this paper, we propose a new defense algorithm called Random Self-Ensemble (RSE) by combining two important concepts: {\bf randomness} and {\bf ensemble}. To protect a targeted model, RSE adds random noise layers to the neural network to prevent the strong gradient-based attacks, and ensembles the prediction over random noises to stabilize the performance. We show that our algorithm is equivalent to ensemble an infinite number of noisy models $f_\epsilon$ without any additional memory overhead, and the proposed training procedure based on noisy stochastic gradient descent can ensure the ensemble model has a good predictive capability. Our algorithm significantly outperforms previous defense techniques on real data sets. For instance, on CIFAR-10 with VGG network (which has 92\% accuracy without any attack), under the strong C\&W attack within a certain distortion tolerance, the accuracy of unprotected model drops to less than 10\%, the best previous defense technique has $48\%$ accuracy, while our method still has $86\%$ prediction accuracy under the same level of attack. Finally, our method is simple and easy to integrate into any neural network.Comment: ECCV 2018 camera read

Generation of Explicit Knowledge from Empirical Data through Pruning of Trainable Neural Networks

This paper presents a generalized technology of extraction of explicit knowledge from data. The main ideas are 1) maximal reduction of network complexity (not only removal of neurons or synapses, but removal all the unnecessary elements and signals and reduction of the complexity of elements), 2) using of adjustable and flexible pruning process (the pruning sequence shouldn't be predetermined - the user should have a possibility to prune network on his own way in order to achieve a desired network structure for the purpose of extraction of rules of desired type and form), and 3) extraction of rules not in predetermined but any desired form. Some considerations and notes about network architecture and training process and applicability of currently developed pruning techniques and rule extraction algorithms are discussed. This technology, being developed by us for more than 10 years, allowed us to create dozens of knowledge-based expert systems. In this paper we present a generalized three-step technology of extraction of explicit knowledge from empirical data.Comment: 9 pages, The talk was given at the IJCNN '99 (Washington DC, July 1999