5,226 research outputs found
Adaptive Learning Method of Recurrent Temporal Deep Belief Network to Analyze Time Series Data
Deep Learning has the hierarchical network architecture to represent the
complicated features of input patterns. Such architecture is well known to
represent higher learning capability compared with some conventional models if
the best set of parameters in the optimal network structure is found. We have
been developing the adaptive learning method that can discover the optimal
network structure in Deep Belief Network (DBN). The learning method can
construct the network structure with the optimal number of hidden neurons in
each Restricted Boltzmann Machine and with the optimal number of layers in the
DBN during learning phase. The network structure of the learning method can be
self-organized according to given input patterns of big data set. In this
paper, we embed the adaptive learning method into the recurrent temporal RBM
and the self-generated layer into DBN. In order to verify the effectiveness of
our proposed method, the experimental results are higher classification
capability than the conventional methods in this paper.Comment: 8 pages, 9 figures. arXiv admin note: text overlap with
arXiv:1807.03487, arXiv:1807.0348
Making Good on LSTMs' Unfulfilled Promise
LSTMs promise much to financial time-series analysis, temporal and cross-sectional inference, but we find that they do not deliver in a real-world financial management task. We examine an alternative called Continual Learning (CL), a memory-augmented approach, which can provide transparent explanations, i.e. which memory did what and when. This work has implications for many financial applications including credit, time-varying fairness in decision making and more. We make three important new observations. Firstly, as well as being more explainable, time-series CL approaches outperform LSTMs as well as a simple sliding window learner using feed-forward neural networks (FFNN). Secondly, we show that CL based on a sliding window learner (FFNN) is more effective than CL based on a sequential learner (LSTM). Thirdly, we examine how real-world, time-series noise impacts several similarity approaches used in CL memory addressing. We provide these insights using an approach called Continual Learning Augmentation (CLA) tested on a complex real-world problem, emerging market equities investment decision making. CLA provides a test-bed as it can be based on different types of time-series learners, allowing testing of LSTM and FFNN learners side by side. CLA is also used to test several distance approaches used in a memory recall-gate: Euclidean distance (ED), dynamic time warping (DTW), auto-encoders (AE) and a novel hybrid approach, warp-AE. We find that ED under-performs DTW and AE but warp-AE shows the best overall performance in a real-world financial task
Born to learn: The inspiration, progress, and future of evolved plastic artificial neural networks
Biological plastic neural networks are systems of extraordinary computational
capabilities shaped by evolution, development, and lifetime learning. The
interplay of these elements leads to the emergence of adaptive behavior and
intelligence. Inspired by such intricate natural phenomena, Evolved Plastic
Artificial Neural Networks (EPANNs) use simulated evolution in-silico to breed
plastic neural networks with a large variety of dynamics, architectures, and
plasticity rules: these artificial systems are composed of inputs, outputs, and
plastic components that change in response to experiences in an environment.
These systems may autonomously discover novel adaptive algorithms, and lead to
hypotheses on the emergence of biological adaptation. EPANNs have seen
considerable progress over the last two decades. Current scientific and
technological advances in artificial neural networks are now setting the
conditions for radically new approaches and results. In particular, the
limitations of hand-designed networks could be overcome by more flexible and
innovative solutions. This paper brings together a variety of inspiring ideas
that define the field of EPANNs. The main methods and results are reviewed.
Finally, new opportunities and developments are presented
Generative Adversarial Networks (GANs): Challenges, Solutions, and Future Directions
Generative Adversarial Networks (GANs) is a novel class of deep generative
models which has recently gained significant attention. GANs learns complex and
high-dimensional distributions implicitly over images, audio, and data.
However, there exists major challenges in training of GANs, i.e., mode
collapse, non-convergence and instability, due to inappropriate design of
network architecture, use of objective function and selection of optimization
algorithm. Recently, to address these challenges, several solutions for better
design and optimization of GANs have been investigated based on techniques of
re-engineered network architectures, new objective functions and alternative
optimization algorithms. To the best of our knowledge, there is no existing
survey that has particularly focused on broad and systematic developments of
these solutions. In this study, we perform a comprehensive survey of the
advancements in GANs design and optimization solutions proposed to handle GANs
challenges. We first identify key research issues within each design and
optimization technique and then propose a new taxonomy to structure solutions
by key research issues. In accordance with the taxonomy, we provide a detailed
discussion on different GANs variants proposed within each solution and their
relationships. Finally, based on the insights gained, we present the promising
research directions in this rapidly growing field.Comment: 42 pages, Figure 13, Table
Forecasting Unemployment Rate Using a Neural Network with Fuzzy Inference System
Greece is a low-productivity economy with an ineffective welfare state, relying almost exclusively on low wages and social transfers. Failure to come to terms with this reality hampers both the appropriateness of EU recommendations and the Greek government's capacity to deal with unemployment. Rather than finding a job in a family business or through relationship contacts, young people stay unemployed. Nor can people move back to their village of origin so easily. The underground economy, and the mass of small companies which characterize the Greek economy are booming, on paper. One in three members of the workforce are "self-employed", compared to one in seven in the EU as a whole. (International Viewpoint) An unemployed person in Greece is 2,15 times more likely to suffer poverty than a person in employment. Yet in Greece there are perhaps even more influential factors in determining increased risk of poverty. Thus while unemployment is a crucial factor in the risk of poverty, it is neither the only nor the most significant factor. The paper presents a new technique in the field of unemployment modeling in order to forecast unemployment index. Techniques from the Artificial Neural Networks and from fuzzy logic have been combined to generate a neuro-fuzzy model. The input is a time series. Classical statistics measures are calculated in order to asses the model performance. Further the results are compared with an ARMA and an AR model.forecasting, neural network, unemployment
- …