2,345 research outputs found
Incremental learning with respect to new incoming input attributes
Neural networks are generally exposed to a dynamic environment where the training patterns or the input attributes (features) will likely be introduced into the current domain incrementally. This paper considers the situation where a new set of input attributes must be considered and added into the existing neural network. The conventional method is to discard the existing network and redesign one from scratch. This approach wastes the old knowledge and the previous effort. In order to reduce computational time, improve generalization accuracy, and enhance intelligence of the learned models, we present ILIA algorithms (namely ILIA1, ILIA2, ILIA3, ILIA4 and ILIA5) capable of Incremental Learning in terms of Input Attributes. Using the ILIA algorithms, when new input attributes are introduced into the original problem, the existing neural network can be retained and a new sub-network is constructed and trained incrementally. The new sub-network and the old one are merged later to form a new network for the changed problem. In addition, ILIA algorithms have the ability to decide whether the new incoming input attributes are relevant to the output and consistent with the existing input attributes or not and suggest to accept or reject them. Experimental results show that the ILIA algorithms are efficient and effective both for the classification and regression problems
Incremental construction of LSTM recurrent neural network
Long Short--Term Memory (LSTM) is a recurrent neural network that
uses structures called memory blocks to allow the net remember
significant events distant in the past input sequence in order to
solve long time lag tasks, where other RNN approaches fail.
Throughout this work we have performed experiments using LSTM
networks extended with growing abilities, which we call GLSTM.
Four methods of training growing LSTM has been compared. These
methods include cascade and fully connected hidden layers as well
as two different levels of freezing previous weights in the
cascade case. GLSTM has been applied to a forecasting problem in a biomedical domain, where the input/output behavior of five
controllers of the Central Nervous System control has to be
modelled. We have compared growing LSTM results against other
neural networks approaches, and our work applying conventional
LSTM to the task at hand.Postprint (published version
Proceedings of the Conference on Human and Economic Resources
Recent development of information technologies and telecommunications have given rise to an extraordinary increase in the data transactions in the financial markets. In large and transparent markets, with lower transactions and information costs, financial participants react more rapidly to changes in the profitability of their assets, and in their perception of the risks of the different financial instruments. In this respect, if the rapidity of reaction of financial players is the main feature of globalized markets, then only advanced information technologies, which uses data resources efficiently are capable of reflecting these complex nature of financial markets. The aim of this paper is to show how the new information technologies affect modelling of financial markets and decisions by using limited data resources within an intelligent system. By using intelligent information systems, mainly neural networks, this paper tries to show how the the limited economic data can be used for efficient economic decisions in the global financial markets. Advances in microprocessors and software technologies make it possible to enable the development of increasingly powerful systems at reasonable costs. The new technologies have created artificial systems, which imitate peopleâs brain for efficient analysis of economic data. According to Hertz, Krogh and Palmer (1991), artificial neural networks which have a similar structure of the brain consist of nodes passing activation signals to each other. Within the nodes, if incoming activation signals from the others are combined some of the nodes will produce an activation signal modified by a connection weight between it and the node to which it is linked. By using financial data from international foreign exchange markets, namely daily time series of EUR/USD parity, and by employing certain neural network algorithms, it has showed that new information technologies have advantages on efficient usage of limited economic data in modeling. By investigating the âartificialâ works on modeling of international financial markets, this paper is tried to show how limited information in the markets can be used for efficient economic decisions. By investigating certain neural networks algorithms, the paper displays how artificial neural networks have been used for efficient economic modeling and decisions in global F/X markets. New information technologies have many advantages over statistics methods in terms of efficient data modeling. They are capable of analyzing complex patterns quickly and with a high degree of accuracy. Since, âartificialâ information systems do not make any assumptions about the nature of the distribution of the data, they are not biased in their analysis. By using different neural network algorithms, the economic data can be modeled in an efficient way. Especially if the markets are non-linear and complex, the intelligent systems are more powerful on explaining the market behavior in the chaotic environments. With more advanced information technologies, in the future, it will be possible to model all the complexity of the economic life. New researches in the future need a more strong interaction between economics and computer science.neural networks,knowledge, information technology, communication technology
Recommended from our members
An incremental approach to MSE-based feature selection
Feature selection plays an important role in classification systems. Using classifier error rate as the evaluation function, feature selection is integrated with incremental training. A neural network classifier is implemented with an incremental training approach to detect and discard irrelevant features. By learning attributes one after another, our classifier can find directly the attributes that make no contribution to classification. These attributes are marked and considered for removal. Incorporated with a Minimum Squared Error (MSE) based feature ranking scheme, four batch removal methods based on classifier error rate have been developed to discard irrelevant features. These feature selection methods reduce the computational complexity involved in searching among a large number of possible solutions significantly. Experimental results show that our feature selection methods work well on several benchmark problems compared with other feature selection methods. The selected subsets are further validated by a Constructive Backpropagation (CBP) classifier, which confirms increased classification accuracy and reduced training cost
Recommended from our members
Recursive Percentage based Hybrid Pattern Training for Supervised Learning
Supervised learning algorithms, often used to find the I/O relationship in data, have the tendency to be trapped in local optima as opposed to the desirable global optima. In this paper, we discuss the RPHP learning algorithm. The algorithm uses Real Coded Genetic Algorithm based global and local searches to find a set of pseudo global optimal solutions. Each pseudo global optimum is a local optimal solution from the point of view of all the patterns but globally optimal from the point of view of a subset of patterns. Together with RPHP, a Kth nearest neighbor algorithm is used as a second level pattern distributor to solve a test pattern. We also show theoretically the condition under which finding several pseudo global optimal solutions requires a shorter training time than finding a single global optimal solution. As the difficulty of curve fitting problems is easily estimated, we verify the capability of the RPHP algorithm against them and compare the RPHP algorithm with three counterparts to show the benefits of hybrid learning and active recursive subset selection. The RPHP shows a clear superiority in performance. We conclude our paper by identifying possible loopholes in the RPHP algorithm and proposing possible solutions
Metaheuristic design of feedforward neural networks: a review of two decades of research
Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era
- âŠ