28,917 research outputs found
On the Impact of Forgetting on Learning Machines
People tend not to have perfect memories when it comes to learning, or to anything else for that matter. Most formal studies of learning, however, assume a perfect memory. Some approaches have restricted the number of items that could be retained. We introduce a complexity theoretic accounting of memory utilization by learning machines. In our new model, memory is measured in bits as a function of the size of the input. There is a hierarchy of learnability based on increasing memory allotment. The lower bound results are proved using an unusual combination of pumping and mutual recursion theorem arguments. For technical reasons, it was necessary to consider two types of memory: long and short term
Combating catastrophic forgetting with developmental compression
Generally intelligent agents exhibit successful behavior across problems in
several settings. Endemic in approaches to realize such intelligence in
machines is catastrophic forgetting: sequential learning corrupts knowledge
obtained earlier in the sequence, or tasks antagonistically compete for system
resources. Methods for obviating catastrophic forgetting have sought to
identify and preserve features of the system necessary to solve one problem
when learning to solve another, or to enforce modularity such that minimally
overlapping sub-functions contain task specific knowledge. While successful,
both approaches scale poorly because they require larger architectures as the
number of training instances grows, causing different parts of the system to
specialize for separate subsets of the data. Here we present a method for
addressing catastrophic forgetting called developmental compression. It
exploits the mild impacts of developmental mutations to lessen adverse changes
to previously-evolved capabilities and `compresses' specialized neural networks
into a generalized one. In the absence of domain knowledge, developmental
compression produces systems that avoid overt specialization, alleviating the
need to engineer a bespoke system for every task permutation and suggesting
better scalability than existing approaches. We validate this method on a robot
control problem and hope to extend this approach to other machine learning
domains in the future
Factors Affecting the Development of Workforce Versatility
Among all strategies supporting the firms' flexibility and agility, the development of human resources versatility holds a promising place. This article presents an investigation of the factors affecting the development of this flexibility lever, related to the problem of planning and scheduling industrial activities, taking into account two dimensions of flexibility: the modulation of working time, which provides the company with fluctuating work capacities, and the versatility of operators: for all the multi-skilled workers, we adopt a dynamic vision of their competences. Therefore, this model takes into account the evolution of their skills over time, depending on how much they were put in practice in previous periods. The model was solved by using an approach relying on genetic algorithm that used an indirect encoding to build the chromosome genotype, and then a serial scheduling scheme is adopted to build the solution
Adaptive Learning Method of Recurrent Temporal Deep Belief Network to Analyze Time Series Data
Deep Learning has the hierarchical network architecture to represent the
complicated features of input patterns. Such architecture is well known to
represent higher learning capability compared with some conventional models if
the best set of parameters in the optimal network structure is found. We have
been developing the adaptive learning method that can discover the optimal
network structure in Deep Belief Network (DBN). The learning method can
construct the network structure with the optimal number of hidden neurons in
each Restricted Boltzmann Machine and with the optimal number of layers in the
DBN during learning phase. The network structure of the learning method can be
self-organized according to given input patterns of big data set. In this
paper, we embed the adaptive learning method into the recurrent temporal RBM
and the self-generated layer into DBN. In order to verify the effectiveness of
our proposed method, the experimental results are higher classification
capability than the conventional methods in this paper.Comment: 8 pages, 9 figures. arXiv admin note: text overlap with
arXiv:1807.03487, arXiv:1807.0348
Echo State Condition at the Critical Point
Recurrent networks with transfer functions that fulfill the Lipschitz
continuity with K=1 may be echo state networks if certain limitations on the
recurrent connectivity are applied. It has been shown that it is sufficient if
the largest singular value of the recurrent connectivity is smaller than 1. The
main achievement of this paper is a proof under which conditions the network is
an echo state network even if the largest singular value is one. It turns out
that in this critical case the exact shape of the transfer function plays a
decisive role in determining whether the network still fulfills the echo state
condition. In addition, several examples with one neuron networks are outlined
to illustrate effects of critical connectivity. Moreover, within the manuscript
a mathematical definition for a critical echo state network is suggested
- …