12,203 research outputs found
A Survey of Parallel Data Mining
With the fast, continuous increase in the number and size of databases, parallel data mining is a natural and cost-effective approach to tackle the problem of scalability in data mining. Recently there has been a considerable research on parallel data mining. However, most projects focus on the parallelization of a single kind of data mining algorithm/paradigm. This paper surveys parallel data mining with a broader perspective. More precisely, we discuss the parallelization of data mining algorithms of four knowledge discovery paradigms, namely rule induction, instance-based learning, genetic algorithms and neural networks. Using the lessons
learned from this discussion, we also derive a set of heuristic principles for designing efficient parallel data mining algorithms
Распараллеливание алгоритмов функционирования классификатора со случайными подпространствами
Нейронные сети являются достаточно популярным средством решения многих задач искуственного интеллекта. В то же время,
любые аппаратные реализации нейросетевых архитектур очень быстро устаревают благодаря стремительному развитию
вычислительной техники. Таким образом, большинство исследователей стремятся использовать в первую очередь программные
реализации алгоритмов, что делает актуальным распараллеливание функционирования нейросетевых систем. В работе
рассматривается нейросетевой классификатор со случайными подпространствами и предлагаются алгоритмы распараллеливания
его основных операций.Neural networks are quite popular for solving many tasks of artificial intelligence. At the same time, any hardware implementation of
some neural architecture becomes obsolete really fast due to rapid development of semiconductor industry. Thus, most researchers first of
all tend to use software algorithm implementations, which makes parallelization of neural network algorithms quite attractive. This article
analyses neural random subspace classifier and suggests algorithms for parallelization off it’s basic operations
Single stream parallelization of generalized LSTM-like RNNs on a GPU
Recurrent neural networks (RNNs) have shown outstanding performance on
processing sequence data. However, they suffer from long training time, which
demands parallel implementations of the training procedure. Parallelization of
the training algorithms for RNNs are very challenging because internal
recurrent paths form dependencies between two different time frames. In this
paper, we first propose a generalized graph-based RNN structure that covers the
most popular long short-term memory (LSTM) network. Then, we present a
parallelization approach that automatically explores parallelisms of arbitrary
RNNs by analyzing the graph structure. The experimental results show that the
proposed approach shows great speed-up even with a single training stream, and
further accelerates the training when combined with multiple parallel training
streams.Comment: Accepted by the 40th IEEE International Conference on Acoustics,
Speech and Signal Processing (ICASSP) 201
Accelerating recurrent neural network training using sequence bucketing and multi-GPU data parallelization
An efficient algorithm for recurrent neural network training is presented.
The approach increases the training speed for tasks where a length of the input
sequence may vary significantly. The proposed approach is based on the optimal
batch bucketing by input sequence length and data parallelization on multiple
graphical processing units. The baseline training performance without sequence
bucketing is compared with the proposed solution for a different number of
buckets. An example is given for the online handwriting recognition task using
an LSTM recurrent neural network. The evaluation is performed in terms of the
wall clock time, number of epochs, and validation loss value.Comment: 4 pages, 5 figures, Comments, 2016 IEEE First International
Conference on Data Stream Mining & Processing (DSMP), Lviv, 201
A Multi-signal Variant for the GPU-based Parallelization of Growing Self-Organizing Networks
Among the many possible approaches for the parallelization of self-organizing
networks, and in particular of growing self-organizing networks, perhaps the
most common one is producing an optimized, parallel implementation of the
standard sequential algorithms reported in the literature. In this paper we
explore an alternative approach, based on a new algorithm variant specifically
designed to match the features of the large-scale, fine-grained parallelism of
GPUs, in which multiple input signals are processed at once. Comparative tests
have been performed, using both parallel and sequential implementations of the
new algorithm variant, in particular for a growing self-organizing network that
reconstructs surfaces from point clouds. The experimental results show that
this approach allows harnessing in a more effective way the intrinsic
parallelism that the self-organizing networks algorithms seem intuitively to
suggest, obtaining better performances even with networks of smaller size.Comment: 17 page
- …