12,203 research outputs found

    A Survey of Parallel Data Mining

    Get PDF
    With the fast, continuous increase in the number and size of databases, parallel data mining is a natural and cost-effective approach to tackle the problem of scalability in data mining. Recently there has been a considerable research on parallel data mining. However, most projects focus on the parallelization of a single kind of data mining algorithm/paradigm. This paper surveys parallel data mining with a broader perspective. More precisely, we discuss the parallelization of data mining algorithms of four knowledge discovery paradigms, namely rule induction, instance-based learning, genetic algorithms and neural networks. Using the lessons learned from this discussion, we also derive a set of heuristic principles for designing efficient parallel data mining algorithms

    Распараллеливание алгоритмов функционирования классификатора со случайными подпространствами

    Get PDF
    Нейронные сети являются достаточно популярным средством решения многих задач искуственного интеллекта. В то же время, любые аппаратные реализации нейросетевых архитектур очень быстро устаревают благодаря стремительному развитию вычислительной техники. Таким образом, большинство исследователей стремятся использовать в первую очередь программные реализации алгоритмов, что делает актуальным распараллеливание функционирования нейросетевых систем. В работе рассматривается нейросетевой классификатор со случайными подпространствами и предлагаются алгоритмы распараллеливания его основных операций.Neural networks are quite popular for solving many tasks of artificial intelligence. At the same time, any hardware implementation of some neural architecture becomes obsolete really fast due to rapid development of semiconductor industry. Thus, most researchers first of all tend to use software algorithm implementations, which makes parallelization of neural network algorithms quite attractive. This article analyses neural random subspace classifier and suggests algorithms for parallelization off it’s basic operations

    Single stream parallelization of generalized LSTM-like RNNs on a GPU

    Full text link
    Recurrent neural networks (RNNs) have shown outstanding performance on processing sequence data. However, they suffer from long training time, which demands parallel implementations of the training procedure. Parallelization of the training algorithms for RNNs are very challenging because internal recurrent paths form dependencies between two different time frames. In this paper, we first propose a generalized graph-based RNN structure that covers the most popular long short-term memory (LSTM) network. Then, we present a parallelization approach that automatically explores parallelisms of arbitrary RNNs by analyzing the graph structure. The experimental results show that the proposed approach shows great speed-up even with a single training stream, and further accelerates the training when combined with multiple parallel training streams.Comment: Accepted by the 40th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 201

    Accelerating recurrent neural network training using sequence bucketing and multi-GPU data parallelization

    Full text link
    An efficient algorithm for recurrent neural network training is presented. The approach increases the training speed for tasks where a length of the input sequence may vary significantly. The proposed approach is based on the optimal batch bucketing by input sequence length and data parallelization on multiple graphical processing units. The baseline training performance without sequence bucketing is compared with the proposed solution for a different number of buckets. An example is given for the online handwriting recognition task using an LSTM recurrent neural network. The evaluation is performed in terms of the wall clock time, number of epochs, and validation loss value.Comment: 4 pages, 5 figures, Comments, 2016 IEEE First International Conference on Data Stream Mining & Processing (DSMP), Lviv, 201

    A Multi-signal Variant for the GPU-based Parallelization of Growing Self-Organizing Networks

    Full text link
    Among the many possible approaches for the parallelization of self-organizing networks, and in particular of growing self-organizing networks, perhaps the most common one is producing an optimized, parallel implementation of the standard sequential algorithms reported in the literature. In this paper we explore an alternative approach, based on a new algorithm variant specifically designed to match the features of the large-scale, fine-grained parallelism of GPUs, in which multiple input signals are processed at once. Comparative tests have been performed, using both parallel and sequential implementations of the new algorithm variant, in particular for a growing self-organizing network that reconstructs surfaces from point clouds. The experimental results show that this approach allows harnessing in a more effective way the intrinsic parallelism that the self-organizing networks algorithms seem intuitively to suggest, obtaining better performances even with networks of smaller size.Comment: 17 page
    corecore