53,504 research outputs found
Parallel Training of Neural Networks for Speech Recognition
Tato diplomová práce je zaměřena na paralelizaci trĂ©novánĂ neuronovĂ˝ch sĂtĂ pro rozpoznávánĂ Ĺ™eÄŤi. V rámci tĂ©to diplomovĂ© práce byly implementovány a porovnány dvÄ› strategie paralelizace. PrvnĂ strategiĂ je paralelizace dat s vyuĹľitĂm rozdÄ›lenĂ trĂ©novánĂ do nÄ›kolika POSIX vláken. Druhou strategiĂ je paralelizace uzlĹŻ s vyuĹľitĂm platformy pro obecnĂ© vĂ˝poÄŤty na grafickĂ˝ch kartách CUDA. V pĹ™ĂpadÄ› prvnĂ strategie bylo dosaĹľeno 4x urychlenĂ, v pĹ™ĂpadÄ› vyuĹľitĂ platformy CUDA bylo dosaĹľeno tĂ©měř 10x urychlenĂ. Pro trĂ©novánĂ byl pouĹľit algoritmus Stochastic Gradient Descent se zpÄ›tnĂ˝m šĂĹ™enĂm chyb. Po krátkĂ©m Ăşvodu následuje druhá kapitola práce, která je motivaÄŤnĂ a zasazuje probĂ©m do kontextu rozpoznávánĂ Ĺ™eÄŤi. TĹ™etĂ kapitola práce je teoretická a diskutuje neuronovĂ© sĂtÄ› a metodu trĂ©novánĂ. NásledujĂcĂ kapitoly jsou zaměřenĂ© na návrh a implementaci a popisujĂ iterativnĂ vĂ˝voj tohoto projektu. PoslednĂ obsáhlá kapitola popisuje testovacĂ systĂ©m a uvádĂ vĂ˝sledky provedenĂ˝ch experimentĹŻ. V závÄ›ru jsou krátce zhodnoceny dosaĹľenĂ© vĂ˝sledky a nastĂnÄ›na perspektiva dalšĂho vĂ˝voje projektu.This thesis deals with different parallelizations of training procedure for artificial neural networks. The networks are trained as phoneme-state acoustic descriptors for speech recognition. Two effective parallelization strategies were implemented and compared. The first strategy is data parallelization, where the training is split into several POSIX threads. The second strategy is node parallelization, which uses CUDA framework for general purpose computing on modern graphic cards. The first strategy showed a 4x speed-up, while using the second strategy we observed nearly 10x speed-up. The Stochastic Gradient Descent algorithm with error backpropagation was used for the training. After a short introduction, the second chapter of this thesis shows the motivation and introduces the neural networks into the context of speech recognition. The third chapter is theoretical, the anatomy of a neural network and the used training method are discussed. The following chapters are focused on the design and implementation of the project, while the phases of the iterative development are described. The last extensive chapter describes the setup of the testing system and reports the experimental results. Finally, the obtained results are concluded and the possible extensions of the project are proposed.
Visual pattern recognition using neural networks
Neural networks have been widely studied in a number of fields, such as neural architectures, neurobiology, statistics of neural network and pattern classification. In the field of pattern classification, neural network models are applied on numerous applications, for instance, character recognition, speech recognition, and object recognition. Among these, character recognition is commonly used to illustrate the feature and classification characteristics of neural networks.
In this dissertation, the theoretical foundations of artificial neural networks are first reviewed and existing neural models are studied. The Adaptive Resonance Theory (ART) model is improved to achieve more reasonable classification results. Experiments in applying the improved model to image enhancement and printed character recognition are discussed and analyzed. We also study the theoretical foundation of Neocognitron in terms of feature extraction, convergence in training, and shift invariance.
We investigate the use of multilayered perceptrons with recurrent connections as the general purpose modules for image operations in parallel architectures. The networks are trained to carry out classification rules in image transformation. The training patterns can be derived from user-defmed transformations or from loading the pair of a sample image and its target image when the prior knowledge of transformations is unknown. Applications of our model include image smoothing, enhancement, edge detection, noise removal, morphological operations, image filtering, etc. With a number of stages stacked up together we are able to apply a series of operations on the image. That is, by providing various sets of training patterns the system can adapt itself to the concatenated transformation. We also discuss and experiment in applying existing neural models, such as multilayered perceptron, to realize morphological operations and other commonly used imaging operations.
Some new neural architectures and training algorithms for the implementation of morphological operations are designed and analyzed. The algorithms are proven correct and efficient. The proposed morphological neural architectures are applied to construct the feature extraction module of a personal handwritten character recognition system. The system was trained and tested with scanned image of handwritten characters. The feasibility and efficiency are discussed along with the experimental results
- …