Visual pattern recognition using neural networks

Neural networks have been widely studied in a number of fields, such as neural architectures, neurobiology, statistics of neural network and pattern classification. In the field of pattern classification, neural network models are applied on numerous applications, for instance, character recognition, speech recognition, and object recognition. Among these, character recognition is commonly used to illustrate the feature and classification characteristics of neural networks. In this dissertation, the theoretical foundations of artificial neural networks are first reviewed and existing neural models are studied. The Adaptive Resonance Theory (ART) model is improved to achieve more reasonable classification results. Experiments in applying the improved model to image enhancement and printed character recognition are discussed and analyzed. We also study the theoretical foundation of Neocognitron in terms of feature extraction, convergence in training, and shift invariance. We investigate the use of multilayered perceptrons with recurrent connections as the general purpose modules for image operations in parallel architectures. The networks are trained to carry out classification rules in image transformation. The training patterns can be derived from user-defmed transformations or from loading the pair of a sample image and its target image when the prior knowledge of transformations is unknown. Applications of our model include image smoothing, enhancement, edge detection, noise removal, morphological operations, image filtering, etc. With a number of stages stacked up together we are able to apply a series of operations on the image. That is, by providing various sets of training patterns the system can adapt itself to the concatenated transformation. We also discuss and experiment in applying existing neural models, such as multilayered perceptron, to realize morphological operations and other commonly used imaging operations. Some new neural architectures and training algorithms for the implementation of morphological operations are designed and analyzed. The algorithms are proven correct and efficient. The proposed morphological neural architectures are applied to construct the feature extraction module of a personal handwritten character recognition system. The system was trained and tested with scanned image of handwritten characters. The feasibility and efficiency are discussed along with the experimental results

Bibliometric Mapping of the Computational Intelligence Field

In this paper, a bibliometric study of the computational intelligence field is presented. Bibliometric maps showing the associations between the main concepts in the field are provided for the periods 1996–2000 and 2001–2005. Both the current structure of the field and the evolution of the field over the last decade are analyzed. In addition, a number of emerging areas in the field are identified. It turns out that computational intelligence can best be seen as a field that is structured around four important types of problems, namely control problems, classification problems, regression problems, and optimization problems. Within the computational intelligence field, the neural networks and fuzzy systems subfields are fairly intertwined, whereas the evolutionary computation subfield has a relatively independent position.neural networks;bibliometric mapping;fuzzy systems;bibliometrics;computational intelligence;evolutionary computation

Combined Wavelet-neural Clasifier For Power Distribution Systems

Tez (Yüksek Lisans) -- İstanbul Teknik Üniversitesi, Fen Bilimleri Enstitüsü, 2002Thesis (M.Sc.) -- İstanbul Technical University, Institute of Science and Technology, 2002Bu çalışmada, dağıtım sistemlerinde hibrid “Dalgacık-Yapay Sinir ağı (YSA) tabanlı” bir yaklaşımla arıza sınıflama işlemi gerçeklenmiştir. 34.5 kV “Sağmalcılar-Maltepe” dağıtım sistemi PSCAD/EMTDC yazılımı kullanılarak arıza sınıflayıcı için gereken veri üretilmiştir. Tezin amacı, on farklı kısa-devre sistem arızalarını tanımlayabilecek bir sınıflayıcı tasarlamaktır. Sistemde kullanılan arıza işaretleri 5 kHZ lik örnekleme frekansı ile üretilmiştir. Farklı arıza noktaları ve farklı arıza oluşum açılarındaki hat-akımları ve hat-toprak gerilimlerini içeren sistem arızaları ile bir veritabanı oluşturulmuştur. “Çoklu-çözünürlük işaret ayrıştırma” tekniği kullanılarak altı-kanal akım ve gerilim örneklerinden karakteristik bigi çıkarılmıştır. PSCAD/EMTDC ile üretilen veri bu şekilde bir ön islemden geçirildikten sonra YSA-tabanlı bir yapı ile sınıflama islemi gerçekleştirilmiştir. Bu yapının görevi çeşitli sistem ve arıza koşullarını kapsayan karmaşık arıza sınıflama problemini çözebilmektir. Bu çalışmada, Kohonen’in öğrenme algoritmasını kullanan bir “Kendine-Organize harita” ile “eğitilebilen vektör kuantalama” teknikleri kullanılmıştır. Bu “dalgacık-sinir ağı” tabanlı arıza sınıflayıcı ile eğitim kümesi için % 99-100 arasında ve sınıflayıcıya daha önce hiç verilmemiş test kümesi ile de %85-92 arasında sınıflama oranları elde edilmiştir. Elde edilen başarım oranları literatürdeki sonuçlara yakındır. Geliştirilen birleşik “dalgacık-sinir ağı” tabanlı sınıflayıcı elektrik dağıtım sistemlerindeki arızaların belirlenmesinde iyi sonuçlar vermiş ve iyi bir performans sağlamıştır.In this study an integrated design of fault classifier in a distribution system by using a hybrid “Wavelet- Artificial neural network (ANN) based” approach is implemented. Data for the fault classifier is produced by using PSCAD/EMTDC simulation program on 34.5 kV “Sagmalcılar-Maltepe” distribution system in Istanbul. The objective is to design a classifier capable of recognizing ten classes of three-phase system faults. The signals are generated at an equivalent sampling rate of 5 KHz per channel. A database of line currents and line-to-ground voltages is built up including system faults at different fault inception angles and fault locations. The characteristic information over six-channel of current and voltage samples is extracted by the “wavelet multi-resolution analysis” technique, which is a preprocessing unit to obtain a small size of interpretable features from the raw data. After preprocessing the raw data, an ANN-based tool was employed for classification task. The main idea in this approach is solving the complex fault (three-phase short-circuit) classification problem under various system and fault conditions. In this project, a self-organizing map, with Kohonen’s learning algorithm and type-one learning vector quantization technique is implemented into the fault classification study. The performance of the wavelet-neural fault classification scheme is found to be around “99-100%” for the training data and around “85-92%” for the test data, which the classifier has not been trained on. This result is comparable to the studied fault classifiers in the literature. Combined wavelet-neural classifier showed a promising future to identify the faults in electric distribution systemsYüksek LisansM.Sc

Dynamic non-linear system modelling using wavelet-based soft computing techniques

The enormous number of complex systems results in the necessity of high-level and cost-efficient modelling structures for the operators and system designers. Model-based approaches offer a very challenging way to integrate a priori knowledge into the procedure. Soft computing based models in particular, can successfully be applied in cases of highly nonlinear problems. A further reason for dealing with so called soft computational model based techniques is that in real-world cases, many times only partial, uncertain and/or inaccurate data is available. Wavelet-Based soft computing techniques are considered, as one of the latest trends in system identification/modelling. This thesis provides a comprehensive synopsis of the main wavelet-based approaches to model the non-linear dynamical systems in real world problems in conjunction with possible twists and novelties aiming for more accurate and less complex modelling structure. Initially, an on-line structure and parameter design has been considered in an adaptive Neuro- Fuzzy (NF) scheme. The problem of redundant membership functions and consequently fuzzy rules is circumvented by applying an adaptive structure. The growth of a special type of Fungus (Monascus ruber van Tieghem) is examined against several other approaches for further justification of the proposed methodology. By extending the line of research, two Morlet Wavelet Neural Network (WNN) structures have been introduced. Increasing the accuracy and decreasing the computational cost are both the primary targets of proposed novelties. Modifying the synoptic weights by replacing them with Linear Combination Weights (LCW) and also imposing a Hybrid Learning Algorithm (HLA) comprising of Gradient Descent (GD) and Recursive Least Square (RLS), are the tools utilised for the above challenges. These two models differ from the point of view of structure while they share the same HLA scheme. The second approach contains an additional Multiplication layer, plus its hidden layer contains several sub-WNNs for each input dimension. The practical superiority of these extensions is demonstrated by simulation and experimental results on real non-linear dynamic system; Listeria Monocytogenes survival curves in Ultra-High Temperature (UHT) whole milk, and consolidated with comprehensive comparison with other suggested schemes. At the next stage, the extended clustering-based fuzzy version of the proposed WNN schemes, is presented as the ultimate structure in this thesis. The proposed Fuzzy Wavelet Neural network (FWNN) benefitted from Gaussian Mixture Models (GMMs) clustering feature, updated by a modified Expectation-Maximization (EM) algorithm. One of the main aims of this thesis is to illustrate how the GMM-EM scheme could be used not only for detecting useful knowledge from the data by building accurate regression, but also for the identification of complex systems. The structure of FWNN is based on the basis of fuzzy rules including wavelet functions in the consequent parts of rules. In order to improve the function approximation accuracy and general capability of the FWNN system, an efficient hybrid learning approach is used to adjust the parameters of dilation, translation, weights, and membership. Extended Kalman Filter (EKF) is employed for wavelet parameters adjustment together with Weighted Least Square (WLS) which is dedicated for the Linear Combination Weights fine-tuning. The results of a real-world application of Short Time Load Forecasting (STLF) further re-enforced the plausibility of the above technique

Hybrid artificial intelligence algorithms for short-term load and price forecasting in competitive electric markets

The liberalization and deregulation of electric markets forced the various participants to accommodate several challenges, including: a considerable accumulation of new generation capacity from renewable sources (fundamentally wind energy), the unpredictability associated with these new forms of generation and new consumption patterns, contributing to further electricity prices volatility (e.g. the Iberian market). Given the competitive framework in which market participants operate, the existence of efficient computational forecasting techniques is a distinctive factor. Based on these forecasts a suitable bidding strategy and an effective generation systems operation planning is achieved, together with an improved installed transmission capacity exploitation, results in maximized profits, all this contributing to a better energy resources utilization. This dissertation presents a new hybrid method for load and electricity prices forecasting, for one day ahead time horizon. The optimization scheme presented in this method, combines the efforts from different techniques, notably artificial neural networks, several optimization algorithms and wavelet transform. The method’s validation was made using different real case studies. The subsequent comparison (accuracy wise) with published results, in reference journals, validated the proposed hybrid method suitability.O processo de liberalização e desregulação dos mercados de energia elétrica, obrigou os diversos participantes a acomodar uma série de desafios, entre os quais: a acumulação considerável de nova capacidade de geração proveniente de origem renovável (fundamentalmente energia eólica), a imprevisibilidade associada a estas novas formas de geração e novos padrões de consumo. Resultando num aumento da volatilidade associada aos preços de energia elétrica (como é exemplo o mercado ibérico). Dado o quadro competitivo em que os agentes de mercado operam, a existência de técnicas computacionais de previsão eficientes, constituí um fator diferenciador. É com base nestas previsões que se definem estratégias de licitação e se efetua um planeamento da operação eficaz dos sistemas de geração que, em conjunto com um melhor aproveitamento da capacidade de transmissão instalada, permite maximizar os lucros, realizando ao mesmo tempo um melhor aproveitamento dos recursos energéticos. Esta dissertação apresenta um novo método híbrido para a previsão da carga e dos preços da energia elétrica, para um horizonte temporal a 24 horas. O método baseia-se num esquema de otimização que reúne os esforços de diferentes técnicas, nomeadamente redes neuronais artificiais, diversos algoritmos de otimização e da transformada de wavelet. A validação do método foi feita em diferentes casos de estudo reais. A posterior comparação com resultados já publicados em revistas de referência, revelou um excelente desempenho do método hibrido proposto

Features and neural net recognition strategies for hand printed digits

The thesis goal is to develop a computer system for hand printed digit recognition based on an investigation into various feature extractors and neural network strategies. Features such as subwindow pixel summation, moments, and orientation vectors will be among those investigated. Morphological thinning of characters prior to feature extraction will be used to assess the impact on network training and testing. Different strategies for implementing a multilayer perceptron neural network will be investigated. A high-level language called MatLab will be used for neural network algorithm development and quick prototyping. The feature extractors will be developed to operate on small (less than or equal to 256 bits) binary hand printed digits (0, 1, 2, 3, 4, 5, 6, 7, 8, 9)

Colour for the Advancement of Deep Learning in Computer Vision

This thesis explores several research areas for Deep Learning related to computer vision concerning colours. First, this thesis considers one of the most long standing challenges that has remained for Deep Learning which is, how can Deep Learning algorithms learn successfully without using human annotated data? To that end, this thesis examines using colours in images to learn meaningful representations of vision as a substitute for learning from hand-annotated data. Second, is another related topic to the previous, which is the application of Deep Learning to automate the complex graphics task of image colourisation, which is the process of adding colours to black and white images. Third, this thesis explores colour spaces and how the representations of colours in images affect the performance in Deep Learning models