Reduced hyperBF networks : practical optimization, regularization, and applications in bioinformatics.

A hyper basis function network (HyperBF) is a generalized radial basis function network (RBF) where the activation function is a radial function of a weighted distance. The local weighting of the distance accounts for the variation in local scaling and discriminative power along each feature. Such generalization makes HyperBF networks capable of interpolating decision functions with high accuracy. However, such complexity makes HyperBF networks susceptible to overfitting. Moreover, training a HyperBF network demands weights, centers and local scaling factors to be optimized simultaneously. In the case of a relatively large dataset with a large network structure, such optimization becomes computationally challenging. In this work, a new regularization method that performs soft local dimension reduction and weight decay is presented. The regularized HyperBF (Reduced HyperBF) network is shown to provide classification accuracy comparable to a Support Vector Machines (SVM) while requiring a significantly smaller network structure. Furthermore, the soft local dimension reduction is shown to be informative for ranking features based on their localized discriminative power. In addition, a practical training approach for constructing HyperBF networks is presented. This approach uses hierarchal clustering to initialize neurons followed by a gradient optimization using a scaled Rprop algorithm with a localized partial backtracking step (iSRprop). Experimental results on a number of datasets show a faster and smoother convergence than the regular Rprop algorithm. The proposed Reduced HyperBF network is applied to two problems in bioinformatics. The first is the detection of transcription start sites (TSS) in human DNA. A novel method for improving the accuracy of TSS recognition for recently published methods is proposed. This method incorporates a new metric feature based on oligonucleotide positional frequencies. The second application is the accurate classification of microarray samples. A new feature selection algorithm based on a Reduced HyperBF network is proposed. The method is applied to two microarray datasets and is shown to select a minimal subset of features with high discriminative information. The algorithm is compared to two widely used methods and is shown to provide competitive results. In both applications, the final Reduced HyperBF network is used for higher level analysis. Significant neurons can indicate subpopulations, while local active features provide insight into the characteristics of the subpopulation in specific and the whole class in general

Feature Extraction Methods for Character Recognition

Not Include

Statistical classification of images

Image classification is a burgeoning field of study. Despite the advances achieved in this camp, there is no general agreement about what is the most effective methods for the classification of digital images. This dissertation contributes to this line of research by developing different statistical methods aim to classifying digital images. In Chapter 1 we introduce basic concepts of image classification and review some results and methodologies proposed previously in the literature. In Chapter 2 we propose a method to classify images by their content. We are able to distinguish between landscape from non-landscape pictures by using three features obtained directly from images. We obtain better classification rates than those obtained by other authors dealing with similar kind of scene classification. In Chapter 3 we address the handwritten digit recognition. We suggest a set of intuitive features to perform the classification. Since the features are calculated with the binary image, we propose a novel technique to obtain the optimum threshold to binarize images, based on statistical concepts associated to the written trace of the digit. The classification is conducted by applying multivariate and probabilistic approaches, concluding that both methods provide similar results in terms of test-error rate (3.5%). In Chapter 4 we propose the application of Functional Data Analysis to analyze and classify images. While a limited number of authors have suggested the application of FDA for image classification [Florindo et al. (2010)], we suggest that this branch of statistics has represents a promising approach and offers several avenues for future research. We close the dissertation in Chapter 5 with a set of concluding remarks. Overall, the methods suggested in this dissertation are simple to apply, intuitive in their interpretation and their performance is comparable with other complex methods applied to the same problem. Moreover, the features suggested require less processing time than other methods (as support vector machine classifiers) and therefore require less computational capacityLa clasificación de imágenes es un campo de estudio de rápido crecimiento. A pesar de los avances logrados en esta área, no existe un acuerdo generalizado acerca de cuál es el método más eficaz para la clasificación de imágenes digitales. Esta tesis contribuye a esta línea de investigación mediante el desarrollo de diferentes métodos estadísticos que tienen como objetivo la clasificación de imágenes digitales. En el capítulo 1 se introduce los conceptos básicos de clasificación y se revisan algunos resultados de las metodologías propuestas previamente en la literatura. En el capítulo 2 se propone un método para clasificar las imágenes por su contenido. Somos capaz de distinguir entre una imagen de un paisaje de una que no lo es a partir del uso de tres variables obtenidas directamente de las imágenes. Obtenemos mejores tasas de clasificación que las alcanzadas por otros autores que han trabajado clasificación de escenas similares. En el capítulo 3 abordamos el reconocimiento de dígitos escritos a mano. Sugerimos una serie de variables intuitivas para llevar a cabo la clasificación. Dado que las variables se calculan con imágenes binarias, se propone una novedosa técnica para obtener el umbral óptimo para imágenes binarizadas, basado en los conceptos estadísticos asociados al trazo de escritura del dígito. La clasificación se lleva a cabo mediante la aplicación de métodos multivariantes y probabilísticos, concluyendo que ambos métodos proporcionan resultados similares en términos de tasa de error (3,5 %). En el capítulo 4 se propone la aplicación del Análisis Funcional de Datos para estudiar y clasificar imágenes digitales. Mientras que un número limitado de autores han sugerido la aplicación de ADF para la clasificación de la imagen [Florindo et al. (2010)], creemos que este rama de la estadística representa un enfoque prometedor y ofrece diversas alternativas para la investigación futura. Cerramos la tesis en el capítulo 5 con un conjunto de observaciones finales. En general, los métodos propuestos en esta tesis son fáciles de aplicar, intuitivos en su interpretación y su rendimiento es comparable con otros métodos complejos aplicados al mismo problema. Por otra parte, las características sugeridas requieren menos tiempo de procesamiento que otros métodos (como los clasificadores de técnicas de vector soporte

Visual pattern recognition using neural networks

Neural networks have been widely studied in a number of fields, such as neural architectures, neurobiology, statistics of neural network and pattern classification. In the field of pattern classification, neural network models are applied on numerous applications, for instance, character recognition, speech recognition, and object recognition. Among these, character recognition is commonly used to illustrate the feature and classification characteristics of neural networks. In this dissertation, the theoretical foundations of artificial neural networks are first reviewed and existing neural models are studied. The Adaptive Resonance Theory (ART) model is improved to achieve more reasonable classification results. Experiments in applying the improved model to image enhancement and printed character recognition are discussed and analyzed. We also study the theoretical foundation of Neocognitron in terms of feature extraction, convergence in training, and shift invariance. We investigate the use of multilayered perceptrons with recurrent connections as the general purpose modules for image operations in parallel architectures. The networks are trained to carry out classification rules in image transformation. The training patterns can be derived from user-defmed transformations or from loading the pair of a sample image and its target image when the prior knowledge of transformations is unknown. Applications of our model include image smoothing, enhancement, edge detection, noise removal, morphological operations, image filtering, etc. With a number of stages stacked up together we are able to apply a series of operations on the image. That is, by providing various sets of training patterns the system can adapt itself to the concatenated transformation. We also discuss and experiment in applying existing neural models, such as multilayered perceptron, to realize morphological operations and other commonly used imaging operations. Some new neural architectures and training algorithms for the implementation of morphological operations are designed and analyzed. The algorithms are proven correct and efficient. The proposed morphological neural architectures are applied to construct the feature extraction module of a personal handwritten character recognition system. The system was trained and tested with scanned image of handwritten characters. The feasibility and efficiency are discussed along with the experimental results

Pattern Recognition

A wealth of advanced pattern recognition algorithms are emerging from the interdiscipline between technologies of effective visual features and the human-brain cognition process. Effective visual features are made possible through the rapid developments in appropriate sensor equipments, novel filter designs, and viable information processing architectures. While the understanding of human-brain cognition process broadens the way in which the computer can perform pattern recognition tasks. The present book is intended to collect representative researches around the globe focusing on low-level vision, filter design, features and image descriptors, data mining and analysis, and biologically inspired algorithms. The 27 chapters coved in this book disclose recent advances and new ideas in promoting the techniques, technology and applications of pattern recognition

NBS monograph

From Introduction: "This report is the first of a series intended to provide a selective overview of research and development efforts and requirements in the somewhat overlapping fields of the computer and information sciences and technologies. The projected series of reports will attempt to outline the probable range of R & D activities in the computer and information sciences and technologies through selective reviews of the literature and to develop a reasonable consensus with respect to the opinions of workers in these and potentially related fields as to areas of continuing R & D concern for research program planning or review in these areas.

Learning and Example Selection for Object and Pattern Detection

This thesis presents a learning based approach for detecting classes of objects and patterns with variable image appearance but highly predictable image boundaries. It consists of two parts. In part one, we introduce our object and pattern detection approach using a concrete human face detection example. The approach first builds a distribution-based model of the target pattern class in an appropriate feature space to describe the target's variable image appearance. It then learns from examples a similarity measure for matching new patterns against the distribution-based target model. The approach makes few assumptions about the target pattern class and should therefore be fairly general, as long as the target class has predictable image boundaries. Because our object and pattern detection approach is very much learning-based, how well a system eventually performs depends heavily on the quality of training examples it receives. The second part of this thesis looks at how one can select high quality examples for function approximation learning tasks. We propose an {em active learning} formulation for function approximation, and show for three specific approximation function classes, that the active example selection strategy learns its target with fewer data samples than random sampling. We then simplify the original active learning formulation, and show how it leads to a tractable example selection paradigm, suitable for use in many object and pattern detection problems

Pattern Recognition

Pattern recognition is a very wide research field. It involves factors as diverse as sensors, feature extraction, pattern classification, decision fusion, applications and others. The signals processed are commonly one, two or three dimensional, the processing is done in real- time or takes hours and days, some systems look for one narrow object class, others search huge databases for entries with at least a small amount of similarity. No single person can claim expertise across the whole field, which develops rapidly, updates its paradigms and comprehends several philosophical approaches. This book reflects this diversity by presenting a selection of recent developments within the area of pattern recognition and related fields. It covers theoretical advances in classification and feature extraction as well as application-oriented works. Authors of these 25 works present and advocate recent achievements of their research related to the field of pattern recognition