Pattern recognition with discrete and mixed data : theory and practice

This thesis is devoted to aspects related to the analysis of medical data bases in the context of pattern recognition. It contains both theoretical aspects and practical applications and its scope includes questions and problems that arise when applying pattern recognition methods and techniques to this type of data. The goal of the application of statistical pattern recognition techniques to medical records, is the classification of the (disease) patterns that may be present in such records in terms of the information they contain. Typically, a medical record contains a description of history, symptoms, results from laboratory tests, signals, etc., all related to a given patient, i.e. all the information normally required by a physician when making a diagnosis and/or a prognosis. Pattern recognition may be used in order to obtain procedures (computer implemented algorithms) to assign diagnostic or prognostic classes to a given patient, on the basis of information also used by a physician. These procedures are not intended to replace but to assist the physician in the decision making process. The procedures are called classifiers or discriminants and the symptoms, signals, etc., are called features. Each individual record is termed an object, and a collection of objects with qualitatively and/or quantitatively similar characteristics, as established by an expert, is called a class. It should be clear that pattern recognition can be applied to a wide variety of areas and problems, of which (computer-aided) medical decision making is just an example. In order to arrive at a classifier and restricting ourselves to what is called supervised learning, a set of objects known a-priori to belong to two or more classes (depending on the problem at hand) is needed. In this set, each object must be represented by a group of features and have a class assigned to it. The role of medical data bases is now clear: they are the set of objects required for supervised learning

Wavelet methods in speech recognition

In this thesis, novel wavelet techniques are developed to improve parametrization of speech signals prior to classification. It is shown that non-linear operations carried out in the wavelet domain improve the performance of a speech classifier and consistently outperform classical Fourier methods. This is because of the localised nature of the wavelet, which captures correspondingly well-localised time-frequency features within the speech signal. Furthermore, by taking advantage of the approximation ability of wavelets, efficient representation of the non-stationarity inherent in speech can be achieved in a relatively small number of expansion coefficients. This is an attractive option when faced with the so-called 'Curse of Dimensionality' problem of multivariate classifiers such as Linear Discriminant Analysis (LDA) or Artificial Neural Networks (ANNs). Conventional time-frequency analysis methods such as the Discrete Fourier Transform either miss irregular signal structures and transients due to spectral smearing or require a large number of coefficients to represent such characteristics efficiently. Wavelet theory offers an alternative insight in the representation of these types of signals. As an extension to the standard wavelet transform, adaptive libraries of wavelet and cosine packets are introduced which increase the flexibility of the transform. This approach is observed to be yet more suitable for the highly variable nature of speech signals in that it results in a time-frequency sampled grid that is well adapted to irregularities and transients. They result in a corresponding reduction in the misclassification rate of the recognition system. However, this is necessarily at the expense of added computing time. Finally, a framework based on adaptive time-frequency libraries is developed which invokes the final classifier to choose the nature of the resolution for a given classification problem. The classifier then performs dimensionaIity reduction on the transformed signal by choosing the top few features based on their discriminant power. This approach is compared and contrasted to an existing discriminant wavelet feature extractor. The overall conclusions of the thesis are that wavelets and their relatives are capable of extracting useful features for speech classification problems. The use of adaptive wavelet transforms provides the flexibility within which powerful feature extractors can be designed for these types of application

On discriminative semi-supervised incremental learning with a multi-view perspective for image concept modeling

This dissertation presents the development of a semi-supervised incremental learning framework with a multi-view perspective for image concept modeling. For reliable image concept characterization, having a large number of labeled images is crucial. However, the size of the training set is often limited due to the cost required for generating concept labels associated with objects in a large quantity of images. To address this issue, in this research, we propose to incrementally incorporate unlabeled samples into a learning process to enhance concept models originally learned with a small number of labeled samples. To tackle the sub-optimality problem of conventional techniques, the proposed incremental learning framework selects unlabeled samples based on an expected error reduction function that measures contributions of the unlabeled samples based on their ability to increase the modeling accuracy. To improve the convergence property of the proposed incremental learning framework, we further propose a multi-view learning approach that makes use of multiple features such as color, texture, etc., of images when including unlabeled samples. For robustness to mismatches between training and testing conditions, a discriminative learning algorithm, namely a kernelized maximal- figure-of-merit (kMFoM) learning approach is also developed. Combining individual techniques, we conduct a set of experiments on various image concept modeling problems, such as handwritten digit recognition, object recognition, and image spam detection to highlight the effectiveness of the proposed framework.PhDCommittee Chair: Lee, Chin-Hui; Committee Member: Clements, Mark; Committee Member: Lee, Hsien-Hsin; Committee Member: McClellan, James; Committee Member: Yuan, Min

Population-level neural coding for higher cognition

Higher cognition encompasses advanced mental processes that enable complex thinking, decision-making, problem-solving, and abstract reasoning. These functions involve integrating information from multiple sensory modalities and organizing action plans based on the abstraction of past information. The neural activity underlying these functions is often complex, and the contribution of single neurons in supporting population-level representations of cognitive variables is not yet clear. In this thesis, I investigated the neural mechanisms underlying higher cognition in higher-order brain regions with single-neuron resolution in human and non-human primates performing working memory tasks. I aimed to understand how representations are arranged and how neurons contribute to the population code. In the first manuscript, I investigated the population-level neural coding for the maintenance of numbers in working memory within the parietal association cortex. By analyzing intra-operative intracranial micro-electrode array recording data, I uncovered distinct representations for numbers in both symbolic and nonsymbolic formats. In the second manuscript, I delved deeper into the neuronal organizing principles of population coding to address the ongoing debate surrounding memory maintenance mechanisms. I unveiled sparse structures in the neuronal implementation of representations and identified biologically meaningful components that can be directly communicated to downstream neurons. These components were linked to subpopulations of neurons with distinct physiological properties and temporal dynamics, enabling the active maintenance of working memory while resisting distraction. Lastly, using an artificial neural network model, I demonstrated that the sparse implementation of temporally modulated working memory representations is preferred in recurrently connected neural populations such as the prefrontal cortex. In summary, this thesis provides a comprehensive investigation of higher cognition in higher-order brain regions, focusing on working memory tasks involving numerical stimuli. By examining neural population coding and unveiling sparse structures in the neuronal implementation of representations, our findings contribute to a deeper understanding of the mechanisms underlying working memory and higher cognitive functions

Algorithms for enhancing pattern separability, feature selection and incremental learning with applications to gas sensing electronic nose systems

Three major issues in pattern recognition and data analysis have been addressed in this study and applied to the problem of identification of volatile organic compounds (VOC) for gas sensing applications. Various approaches have been proposed and discussed. These approaches are not only applicable to the VOC identification, but also to a variety of pattern recognition and data analysis problems. In particular, (1) enhancing pattern separability for challenging classification problems, (2) optimum feature selection problem, and (3) incremental learning for neural networks have been investigated;Three different approaches are proposed for enhancing pattern separability for classification of closely spaced, or possibly overlapping clusters. In the neurofuzzy approach, a fuzzy inference system that considers the dynamic ranges of individual features is developed. Feature range stretching (FRS) is introduced as an alternative approach for increasing intercluster distances by mapping the tight dynamic range of each feature to a wider range through a nonlinear function. Finally, a third approach, nonlinear cluster transformation (NCT), is proposed, which increases intercluster distances while preserving intracluster distances. It is shown that NCT achieves comparable, or better, performance than the other two methods at a fraction of the computational burden. The implementation issues and relative advantages and disadvantages of these approaches are systematically investigated;Selection of optimum features is addressed using both a decision tree based approach, and a wrapper approach. The hill-climb search based wrapper approach is applied for selection of the optimum features for gas sensing problems;Finally, a new method, Learn++, is proposed that gives classification algorithms, the capability of incrementally learning from new data. Learn++ is introduced for incremental learning of new data, when the original database is no longer available. Learn++ algorithm is based on strategically combining an ensemble of classifiers, each of which is trained to learn only a small portion of the pattern space. Furthermore, Learn++ is capable of learning new data even when new classes are introduced, and it also features a built-in mechanism for estimating the reliability of its classification decision;All proposed methods are explained in detail and simulation results are discussed along with directions for future work

Combining classification algorithms

Dissertação de Doutoramento em Ciência de Computadores apresentada à Faculdade de Ciências da Universidade do PortoA capacidade de um algoritmo de aprendizagem induzir, para um determinado problema, uma boa generalização depende da linguagem de representação usada para generalizar os exemplos. Como diferentes algoritmos usam diferentes linguagens de representação e estratégias de procura, são explorados espaços diferentes e são obtidos resultados diferentes. O problema de encontrar a representação mais adequada para o problema em causa, é uma área de investigação bastante activa. Nesta dissertação, em vez de procurar métodos que fazem o ajuste aos dados usando uma única linguagem de representação, apresentamos uma família de algoritmos, sob a designação genérica de Generalização em Cascata, onde o espaço de procura contem modelos que utilizam diferentes linguagens de representação. A ideia básica do método consiste em utilizar os algoritmos de aprendizagem em sequência. Em cada iteração ocorre um processo com dois passos. No primeiro passo, um classificador constrói um modelo. No segundo passo, o espaço definido pelos atributos é estendido pela inserção de novos atributos gerados utilizando este modelo. Este processo de construção de novos atributos constrói atributos na linguagem de representação do classificador usado para construir o modelo. Se posteriormente na sequência, um classificador utiliza um destes novos atributos para construir o seu modelo, a sua capacidade de representação foi estendida. Desta forma as restrições da linguagem de representação dosclassificadores utilizados a mais alto nível na sequência, são relaxadas pela incorporação de termos da linguagem derepresentação dos classificadores de base. Esta é a metodologia base subjacente ao sistema Ltree e à arquitecturada Generalização em Cascata.O método é apresentado segundo duas perspectivas. Numa primeira parte, é apresentado como uma estratégia paraconstruir árvores de decisão multivariadas. É apresentado o sistema Ltree que utiliza como operador para a construção de atributos um discriminante linear. ..