Knowledge-Based Analysis of Genomic Expression Data by Using Different Machine Learning Algorithms for the Purpose of Diagnostic, Prognostic or Therapeutic Application

With more and more biological information generated, the most pressing task of bioinformatics has become to analyze and interpret various types of data, including nucleotide and amino acid sequences, protein structures, gene expression profiling and so on. In this dissertation, we apply the data mining techniques of feature generation, feature selection, and feature integration with learning algorithms to tackle the problems of disease phenotype classification, clinical outcome and patient survival prediction from gene expression profiles. We analyzed the effect of batch noise in microarray data on the performance of classification. Batchmatch, a batch adjusting algorithm based on double scaling method is advantageous over Combat, another batch correcting algorithm based on the empirical bayes frame work. In order to identify genes associated with disease phenotype classification or patient survival prediction from gene expression data, we compared and analyzed the performance of five feature selection algorithms. Our observations from these studies indicated that Gainratio algorithm performs better and more consistently over the other algorithms studied. When it comes to performance metric to choose the best classifiers, MCC gives unbiased performance results over accuracy in some endpoints, where class imbalance is more. In the aspect of classification algorithms, no single algorithm is absolutely superior to all others, though SVM achieved fairly good results in most endpoints. Naive bayes algorithm also performed well in some endpoints. Overall, from the total 65 models we reported (5 top models for 13 end points) SVM and SMO (a variant of SVM) dominate mostly, also the linear kernel performed well over RBF in our binary classifications

Prediction of catalytic residues using Support Vector Machine with selected protein sequence and structural properties

BACKGROUND: The number of protein sequences deriving from genome sequencing projects is outpacing our knowledge about the function of these proteins. With the gap between experimentally characterized and uncharacterized proteins continuing to widen, it is necessary to develop new computational methods and tools for functional prediction. Knowledge of catalytic sites provides a valuable insight into protein function. Although many computational methods have been developed to predict catalytic residues and active sites, their accuracy remains low, with a significant number of false positives. In this paper, we present a novel method for the prediction of catalytic sites, using a carefully selected, supervised machine learning algorithm coupled with an optimal discriminative set of protein sequence conservation and structural properties. RESULTS: To determine the best machine learning algorithm, 26 classifiers in the WEKA software package were compared using a benchmarking dataset of 79 enzymes with 254 catalytic residues in a 10-fold cross-validation analysis. Each residue of the dataset was represented by a set of 24 residue properties previously shown to be of functional relevance, as well as a label {+1/-1} to indicate catalytic/non-catalytic residue. The best-performing algorithm was the Sequential Minimal Optimization (SMO) algorithm, which is a Support Vector Machine (SVM). The Wrapper Subset Selection algorithm further selected seven of the 24 attributes as an optimal subset of residue properties, with sequence conservation, catalytic propensities of amino acids, and relative position on protein surface being the most important features. CONCLUSION: The SMO algorithm with 7 selected attributes correctly predicted 228 of the 254 catalytic residues, with an overall predictive accuracy of more than 86%. Missing only 10.2% of the catalytic residues, the method captures the fundamental features of catalytic residues and can be used as a "catalytic residue filter" to facilitate experimental identification of catalytic residues for proteins with known structure but unknown function

Smartphone-based human activity recognition

Cotutela Universitat Politècnica de Catalunya i Università degli Studi di GenovaHuman Activity Recognition (HAR) is a multidisciplinary research field that aims to gather data regarding people's behavior and their interaction with the environment in order to deliver valuable context-aware information. It has nowadays contributed to develop human-centered areas of study such as Ambient Intelligence and Ambient Assisted Living, which concentrate on the improvement of people's Quality of Life. The first stage to accomplish HAR requires to make observations from ambient or wearable sensor technologies. However, in the second case, the search for pervasive, unobtrusive, low-powered, and low-cost devices for achieving this challenging task still has not been fully addressed. In this thesis, we explore the use of smartphones as an alternative approach for performing the identification of physical activities. These self-contained devices, which are widely available in the market, are provided with embedded sensors, powerful computing capabilities and wireless communication technologies that make them highly suitable for this application. This work presents a series of contributions regarding the development of HAR systems with smartphones. In the first place we propose a fully operational system that recognizes in real-time six physical activities while also takes into account the effects of postural transitions that may occur between them. For achieving this, we cover some research topics from signal processing and feature selection of inertial data, to Machine Learning approaches for classification. We employ two sensors (the accelerometer and the gyroscope) for collecting inertial data. Their raw signals are the input of the system and are conditioned through filtering in order to reduce noise and allow the extraction of informative activity features. We also emphasize on the study of Support Vector Machines (SVMs), which are one of the state-of-the-art Machine Learning techniques for classification, and reformulate various of the standard multiclass linear and non-linear methods to find the best trade off between recognition performance, computational costs and energy requirements, which are essential aspects in battery-operated devices such as smartphones. In particular, we propose two multiclass SVMs for activity classification:one linear algorithm which allows to control over dimensionality reduction and system accuracy; and also a non-linear hardware-friendly algorithm that only uses fixed-point arithmetic in the prediction phase and enables a model complexity reduction while maintaining the system performance. The efficiency of the proposed system is verified through extensive experimentation over a HAR dataset which we have generated and made publicly available. It is composed of inertial data collected from a group of 30 participants which performed a set of common daily activities while carrying a smartphone as a wearable device. The results achieved in this research show that it is possible to perform HAR in real-time with a precision near 97\% with smartphones. In this way, we can employ the proposed methodology in several higher-level applications that require HAR such as ambulatory monitoring of the disabled and the elderly during periods above five days without the need of a battery recharge. Moreover, the proposed algorithms can be adapted to other commercial wearable devices recently introduced in the market (e.g. smartwatches, phablets, and glasses). This will open up new opportunities for developing practical and innovative HAR applications.El Reconocimiento de Actividades Humanas (RAH) es un campo de investigación multidisciplinario que busca recopilar información sobre el comportamiento de las personas y su interacción con el entorno con el propósito de ofrecer información contextual de alta significancia sobre las acciones que ellas realizan. Recientemente, el RAH ha contribuido en el desarrollo de áreas de estudio enfocadas a la mejora de la calidad de vida del hombre tales como: la inteligència ambiental (Ambient Intelligence) y la vida cotidiana asistida por el entorno para personas dependientes (Ambient Assisted Living). El primer paso para conseguir el RAH consiste en realizar observaciones mediante el uso de sensores fijos localizados en el ambiente, o bien portátiles incorporados de forma vestible en el cuerpo humano. Sin embargo, para el segundo caso, aún se dificulta encontrar dispositivos poco invasivos, de bajo consumo energético, que permitan ser llevados a cualquier lugar, y de bajo costo. En esta tesis, nosotros exploramos el uso de teléfonos móviles inteligentes (Smartphones) como una alternativa para el RAH. Estos dispositivos, de uso cotidiano y fácilmente asequibles en el mercado, están dotados de sensores embebidos, potentes capacidades de cómputo y diversas tecnologías de comunicación inalámbrica que los hacen apropiados para esta aplicación. Nuestro trabajo presenta una serie de contribuciones en relación al desarrollo de sistemas para el RAH con Smartphones. En primera instancia proponemos un sistema que permite la detección de seis actividades físicas en tiempo real y que, además, tiene en cuenta las transiciones posturales que puedan ocurrir entre ellas. Con este fin, hemos contribuido en distintos ámbitos que van desde el procesamiento de señales y la selección de características, hasta algoritmos de Aprendizaje Automático (AA). Nosotros utilizamos dos sensores inerciales (el acelerómetro y el giroscopio) para la captura de las señales de movimiento de los usuarios. Estas han de ser procesadas a través de técnicas de filtrado para la reducción de ruido, segmentación y obtención de características relevantes en la detección de actividad. También hacemos énfasis en el estudio de Máquinas de soporte vectorial (MSV) que son uno de los algoritmos de AA más usados en la actualidad. Para ello reformulamos varios de sus métodos estándar (lineales y no lineales) con el propósito de encontrar la mejor combinación de variables que garanticen un buen desempeño del sistema en cuanto a precisión, coste computacional y requerimientos de energía, los cuales son aspectos esenciales en dispositivos portátiles con suministro de energía mediante baterías. En concreto, proponemos dos MSV multiclase para la clasificación de actividad: un algoritmo lineal que permite el balance entre la reducción de la dimensionalidad y la precisión del sistema; y asimismo presentamos un algoritmo no lineal conveniente para dispositivos con limitaciones de hardware que solo utiliza aritmética de punto fijo en la fase de predicción y que permite reducir la complejidad del modelo de aprendizaje mientras mantiene el rendimiento del sistema. La eficacia del sistema propuesto es verificada a través de una experimentación extensiva sobre la base de datos RAH que hemos generado y hecho pública en la red. Esta contiene la información inercial obtenida de un grupo de 30 participantes que realizaron una serie de actividades de la vida cotidiana en un ambiente controlado mientras tenían sujeto a su cintura un smartphone que capturaba su movimiento. Los resultados obtenidos en esta investigación demuestran que es posible realizar el RAH en tiempo real con una precisión cercana al 97%. De esta manera, podemos emplear la metodología propuesta en aplicaciones de alto nivel que requieran el RAH tales como monitorizaciones ambulatorias para personas dependientes (ej. ancianos o discapacitados) durante periodos mayores a cinco días sin la necesidad de recarga de baterías.Postprint (published version

Classification of EMG signals to control a prosthetic hand using time-frequesncy representations and Support Vector Machines

Myoelectric signals (MES) are viable control signals for externally-powered prosthetic devices. They may improve both the functionality and the cosmetic appearance of these devices. Conventional controllers, based on the signal\u27s amplitude features in the control strategy, lack a large number of controllable states because signals from independent muscles are required for each degree of freedom (DoF) of the device. Myoelectric pattern recognition systems can overcome this problem by discriminating different residual muscle movements instead of contraction levels of individual muscles. However, the lack of long-term robustness in these systems and the design of counter-intuitive control/command interfaces have resulted in low clinical acceptance levels. As a result, the development of robust, easy to use myoelectric pattern recognition-based control systems is the main challenge in the field of prosthetic control. This dissertation addresses the need to improve the controller\u27s robustness by designing a pattern recognition-based control system that classifies the user\u27s intention to actuate the prosthesis. This system is part of a cost-effective prosthetic hand prototype developed to achieve an acceptable level of functional dexterity using a simple to use interface. A Support Vector Machine (SVM) classifier implemented as a directed acyclic graph (DAG) was created. It used wavelet features from multiple surface EMG channels strategically placed over five forearm muscles. The classifiers were evaluated across seven subjects. They were able to discriminate five wrist motions with an accuracy of 91.5%. Variations of electrode locations were artificially introduced at each recording session as part of the procedure, to obtain data that accounted for the changes in the user\u27s muscle patterns over time. The generalization ability of the SVM was able to capture most of the variability in the data and to maintain an average classification accuracy of 90%. Two principal component analysis (PCA) frameworks were also evaluated to study the relationship between EMG recording sites and the need for feature space reduction. The dimension of the new feature set was reduced with the goal of improving the classification accuracy and reducing the computation time. The analysis indicated that the projection of the wavelet features into a reduced feature space did not significantly improve the accuracy and the computation time. However, decreasing the number of wavelet decomposition levels did lower the computational load without compromising the average signal classification accuracy. Based on the results of this work, a myoelectric pattern recognition-based control system that uses an SVM classifier applied to time-frequency features may be used to discriminate muscle contraction patterns for prosthetic applications

Energy Efficient Smartphone-Based Activity Recognition Using Fixed-Point Arithmetic

In this paper we propose a novel energy efficient approach for the recog- nition of human activities using smartphones as wearable sensing devices, targeting assisted living applications such as remote patient activity monitoring for the disabled and the elderly. The method exploits fixed-point arithmetic to propose a modified multiclass Support Vector Machine (SVM) learning algorithm, allowing to better pre- serve the smartphone battery lifetime with respect to the conventional floating-point based formulation while maintaining comparable system accuracy levels. Experiments show comparative results between this approach and the traditional SVM in terms of recognition performance and battery consumption, highlighting the advantages of the proposed method

Machine Learning Based Classification of Textual Stimuli to Promote Ideation in Bioinspired Design

Bioinspired design uses biological systems to inspire engineering designs. One of bioinspired design’s challenges is identifying relevant information sources in biology for an engineering design task. Currently information can be retrieved by searching biology texts or journals using biology-focused keywords that map to engineering functions. However, this search technique can overwhelm designers with unusable results. This work explores the use of text classification tools to identify relevant biology passages for design. Further, this research examines the effects of using biology passages as stimuli during idea generation. Four human-subjects studies are examined in this work. Two surveys are performed in which participants evaluate sentences from a biology corpus and indicate whether each sentence prompts an idea for solving a specific design problem. The surveys are used to develop and evaluate text classification tools. Two idea generation studies are performed in which participants generate and record solutions for designing a corn shucker using either different sets of biology passages as design stimuli, or no stimuli. Based 286 sentences from the surveys, a k Nearest Neighbor classifier is developed that is able to identify helpful sentences relating to the function “separate” with a precision of 0.62 and recall of 0.48. This classifier could potentially double the number of helpful results found using a keyword search. The developed classifier is specific to the function “separate” and performs poorly when used for another function. Classifiers developed using all sentences and participant responses from the surveys are not able to reliably identify helpful sentences. From the idea generation studies, we determine that using any biology passages as design stimuli increases the quantity and variety of participant solutions. Solution quantity and variety are also significantly increased when biology passages are presented one at a time instead of all at once. Quality and variety are not significantly affected by the presence of design stimuli. Biological stimuli are also found to lead designers to types of solution that are not typically produced otherwise. This work develops a means for designers to find more useful information when searching biology and demonstrates several ways that biology passages can improve ideation

Multimodal Approach for Malware Detection

Although malware detection is a very active area of research, few works were focused on using physical properties (e.g., power consumption) and multimodal features for malware detection. We designed an experimental testbed that allowed us to run samples of malware and non-malicious software applications and to collect power consumption, network traffic, and system logs data, and subsequently to extract dynamic behavioral-based features. We also extracted code-based static features of both malware and non-malicious software applications. These features were used for malware detection based on: feature level fusion using power consumption and network traffic data, feature level fusion using network traffic data and system logs, and multimodal feature level and decision level fusion. The contributions when using feature level fusion of power consumption and network traffic data are: (1) We focused on detecting real malware using the extracted dynamic behavioral features (both power-based and network traffic-based) and supervised machine learning algorithms, which has not been done by any of the prior works. (2) We ran a large number of machine learning experiments, which allowed us to identify the best performing learner, DC voltage rails that led to the best malware detection performance, and the subset of features that are the best predictors for malware detection. (3) The comparison of malware detection performance was done using a comprehensive set of metrics that reflect different aspects of the quality of malware detection. In the case of the feature level fusion using network traffic data and system logs, the contributions are: (1) Most of the previous works that have used network flows-based features have done classification of the network traffic, while our focus was on classifying the software running in a machine as malware and non-malicious software using the extracted dynamic behavioral features. (2) We experimented with different sizes of the training set (i.e., 90%, 75%, 50%, and 25% of the data) and found that smaller training sets produced very good classification results. This aspect of our work has a practical value because the manual labeling of the training set is a tedious and time consuming process. In this dissertation we present a multimodal deep learning neural network that integrates different modalities (i.e., power consumption, system logs, network traffic, and code-based static data) using decision level fusion. We evaluated the performance of each modality individually, when using feature level fusion, and when using decision level fusion. The contributions of our multimodal approach are as follow: (1) Collecting data from different modalities allowed us to develop a multimodal approach to malware detection, which has not been widely explored by prior works. Even more, none of the previous works compared the performance of feature level fusion with decision level fusion, which is explored in this dissertation. (2) We proposed a multimodal decision level fusion malware detection approach using a deep neural network and compared its performance with the performance of feature level fusion approaches based on deep neural network and standard supervised machine learning algorithms (i.e., Random Forest, J48, JRip, PART, Naive Bayes, and SMO)

Bayesian methods for Support Vector machines and Gaussian processes

We present a common probabilistic framework for kernel or spline smooth- ing methods, including popular architectures such as Gaussian processes and Support Vector machines. We identify the problem of unnormalized loss func- tions and suggest a general technique to overcome this problem at least ap- proximately. We give an intuitive interpretation of the effect an unnormalized loss function can induce, by comparing Support Vector classification (SVC) with Gaussian process classification (GPC) as a nonparametric generalization of logistic regression. This interpretation relates SVC to boosting techniques. We propose a variational Bayesian model selection algorithm for general nor- malized loss functions. This algorithm has a wider applicability than other previously suggested Bayesian techniques and exhibits comparable perfor- mance in cases where both techniques are applicable. We present and discuss results of a substantial number of experiments in which we applied the vari- ational algorithm to common real-world classification tasks and compared it to a range of other known methods. The wider scope of this thesis is to provide a bridge between the fields of probabilistic Bayesian techniques and Statistical Learning Theory, and we present some material of tutorial nature which we hope will be useful to researchers of both fields