Computing Network of Diseases and Pharmacological Entities through the Integration of Distributed Literature Mining and Ontology Mapping

The proliferation of -omics (such as, Genomics, Proteomics) and -ology (such as, System Biology, Cell Biology, Pharmacology) have spawned new frontiers of research in drug discovery and personalized medicine. A vast amount (21 million) of published research results are archived in the PubMed and are continually growing in size. To improve the accessibility and utility of such a large number of literatures, it is critical to develop a suit of semantic sensitive technology that is capable of discovering knowledge and can also infer possible new relationships based on statistical co-occurrences of meaningful terms or concepts. In this context, this thesis presents a unified framework to mine a large number of literatures through the integration of latent semantic analysis (LSA) and ontology mapping. In particular, a parameter optimized, robust, scalable, and distributed LSA (DiLSA) technique was designed and implemented on a carefully selected 7.4 million PubMed records related to pharmacology. The DiLSA model was integrated with MeSH to make the model effective and efficient for a specific domain. An optimized multi-gram dictionary was customized by mapping the MeSH to build the DiLSA model. A fully integrated web-based application, called PharmNet, was developed to bridge the gap between biological knowledge and clinical practices. Preliminary analysis using the PharmNet shows an improved performance over global LSA model. A limited expert evaluation was performed to validate the retrieved results and network with biological literatures. A thorough performance evaluation and validation of results is in progress

Novel Deep Learning Techniques For Computer Vision and Structure Health Monitoring

This thesis proposes novel techniques in building a generic framework for both the regression and classification tasks in vastly different applications domains such as computer vision and civil engineering. Many frameworks have been proposed and combined into a complex deep network design to provide a complete solution to a wide variety of problems. The experiment results demonstrate significant improvements of all the proposed techniques towards accuracy and efficiency

Relevant data representation by a Kernel-based framework

Nowadays, the analysis of a large amount of data has emerged as an issue of great interest taking increasing place in the scientific community, especially in automation, signal processing, pattern recognition, and machine learning. In this sense, the identification, description, classification, visualization, and clustering of events or patterns are important problems for engineering developments and scientific issues, such as biology, medicine, economy, artificial vision, artificial intelligence, and industrial production. Nonetheless, it is difficult to interpret the available information due to its complexity and a large amount of obtained features. In addition, the analysis of the input data requires the development of methodologies that allow to reveal the relevant behaviors of the studied process, particularly, when such signals contain hidden structures varying over a given domain, e.g., space and/or time. When the analyzed signal contains such kind of properties, directly applying signal processing and machine learning procedures without considering a suitable model that deals with both the statistical distribution and the data structure, can lead in unstable performance results. Regarding this, kernel functions appear as an alternative approach to address the aforementioned issues by providing flexible mathematical tools that allow enhancing data representation for supporting signal processing and machine learning systems. Moreover, kernelbased methods are powerful tools for developing better-performing solutions by adapting the kernel to a given problem, instead of learning data relationships from explicit raw vector representations. However, building suitable kernels requires some user prior knowledge about input data, which is not available in most of the practical cases. Furthermore, using the definitions of traditional kernel methods directly, possess a challenging estimation problem that often leads to strong simplifications that restrict the kind of representation that we can use on the data. In this study, we propose a data representation framework based on kernel methods to learn automatically relevant sample relationships in learning systems. Namely, the proposed framework is divided into five kernel-based approaches, which aim to compute relevant data representations by adapting them according to both the imposed sample relationships constraints and the learning scenario (unsupervised or supervised task). First, we develop a kernel-based representation approach that allows revealing the main input sample relations by including relevant data structures using graph-based sparse constraints. Thus, salient data structures are highlighted aiming to favor further unsupervised clustering stages. This approach can be viewed as a graph pruning strategy within a spectral clustering framework which allows enhancing both the local and global data consistencies for a given input similarity matrix. Second, we introduce a kernel-based representation methodology that captures meaningful data relations in terms of their statistical distribution. Thus, an information theoretic learning (ITL) based penalty function is introduced to estimate a kernel-based similarity that maximizes the whole information potential variability. So, we seek for a reproducing kernel Hilbert space (RKHS) that spans the widest information force magnitudes among data points to support further clustering stages. Third, an entropy-like functional on positive definite matrices based on Renyi’s definition is adapted to develop a kernel-based representation approach which considers the statistical distribution and the salient data structures. Thereby, relevant input patterns are highlighted in unsupervised learning tasks. Particularly, the introduced approach is tested as a tool to encode relevant local and global input data relationships in dimensional reduction applications. Fourth, a supervised kernel-based representation is introduced via a metric learning procedure in RKHS that takes advantage of the user-prior knowledge, when available, regarding the studied learning task. Such an approach incorporates the proposed ITL-based kernel functional estimation strategy to adapt automatically the relevant representation using both the supervised information and the input data statistical distribution. As a result, relevant sample dependencies are highlighted by weighting the input features that mostly encode the supervised learning task. Finally, a new generalized kernel-based measure is proposed by taking advantage of different RKHSs. In this way, relevant dependencies are highlighted automatically by considering the input data domain-varying behavior and the user-prior knowledge (supervised information) when available. The proposed measure is an extension of the well-known crosscorrentropy function based on Hilbert space embeddings. Throughout the study, the proposed kernel-based framework is applied to biosignal and image data as an alternative to support aided diagnosis systems and image-based object analysis. Indeed, the introduced kernel-based framework improve, in most of the cases, unsupervised and supervised learning performances, aiding researchers in their quest to process and to favor the understanding of complex dataResumen: Hoy en día, el análisis de datos se ha convertido en un tema de gran interés para la comunidad científica, especialmente en campos como la automatización, el procesamiento de señales, el reconocimiento de patrones y el aprendizaje de máquina. En este sentido, la identificación, descripción, clasificación, visualización, y la agrupación de eventos o patrones son problemas importantes para desarrollos de ingeniería y cuestiones científicas, tales como: la biología, la medicina, la economía, la visión artificial, la inteligencia artificial y la producción industrial. No obstante, es difícil interpretar la información disponible debido a su complejidad y la gran cantidad de características obtenidas. Además, el análisis de los datos de entrada requiere del desarrollo de metodologías que permitan revelar los comportamientos relevantes del proceso estudiado, en particular, cuando tales señales contienen estructuras ocultas que varían sobre un dominio dado, por ejemplo, el espacio y/o el tiempo. Cuando la señal analizada contiene este tipo de propiedades, los rendimientos pueden ser inestables si se aplican directamente técnicas de procesamiento de señales y aprendizaje automático sin tener en cuenta la distribución estadística y la estructura de datos. Al respecto, las funciones núcleo (kernel) aparecen como un enfoque alternativo para abordar las limitantes antes mencionadas, proporcionando herramientas matemáticas flexibles que mejoran la representación de los datos de entrada. Por otra parte, los métodos basados en funciones núcleo son herramientas poderosas para el desarrollo de soluciones de mejor rendimiento mediante la adaptación del núcleo de acuerdo al problema en estudio. Sin embargo, la construcción de funciones núcleo apropiadas requieren del conocimiento previo por parte del usuario sobre los datos de entrada, el cual no está disponible en la mayoría de los casos prácticos. Por otra parte, a menudo la estimación de las funciones núcleo conllevan sesgos el modelo, siendo necesario apelar a simplificaciones matemáticas que no siempre son acordes con la realidad. En este estudio, se propone un marco de representación basado en métodos núcleo para resaltar relaciones relevantes entre los datos de forma automática en sistema de aprendizaje de máquina. A saber, el marco propuesto consta de cinco enfoques núcleo, en aras de adaptar la representación de acuerdo a las relaciones impuestas sobre las muestras y sobre el escenario de aprendizaje (sin/con supervisión). En primer lugar, se desarrolla un enfoque de representación núcleo que permite revelar las principales relaciones entre muestras de entrada mediante la inclusión de estructuras relevantes utilizando restricciones basadas en modelado por grafos. Por lo tanto, las estructuras de datos más sobresalientes se destacan con el objetivo de favorecer etapas posteriores de agrupamiento no supervisado. Este enfoque puede ser visto como una estrategia de depuración de grafos dentro de un marco de agrupamiento espectral que permite mejorar las consistencias locales y globales de los datos En segundo lugar, presentamos una metodología de representación núcleo que captura relaciones significativas entre muestras en términos de su distribución estadística. De este modo, se introduce una función de costo basada en aprendizaje por teoría de la información para estimar una similitud que maximice la variabilidad del potencial de información de los datos de entrada. Así, se busca un espacio de Hilbert generado por el núcleo que contenga altas fuerzas de información entre los puntos para favorecer el agrupamiento entre los mismos. En tercer lugar, se propone un esquema de representación que incluye un funcional de entropía para matrices definidas positivas a partir de la definición de Renyi. En este sentido, se pretenden incluir la distribución estadística de las muestras y sus estructuras relevantes. Por consiguiente, los patrones de entrada pertinentes se destacan en tareas de aprendizaje sin supervisión. En particular, el enfoque introducido se prueba como una herramienta para codificar las relaciones locales y globales de los datos en tareas de reducción de dimensión. En cuarto lugar, se introduce una metodología de representación núcleo supervisada a través de un aprendizaje de métrica en el espacio de Hilbert generado por una función núcleo en aras de aprovechar el conocimiento previo del usuario con respecto a la tarea de aprendizaje. Este enfoque incorpora un funcional por teoría de información que permite adaptar automáticamente la representación utilizando tanto información supervisada y la distribución estadística de los datos de entrada. Como resultado, las dependencias entre las muestras se resaltan mediante la ponderación de las características de entrada que codifican la tarea de aprendizaje supervisado. Por último, se propone una nueva medida núcleo mediante el aprovechamiento de diferentes espacios de representación. De este modo, las dependencias más relevantes entre las muestras se resaltan automáticamente considerando el dominio de interés de los datos de entrada y el conocimiento previo del usuario (información supervisada). La medida propuesta es una extensión de la función de cross-correntropia a partir de inmersiones en espacios de Hilbert. A lo largo del estudio, el esquema propuesto se valida sobre datos relacionados con bioseñales e imágenes como una alternativa para apoyar sistemas de apoyo diagnóstico y análisis objetivo basado en imágenes. De hecho, el marco introducido permite mejorar, en la mayoría de los casos, el rendimiento de sistemas de aprendizaje supervisado y no supervisado, favoreciendo la precisión de la tarea y la interpretabilidad de los datosDoctorad

Development of nonlinear techniques based on time-frequency representation and information theory for the analysis of EEG signals to assess different states of consciousness

Electroencephalogram (EEG) recordings provide insight into the changes in brain activity associated with various states of anesthesia, epilepsy, brain attentiveness, sleep disorders, brain disorders, etc. EEG's are complex signals whose statistical properties depend on both space and time. Their randomness and non-stationary characteristics make them impossible to be described in an accurate way with a simple technique, requiring analysis and characterization involves techniques that take into account their non-stationarity. For that, new advanced techniques in order to improve the efficiency of the EEG based methods used in the clinical practice have to be developed. The main objective of this thesis was to investigate and implement different methods based on nonlinear techniques in order to develop indexes able to characterize the frequency spectrum, the nonlinear dynamics and the complexity of the EEG signals recorded in different state of consciousness. Firstly, a new method for removing peak and spike in biological signal based on the signal envelope was successfully designed and applied to simulated and real EEG signals, obtaining performances significantly better than the traditional adaptive filters. Then, several studies were carried out in order to extract and evaluate EEG measures based on nonlinear techniques in different contexts such as the automatic detection of sleepiness and the characterization and prediction of the nociceptive stimuli and the assessment of the sedation level. Four novel indexes were defined by calculating entropy of the Choi-Williams distribution (CWD) with respect to time or frequency, by using the probability mass function at each time instant taken independently or by using the probability mass function of the entire CWD. The values of these indexes tend to decrease, with different proportion, when the behavior of the signals evolved from chaos or randomness to periodicity and present differences when comparing EEG recorded in eyes-open and eyes-closed states and in ictal and non-ictal states. Measures obtained with time-frequency representation, mutual information function and correntropy, were applied to EEG signals for the automatic sleepiness detection in patients suffering sleep disorders. The group of patients with excessive daytime sleepiness presented more power in ¿ band than the group without sleepiness, which presented higher spectral and cross-spectral entropy in the frontal zone in d band. More complexity in the occipital zone was found in the group of patients without sleepiness in ß band, while a stronger nonlinear coupling between the occipital and frontal regions was detected in patients with excessive daytime sleepiness, in ß band. Time-frequency representation and non-linear measures were also used in order to study how adaptation and fatigue affect the event-related brain potentials to stimuli of different modalities. Differences between the responses to infrequent and frequent stimulation in different recording periods were found in series of averaged EEG epochs recorded after thermal, electrical and auditory stimulation. Nonlinear measures calculated on EEG filtered in the traditional frequency bands and in higher frequency bands improved the assessment of the sedation level. These measures were obtained by applying all the developed techniques on signals recorded from patients sedated, in order to predict the responses to pain stimulation such as nail bad compression and endoscopy tube insertion. The proposed measures exhibit better performances than the bispectral index (BIS), a traditional indexes used for hypnosis assessment. In conclusion, nonlinear measures based on time-frequency representation, mutual information functions and correntropy provided additional information that helped to improve the automatic sleepiness detection, the characterization and prediction of the nociceptive responses and thus the assessment of the sedation level.El registro de la señal Electroencefalografíca (EEG) proporciona información sobre los cambios en la actividad cerebral asociados con varios estados de la anestesia, la epilepsia, la atención cerebral, los trastornos del sueño, los trastornos cerebrales, etc. Los EEG son señales complejas cuyas propiedades estadísticas dependen del espacio y del tiempo. Sus características aleatorias y no estacionarias hacen imposible que el EEG se describa de forma precisa con una técnica sencilla requiriendo un análisis y una caracterización que implica técnicas que tengan en cuenta su no estacionariedad. Todo esto aumenta la necesidad de desarrollar nuevas técnicas avanzadas con el fin de mejorar la eficiencia de los métodos utilizados en la práctica clínica que son basados en el análisis de EEG. En esta tesis se han investigado y aplicado diferentes métodos utilizando técnicas no lineales con el fin de desarrollar índices capaces de caracterizar el espectro de frecuencias, la dinámica no lineal y la complejidad de las señales EEG registradas en diferentes estados de conciencia. En primer lugar, se ha desarrollado un nuevo algoritmo basado en la envolvente de la señal para la eliminación de ruido de picos en las señales biológicas. Este algoritmo ha sido aplicado a señales simuladas y reales obteniendo resultados significativamente mejores comparados con los filtros adaptativos tradicionales. Seguidamente, se han llevado a cabo varios estudios con el fin de extraer y evaluar las medidas de EEG basadas en técnicas no lineales en diferentes contextos. Se han definido nuevos índices mediante el cálculo de la entropía de la distribución de Choi-Williams (DCW) con respecto al tiempo o la frecuencia. Se ha observado que los valores de estos índices tienden a disminuir, en diferentes proporciones, cuando el comportamiento de las señales evoluciona de caótico o aleatorio a periódico. Además, se han encontrado valores diferentes de estos índices aplicados a la señal EEG registrada en diferentes estados. Diferentes medidas basadas en la representación tiempo-frecuencia, la función de información mutua y la correntropia se han aplicado al EEG para la detección automática de la somnolencia en pacientes que sufren trastornos del sueño. Se ha observado en la zona frontal que la potencia en la banda θ es mayor en los pacientes con somnolencia diurna excesiva, mientras que la entropía espectral y la entropía espectral cruzada en la banda δ es mayor en los pacientes sin somnolencia. En el grupo sin somnolencia se ha encontrado más complejidad en la zona occipital, mientras que el acoplamiento no lineal entre las regiones occipital y frontal ha resultado más fuerte en pacientes con somnolencia diurna excesiva, en la banda β. La representación tiempo-frecuencia y las medidas no lineales se han utilizado para estudiar cómo la adaptación y la fatiga afectan a los potenciales cerebrales relacionados con estímulos térmicos, eléctricos y auditivos. Analizando el promedio de varias épocas de EEG grabadas después de la estimulación, se han encontrado diferencias entre las respuestas a la estimulación frecuente e infrecuente en diferentes períodos de registro. Todas las técnicas que se han desarrollado, se han aplicado a señales EEG registradas en pacientes sedados, con el fin de predecir las respuestas a la estimulación del dolor. Un conjunto de medidas calculadas en señales EEG filtradas en diferentes bandas de frecuencia ha permitido mejorar la evaluación del nivel de sedación. Las medidas propuestas han presentado un mejor rendimiento comparado con el índice bispectral, un indicador de hipnosis tradicional. En conclusión, las medidas no lineales basadas en la representación tiempofrecuencia, funciones de información mutua y correntropia han proporcionado informaciones adicionales que contribuyeron a mejorar la detección automática de la somnolencia, la caracterización y predicción de las respuestas nociceptivas y por lo tanto la evaluación del nivel de sedación

Novel Computational Methods for State Space Filtering

The state-space formulation for time-dependent models has been long used invarious applications in science and engineering. While the classical Kalman filter(KF) provides optimal posterior estimation under linear Gaussian models, filteringin nonlinear and non-Gaussian environments remains challenging.Based on the Monte Carlo approximation, the classical particle filter (PF) can providemore precise estimation under nonlinear non-Gaussian models. However, it suffers fromparticle degeneracy. Drawing from optimal transport theory, the stochastic map filter(SMF) accommodates a solution to this problem, but its performance is influenced bythe limited flexibility of nonlinear map parameterisation. To account for these issues,a hybrid particle-stochastic map filter (PSMF) is first proposed in this thesis, wherethe two parts of the split likelihood are assimilated by the PF and SMF, respectively.Systematic resampling and smoothing are employed to alleviate the particle degeneracycaused by the PF. Furthermore, two PSMF variants based on the linear and nonlinearmaps (PSMF-L and PSMF-NL) are proposed, and their filtering performance is comparedwith various benchmark filters under different nonlinear non-Gaussian models.Although achieving accurate filtering results, the particle-based filters require expensive computations because of the large number of samples involved. Instead, robustKalman filters (RKFs) provide efficient solutions for the linear models with heavy-tailednoise, by adopting the recursive estimation framework of the KF. To exploit the stochasticcharacteristics of the noise, the use of heavy-tailed distributions which can fit variouspractical noises constitutes a viable solution. Hence, this thesis also introduces a novelRKF framework, RKF-SGαS, where the signal noise is assumed to be Gaussian and theheavy-tailed measurement noise is modelled by the sub-Gaussian α-stable (SGαS) distribution. The corresponding joint posterior distribution of the state vector and auxiliaryrandom variables is estimated by the variational Bayesian (VB) approach. Four differentminimum mean square error (MMSE) estimators of the scale function are presented.Besides, the RKF-SGαS is compared with the state-of-the-art RKFs under three kinds ofheavy-tailed measurement noises, and the simulation results demonstrate its estimationaccuracy and efficiency.One notable limitation of the proposed RKF-SGαS is its reliance on precise modelparameters, and substantial model errors can potentially impede its filtering performance. Therefore, this thesis also introduces a data-driven RKF method, referred to asRKFnet, which combines the conventional RKF framework with a deep learning technique. An unsupervised scheduled sampling technique (USS) is proposed to improve theistability of the training process. Furthermore, the advantages of the proposed RKFnetare quantified with respect to various traditional RKFs

Deep Neural Network Architectures for Modulation Classification

This thesis investigates the value of employing deep learning for the task of wireless signal modulation recognition. Recently in deep learning research on AMC, a framework has been introduced by generating a dataset using GNU radio that mimics the imperfections in a real wireless channel, and uses 10 different modulation types. Further, a CNN architecture was developed and shown to deliver performance that exceeds that of expert-based approaches. Here, we follow the framework of O’shea [1] and find deep neural network architectures that deliver higher accuracy than the state of the art. We tested the architecture of O’shea [1] and found it to achieve an accuracy of approximately 75% of correctly recognizing the modulation type. We first tune the CNN architecture and find a design with four convolutional layers and two dense layers that gives an accuracy of approximately 83.8% at high SNR. We then develop architectures based on the recently introduced ideas of Residual Networks (ResNet) and Densely Connected Network (DenseNet) to achieve high SNR accuracies of approximately 83% and 86.6%, respectively. We also introduce a CLDNN to achieve an accuracy of approximately 88.5% at high SNR. To improve the classification accuracy of QAM, we calculate the high order cumulants of QAM16 and QAM64 as the expert feature and improve the total accuracy to approximately 90%. Finally, by preprocessing the input and send them into a LSTM model, we improve all classification success rates to 100% except the WBFM which is 46%. The average modulation classification accuracy got a improvement of roughly 22% in this thesis