Hypernode Graphs for Spectral Learning on Binary Relations over Sets

Paper accepted for publication at ECML/PKDD 2014International audienceWe introduce hypernode graphs as weighted binary relations between sets of nodes: a hypernode is a set of nodes, a hyperedge is a pair of hypernodes, and each node in a hypernode of a hyperedge is given a non negative weight that represents the node contribution to the relation. Hypernode graphs model binary relations between sets of individuals while allowing to reason at the level of individuals. We present a spectral theory for hypernode graphs that allows us to introduce an unnormalized Laplacian and a smoothness semi-norm. In this framework, we are able to extend spectral graph learning algorithms to the case of hypernode graphs. We show that hypernode graphs are a proper extension of graphs from the expressive power point of view and from the spectral analysis point of view. Therefore hypernode graphs allow to model higher order relations whereas it is not true for hypergraphs as shown in~\cite{Agarwal2006}. In order to prove the potential of the model, we represent multiple players games with hypernode graphs and introduce a novel method to infer skill ratings from game outcomes. We show that spectral learning algorithms over hypernode graphs obtain competitive results with skill ratings specialized algorithms such as Elo duelling and TrueSkill

Hypernode Graphs for Learning from Binary Relations between Groups in Networks

International audienceThe aim of this paper is to propose methods for learning from interactions between groups in networks. We introduced hypernode graphs in Ricatte et al (2014) a formal model able to represent group interactions and able to infer individual properties as well. Spectral graph learning algorithms were extended to the case of hypern-ode graphs. As a proof-of-concept, we have shown how to model multiple players games with hypernode graphs and that spectral learning algorithms over hyper-node graphs obtain competitive results with skill ratings specialized algorithms. In this paper, we explore theoretical issues for hypernode graphs. We show that hypernode graph kernels strictly generalize over graph kernels and hypergraph kernels. We show that hypernode graphs correspond to signed graphs such that the matrix D − W is positive semidefinite. It should be noted that homophilic relations between groups may lead to non homophilic relations between individ-uals. Moreover, we also present some issues concerning random walks and the resistance distance for hypernode graphs

Skill Rating for Multiplayer Games Introducing Hypernode Graphs and their Spectral Theory

International audienceWe consider the skill rating problem for multiplayer games, that is how to infer player skills from game outcomes in multiplayer games. We formulate the problem as a minimization problem arg min s s T ∆s where ∆ is a positive semidefinite matrix and s a real-valued function, of which some entries are the skill values to be inferred and other entries are constrained by the game outcomes. We leverage graph-based semi-supervised learning (SSL) algorithms for this problem. We apply our algorithms on several data sets of multiplayer games and obtain very promising results compared to Elo Duelling (see Elo, 1978) and TrueSkill (see Herbrich et al., 2006). As we leverage graph-based SSL algorithms and because games can be seen as relations between sets of players, we then generalize the approach. For this aim, we introduce a new finite model, called hypernode graph, defined to be a set of weighted binary relations between sets of nodes. We define Laplacians of hy-pernode graphs. Then, we show that the skill rating problem for multiplayer games can be formulated as arg min s s T ∆s where ∆ is the Laplacian of a hypernode graph constructed from a set of games. From a fundamental perspective, we show that hypernode graph Laplacians are symmetric positive semidefinite matrices with constant functions in their null space. We show that problems on hypernode graphs can not be solved with graph constructions and graph kernels. We relate hypernode graphs to signed graphs showing that positive relations between groups can lead to negative relations between individuals

Hypernode Graphs for Learning from Binary Relations between Groups in Networks

The aim of this paper is to propose methods for learning from interactions between groups in networks. We propose a proper extension of graphs, called hypernode graphs as a formal tool able to model group interactions. A hypernode graph is a collection of weighted relations between two disjoint groups of nodes. Weights quantify the individual participation of nodes to a given relation. We define Laplacians and kernels for hypernode graphs and prove that they strictly generalize over graph kernels and hypergraph kernels. We then proceed to prove that hypernode graphs correspond to signed graphs such that the matrix D − W is positive semi-definite. As a consequence, homophilic relations between groups may lead to non homophilic relations between individuals. We also define the notion of connected hypernode graphs and a resistance distance for connected hypernode graphs. Then, we propose spectral learning algorithms on hypernode graphs allowing to infer node ratings or node labelings. As a proof of concept, we model multiple players games with hypernode graphs and we define skill rating algorithms competitive with specialized algorithms

Community detection for correlation matrices

A challenging problem in the study of complex systems is that of resolving, without prior information, the emergent, mesoscopic organization determined by groups of units whose dynamical activity is more strongly correlated internally than with the rest of the system. The existing techniques to filter correlations are not explicitly oriented towards identifying such modules and can suffer from an unavoidable information loss. A promising alternative is that of employing community detection techniques developed in network theory. Unfortunately, this approach has focused predominantly on replacing network data with correlation matrices, a procedure that tends to be intrinsically biased due to its inconsistency with the null hypotheses underlying the existing algorithms. Here we introduce, via a consistent redefinition of null models based on random matrix theory, the appropriate correlation-based counterparts of the most popular community detection techniques. Our methods can filter out both unit-specific noise and system-wide dependencies, and the resulting communities are internally correlated and mutually anti-correlated. We also implement multiresolution and multifrequency approaches revealing hierarchically nested sub-communities with `hard' cores and `soft' peripheries. We apply our techniques to several financial time series and identify mesoscopic groups of stocks which are irreducible to a standard, sectorial taxonomy, detect `soft stocks' that alternate between communities, and discuss implications for portfolio optimization and risk management.Comment: Final version, accepted for publication on PR

Biclustering fMRI time series

Tese de mestrado, Ciência de Dados, Universidade de Lisboa, Faculdade de Ciências, 2020Biclustering é um método de análise que procura gerar clusters tendo em conta simultaneamente as linhas e as colunas de uma matriz de dados. Este método tem sido vastamente explorado em análise de dados genéticos. Apesar de diversos estudos reconhecerem as capacidades deste método de análise em outras áreas de investigação, as últimas duas décadas tem sido marcadas por um número elevado de estudos aplicados em dados genéticos e pela ausência de uma linha de investigação que explore as capacidades de biclustering fora desta área tradicional Esta tese segue pistas que sugerem potencial no uso de biclustering em dados de natureza espaço-temporal. Considerando o contexto particular das neurociências, esta tese explora as capacidades dos algoritmos de biclustering em extrair conhecimento das séries temporais geradas por técnicas de imagem por ressonância magnética funcional (fMRI). Eta tese propõe uma metodologia para avaliar a capacidade de algoritmos de biclustering em estudar dados fMRI, considerando tanto dados sintéticos como dados reais. Para avaliar estes algoritmos, usamos métricas de avaliação interna. Os nossos resultados discutem o uso de diversas estratégias de busca, revelando a superioridade de estratégias exaustivos para obter os biclusters mais homogéneos. No entanto, o elevado custo computacional de estratégias exaustivas ainda são um desafio e é necessário pesquisa adicional para a busca eficiente de biclusters no contexto de análise de dados fMRI. Propomos adicionalmente uma nova metodologia de análise de biclusters baseada em algoritmos de descoberta de padrões para determinar os padrões mais frequentes presentes nas soluções de biclustering geradas. Um bicluster não é mais que um hipervértice num hipergrafo . Extrair padrões frequentes numa solução de biclustering implica extrair os hipervértices mais significativos. Numa primeira abordagem, isto permite entender relações entre regiões do cérebro e traçar perfis temporais que métodos tradicionais de estudos de correlação não são capazes de detetar. Adicionalmente, o processo de gerar os biclusters permite filtrar ligações pouco interessantes, permitindo potencialmente gerar hipergrafos de forma eficiente. A questão final é o que podemos fazer com este conhecimento. Conhecer a relação entre regiões do cérebro é o objetivo central das neurociências. Entender as ligações entre regiões do cérebro para vários sujeitos permitem traçar perfis. Nesse caso, propomos uma metodologia para extrapolar biclusters para dados tridimensionais e efetuar triclustering. Adicionalmente, entender a ligação entre zonas cerebrais permite identificar doenças como a esquizofrenia, demência ou o Alzheimer. Este trabalho aponta caminhos para o uso de biclustering na análise de dados espaço-temporais, em particular em neurociências. A metodologia de avaliação proposta mostra evidências da eficácia do biclustering para encontrar padrões locais em dados de fMRI, embora mais trabalhos sejam necessários em relação à escalabilidade para promover a aplicação em cenários reais.The effectiveness of biclustering, simultaneous clustering of both rows and columns in a data matrix, has been primarily shown in gene expression data analysis. Furthermore, several researchers recognize its potentialities in other research areas. Nevertheless, the last two decades witnessed many biclustering algorithms targeting gene expression data analysis and a lack of consistent studies exploring the capacities of biclustering outside this traditional application domain. Following hints that suggest potentialities for biclustering on Spatiotemporal data, particularly in neurosciences, this thesis explores biclustering’s capacity to extract knowledge from fMRI time series. This thesis proposes a methodology to evaluate biclustering algorithms’ feasibility to study the fMRI signal, considering both synthetic and realworld fMRI datasets. In the absence of ground truth to compare bicluster solutions with a reference one, we used internal valuation metrics. Results discussing the use of different search strategies showed the superiority of exhaustive approaches, obtaining the most homogeneous biclusters. However, their high computational cost is still a challenge, and further work is needed for the efficient use of biclustering in fMRI data analysis. We propose a new methodology for analyzing biclusters based on performing pattern mining algorithms to determine the most frequent patterns present in the generated biclustering solutions. A bicluster is nothing more than a hyperlink in a hypergraph. Extracting frequent patterns in a biclustering solution implies extracting the most significant hyperlinks. In a first approach, this allows to understand relationships between regions of the brain and draw temporal profiles that traditional methods of correlation studies cannot detect. Additionally, the process of generating biclusters allows filtering uninteresting links, potentially allowing to generate hypergraphs efficiently. The final question is, what can we do with this knowledge. Knowing the relationship between brain regions is the central objective of neurosciences. Understanding the connections between regions of the brain for various subjects allows one to draw profiles. In this case, we propose a methodology to extrapolate biclusters to threedimensional data and perform triclustering. Additionally, understanding the link between brain zones allows identifying diseases like schizophrenia, dementia, or Alzheimer’s. This work pinpoints avenues for the use of biclustering in Spatiotemporal data analysis, in particular neurosciences applications. The proposed evaluation methodology showed evidence of biclustering’s effectiveness in finding local fMRI data patterns, although further work is needed regarding scalability to promote the application in real scenarios

Identificación de múltiples intenciones y sus dependencias subsumidas en múltiples utterances para el desarrollo de Chatbots

Los chatbots son sistemas de procesamiento de lenguaje natural con los que se puede interactuar mediante una interfaz de texto o voz, y han sido adoptados en muchas industrias para responder las preguntas y solicitudes de los usuarios a través de interfaces de chat. Por ende, los chatbots tienen un valor comercial como asistentes virtuales. Tanto es así que se está trabajando en que los chatbots puedan comunicarse con los usuarios de manera similar a la comunicación que hay entre dos humanos; en otras palabras, un usuario debe experimentar la sensación de comunicarse con una persona. A su vez, dado que los chatbots eliminan los factores humanos y están disponibles las 24 horas del día, hay un incremento en la demanda de las capacidades de inteligencia artificial para interactuar con los clientes. En este aspecto, la sensación de comunicarse con una persona puede ser lograda mediante la inclusión de técnicas de comprensión del lenguaje natural, procesamiento del lenguaje natural, generación del lenguaje natural y aprendizaje automático. De este modo, los chatbots son capaces de interpretar una o varias intenciones comunicativas en cada “utterance” de un usuario, siendo que un “utterance” es todo lo que el usuario o chatbot mencionan mientras es su turno de hablar o escribir. Así mismo, los chatbots pueden asociar una o varias intenciones comunicativas a un identificador de “utterances” que contiene varios “utterances”. Por ende, a partir del “utterance” de un usuario, un chatbot es capaz de interpretar una o varias intenciones comunicativas asociadas a un identificador de “utterances”, a través del cual usa los “utterances” contenidos para escoger o generar un “utterance” como respuesta al usuario. No obstante, si bien un chatbot puede identificar múltiples intenciones comunicativas en un enunciado, de un usuario, con un “utterance”, no puede identificar múltiples intenciones comunicativas en un enunciado, de un usuario, que contenga múltiples “utterances”. En consecuencia, tampoco se ha investigado como encontrar los “utterances” de respuesta del chatbot cuando se tiene múltiples “utterances”. Por lo descrito previamente, en este proyecto se propone la implementación de una herramienta para: identificar múltiples intenciones comunicativas en múltiples “utterances”, identificar las dependencias entre intenciones, agrupar las intenciones a partir de sus dependencias, identificar las dependencias entre los grupos de intenciones respecto de los identificadores de “utterances” y los identificadores de “utterances” respecto de los “utterances”. Además, para facilitar el uso de la herramienta, se elabora una interfaz de programación de aplicaciones que recibe múltiples “utterances” en forma de texto, y devuelve los “utterances” segmentados, las intenciones identificadas, los grupos entre intenciones y los “utterances” de respuesta del chatbot para cada grupo de intenciones. Los resultados obtenidos evidencian que los enfoques utilizados son exitosos. Por último, se espera mejorar los resultados con técnicas de inteligencia artificial y computación lingüística

Learning with Graphs using Kernels from Propagated Information

Traditional machine learning approaches are designed to learn from independent vector-valued data points. The assumption that instances are independent, however, is not always true. On the contrary, there are numerous domains where data points are cross-linked, for example social networks, where persons are linked by friendship relations. These relations among data points make traditional machine learning diffcult and often insuffcient. Furthermore, data points themselves can have complex structure, for example molecules or proteins constructed from various bindings of different atoms. Networked and structured data are naturally represented by graphs, and for learning we aimto exploit their structure to improve upon non-graph-based methods. However, graphs encountered in real-world applications often come with rich additional information. This naturally implies many challenges for representation and learning: node information is likely to be incomplete leading to partially labeled graphs, information can be aggregated from multiple sources and can therefore be uncertain, or additional information on nodes and edges can be derived from complex sensor measurements, thus being naturally continuous. Although learning with graphs is an active research area, learning with structured data, substantially modeling structural similarities of graphs, mostly assumes fully labeled graphs of reasonable sizes with discrete and certain node and edge information, and learning with networked data, naturally dealing with missing information and huge graphs, mostly assumes homophily and forgets about structural similarity. To close these gaps, we present a novel paradigm for learning with graphs, that exploits the intermediate results of iterative information propagation schemes on graphs. Originally developed for within-network relational and semi-supervised learning, these propagation schemes have two desirable properties: they capture structural information and they can naturally adapt to the aforementioned issues of real-world graph data. Additionally, information propagation can be efficiently realized by random walks leading to fast, flexible, and scalable feature and kernel computations. Further, by considering intermediate random walk distributions, we can model structural similarity for learning with structured and networked data. We develop several approaches based on this paradigm. In particular, we introduce propagation kernels for learning on the graph level and coinciding walk kernels and Markov logic sets for learning on the node level. Finally, we present two application domains where kernels from propagated information successfully tackle real-world problems

Application of Deep Learning Methods in Monitoring and Optimization of Electric Power Systems

This PhD thesis thoroughly examines the utilization of deep learning techniques as a means to advance the algorithms employed in the monitoring and optimization of electric power systems. The first major contribution of this thesis involves the application of graph neural networks to enhance power system state estimation. The second key aspect of this thesis focuses on utilizing reinforcement learning for dynamic distribution network reconfiguration. The effectiveness of the proposed methods is affirmed through extensive experimentation and simulations.Comment: PhD thesi