Learning the Structure for Structured Sparsity

Structured sparsity has recently emerged in statistics, machine learning and signal processing as a promising paradigm for learning in high-dimensional settings. All existing methods for learning under the assumption of structured sparsity rely on prior knowledge on how to weight (or how to penalize) individual subsets of variables during the subset selection process, which is not available in general. Inferring group weights from data is a key open research problem in structured sparsity.In this paper, we propose a Bayesian approach to the problem of group weight learning. We model the group weights as hyperparameters of heavy-tailed priors on groups of variables and derive an approximate inference scheme to infer these hyperparameters. We empirically show that we are able to recover the model hyperparameters when the data are generated from the model, and we demonstrate the utility of learning weights in synthetic and real denoising problems

Adaptive-Step Graph Meta-Learner for Few-Shot Graph Classification

Graph classification aims to extract accurate information from graph-structured data for classification and is becoming more and more important in graph learning community. Although Graph Neural Networks (GNNs) have been successfully applied to graph classification tasks, most of them overlook the scarcity of labeled graph data in many applications. For example, in bioinformatics, obtaining protein graph labels usually needs laborious experiments. Recently, few-shot learning has been explored to alleviate this problem with only given a few labeled graph samples of test classes. The shared sub-structures between training classes and test classes are essential in few-shot graph classification. Exiting methods assume that the test classes belong to the same set of super-classes clustered from training classes. However, according to our observations, the label spaces of training classes and test classes usually do not overlap in real-world scenario. As a result, the existing methods don't well capture the local structures of unseen test classes. To overcome the limitation, in this paper, we propose a direct method to capture the sub-structures with well initialized meta-learner within a few adaptation steps. More specifically, (1) we propose a novel framework consisting of a graph meta-learner, which uses GNNs based modules for fast adaptation on graph data, and a step controller for the robustness and generalization of meta-learner; (2) we provide quantitative analysis for the framework and give a graph-dependent upper bound of the generalization error based on our framework; (3) the extensive experiments on real-world datasets demonstrate that our framework gets state-of-the-art results on several few-shot graph classification tasks compared to baselines

A Kernel of Truth: Determining Rumor Veracity on Twitter by Diffusion Pattern Alone

Recent work in the domain of misinformation detection has leveraged rich signals in the text and user identities associated with content on social media. But text can be strategically manipulated and accounts reopened under different aliases, suggesting that these approaches are inherently brittle. In this work, we investigate an alternative modality that is naturally robust: the pattern in which information propagates. Can the veracity of an unverified rumor spreading online be discerned solely on the basis of its pattern of diffusion through the social network? Using graph kernels to extract complex topological information from Twitter cascade structures, we train accurate predictive models that are blind to language, user identities, and time, demonstrating for the first time that such "sanitized" diffusion patterns are highly informative of veracity. Our results indicate that, with proper aggregation, the collective sharing pattern of the crowd may reveal powerful signals of rumor truth or falsehood, even in the early stages of propagation.Comment: Published at The Web Conference (WWW) 202

Skalierbare Graphkerne

Graph-structured data are becoming more and more abundant in many fields of science and engineering, such as social network analysis, molecular biology, chemistry, computer vision, or program verification. To exploit these data, one needs data analysis and machine learning methods that are able to efficiently handle large-scale graph data sets. Successfully applying machine learning and data analysis methods to graphs requires the ability to efficiently compare and represent graphs. Standard solutions to these problems are either NP-hard, not expressive enough, or difficult to adapt to a problem at hand. Graph kernels have attracted considerable interest in the machine learning community in the last decade as a promising solution to the above-mentioned issues. Despite significant progress in the design and improvement of graph kernels in the past few years, existing graph kernels do not measure up to the current needs of machine learning on large, labeled graphs: Even the most efficient existing kernels need O(n^3) runtime to compare a pair of graphs with n nodes, or cannot take into account node and edge labels. Our primary goal in this thesis is the design of efficient and expressive kernels for machine learning on graphs. We first focus on the design of generic graph kernels that can be applied to graphs with or without labels. Our main contributions to this end are the following: First, we speed up the exact computation of graphlet kernels from O(n^k) to O(nd^{k-1}) for a pair of graphs, where n is the size of the graphs, k is the size of considered graphlets, and d is the maximum degree in the given graphs. Second, we define a new kernel on graphs, the Weisfeiler-Lehman subtree kernel, which is the first graph kernel scaling linearly in the number of edges in the given graph set. In our experiments on benchmark graph data sets from chemoinformatics and bioinformatics, the Weisfeiler-Lehman subtree kernel gracefully scales up to large graphs, outperforms all existing graph kernels in speed, and yields highly competitive performance in graph classification. Third, we generalize the Weisfeiler-Lehman subtree kernel to a family of kernels that includes many known graph kernels as special cases. This generalization enables existing graph kernels to take into account more information about the graph topology, and thereby become more expressive. In the last part of this thesis, we present two examples of applications: Based on our previous contributions, we propose specialized node kernels for pixel classification in remote sensing images, and graph kernels for chemical shift prediction in structural bioinformatics. Our kernels make it possible for the first time to take advantage of the rich graph structure in these applications. The Weisfeiler-Lehman kernels we propose here now allow graph kernels to scale to large, labeled graphs. They open the door to manifold applications of graph kernels in numerous domains which deal with graphs whose size and attributes could not be handled by graph kernels before.Daten in Form von Graphen spielen in immer mehr Bereichen der Wissenschaft und der Wirtschaft eine Rolle, wie in der Analyse sozialer Netzwerke, in der Molekularbiologie, in der Chemie, in der Computervision und in der Programmverifikation. Um diese Daten zu nutzen, braucht man Methoden zur Datenanalyse und zum Maschinenlernen, die in der Lage sind, große Datenmengen effizient zu verarbeiten. Um die Algorithmen des Maschinenlernens erfolgreich auf Graphen anzuwenden, benötigt man Verfahren, um Graphen effizient vergleichen oder repräsentieren zu können. Standardlösungen für diese Probleme sind entweder NP-schwer, nicht expressiv genug, oder schwierig auf das jeweilige Problem anzuwenden. Graphkerne haben im letzten Jahrzehnt im Bereich Maschinenlernen viel Aufmerksamkeit auf sich gezogen, da sie einen vielversprechenden Lösungsansatz für die obengenannten Probleme darstellen. Trotz signifikanter Fortschritte im Bereich der Graphkerne in den letzten Jahren reichen bekannte Graphkerne nicht für die gegenwärtigen Anforderungen im Maschinenlernen aus, wenn die zu untersuchenden Graphen sehr groß oder gelabelt sind. Selbst die effizientesten Kerne erfordern eine Laufzeit von O(n^3), um ein Paar Graphen mit n Knoten zu vergleichen oder um Knoten- und Kantenlabels zu berücksichtigen. Unser wichtigstes Ziel in dieser Dissertation ist daher die Entwicklung effizienter und expressiver Kerne für das Maschinenlernen auf Graphen. Zuerst konzentrieren wir uns auf die Entwicklung allgemeiner Graphkerne, die auf Graphen mit oder ohne Labels angewendet werden können. Unsere Hauptbeiträge sind dabei die Folgenden: Erstens beschleunigen wir die exakte Berechnung von Graphlet-Kernen von O(n^k) bis O(nd^{k-1}) für ein Paar Graphen, wobei n die Größe des Graphen ist, k die Größe der angewandten Graphlets, und d der maximale Grad der gegebenen Graphen. Zweitens definieren wir einen neuen Kern für Graphen, den Weisfeiler-Lehman-Unterbaumkern, der der erste Graphkern ist, der linear in der Anzahl der Kanten in dem gegebenen Graphensatz skaliert. In unseren Experimenten auf Vergleichsdatensätzen aus der Chemoinformatik und der Bioinformatik skaliert der Weisfeiler-Lehman-Unterbaumkern bis zu großen Graphen und übertrifft alle bekannten Graphkerne an Geschwindigkeit mit vergleichbarer oder bessere Vorhersagegenauigkeit bei der Graphenklassifizierung. Drittens verallgemeinen wir den Weisfeiler-Lehman-Unterbaumkern zu einer Kernfamilie, die viele bekannte Graphkerne als Spezialfälle beinhaltet. Diese Verallgemeinerung ermöglicht es bekannten Graphkerne, mehr Informationen über die Topologie der Graphen zu berücksichtigen. Im letzten Teil dieser Dissertation präsentieren wir zwei Anwendungsbeispiele: Basierend auf den vorherigen Beiträgen schlagen wir spezialisierte Kerne für Pixelklassifizierung in Fernerkundungsbildern und Graphkerne für die Vorhersage von chemischen Verschiebungen in der strukturellen Bioinformatik vor. Unsere Kerne ermöglichen es erstmals, in diesen Anwendungen die reichhaltige Graphenstruktur beim Lernen zu nutzen. Die Weisfeiler-Lehman-Kerne, die wir hier vorschlagen, ermöglichen es Graphkerne, auf große und gelabelte Graphen zu skalieren. Sie erlauben die Nutzung von Graphkernen in zahlreichen Anwendungsgebieten, die sich mit Graphen beschäftigen, dessen Größe und Labels mit bekannten Graphkernen vorher nicht verarbeitet werden konnten

The Graphlet Spectrum

Current graph kernels suffer from two limitations: graph kernels based on counting particular types of subgraphs ignore the relative position of these subgraphs to each other, while graph kernels based on algebraic methods are limited to graphs without node labels. In this paper we present the graphlet spectrum, a system of graph invariants derived by means of group representation theory that capture information about the number as well as the position of labeled subgraphs in a given graph. In our experimental evaluation the graphlet spectrum outperforms state-of-the-art graph kernels