814 research outputs found
Learning node labels with multi-category Hopfield networks
In several real-world node label prediction problems on graphs, in fields ranging from computational
biology to World Wide Web analysis, nodes can be partitioned into categories different from the classes to be predicted, on the basis of their characteristics or their common properties. Such partitions may provide further information about node classification that classical machine learning algorithms do not take into account. We introduce a novel family of parametric Hopfield networks (m-category Hopfield networks) and a novel algorithm (Hopfield multi-category \u2014 HoMCat ), designed to appropriately exploit the presence of property-based partitions of nodes into multiple categories. Moreover, the proposed model adopts a cost-sensitive learning strategy to prevent the remarkable decay in performance usually observed when instance labels are unbalanced, that is, when one class of labels is highly underrepresented than the other one. We validate the proposed model on both synthetic and real-world data, in the context of multi-species function
prediction, where the classes to be predicted are the Gene Ontology terms and the categories the different species in the multi-species protein network. We carried out an intensive experimental validation, which on the one hand compares HoMCat with several state-of-the-art graph-based algorithms, and on the other hand reveals that exploiting meaningful prior partitions of input data can substantially improve classification performances
Classification in Networked Data: A Toolkit and a Univariate Case Study
This paper1 is about classifying entities that are interlinked with entities for which the class is
known. After surveying prior work, we present NetKit, a modular toolkit for classification in networked
data, and a case-study of its application to networked data used in prior machine learning
research. NetKit is based on a node-centric framework in which classifiers comprise a local classifier,
a relational classifier, and a collective inference procedure. Various existing node-centric
relational learning algorithms can be instantiated with appropriate choices for these components,
and new combinations of components realize new algorithms. The case study focuses on univariate
network classification, for which the only information used is the structure of class linkage in
the network (i.e., only links and some class labels). To our knowledge, no work previously has
evaluated systematically the power of class-linkage alone for classification in machine learning
benchmark data sets. The results demonstrate that very simple network-classification models perform
quite well—well enough that they should be used regularly as baseline classifiers for studies
of learning with networked data. The simplest method (which performs remarkably well) highlights
the close correspondence between several existing methods introduced for different purposes—that
is, Gaussian-field classifiers, Hopfield networks, and relational-neighbor classifiers. The case study
also shows that there are two sets of techniques that are preferable in different situations, namely
when few versus many labels are known initially. We also demonstrate that link selection plays an
important role similar to traditional feature selectionNYU, Stern School of Business, IOMS Department, Center for Digital Economy Researc
Energy Transformer
Transformers have become the de facto models of choice in machine learning,
typically leading to impressive performance on many applications. At the same
time, the architectural development in the transformer world is mostly driven
by empirical findings, and the theoretical understanding of their architectural
building blocks is rather limited. In contrast, Dense Associative Memory models
or Modern Hopfield Networks have a well-established theoretical foundation, but
have not yet demonstrated truly impressive practical results. We propose a
transformer architecture that replaces the sequence of feedforward transformer
blocks with a single large Associative Memory model. Our novel architecture,
called Energy Transformer (or ET for short), has many of the familiar
architectural primitives that are often used in the current generation of
transformers. However, it is not identical to the existing architectures. The
sequence of transformer layers in ET is purposely designed to minimize a
specifically engineered energy function, which is responsible for representing
the relationships between the tokens. As a consequence of this computational
principle, the attention in ET is different from the conventional attention
mechanism. In this work, we introduce the theoretical foundations of ET,
explore it's empirical capabilities using the image completion task, and obtain
strong quantitative results on the graph anomaly detection task
Using PPI network autocorrelation in hierarchical multi-label classification trees for gene function prediction
BACKGROUND: Ontologies and catalogs of gene functions, such as the Gene Ontology (GO) and MIPS-FUN, assume that functional classes are organized hierarchically, that is, general functions include more specific ones. This has recently motivated the development of several machine learning algorithms for gene function prediction that leverages on this hierarchical organization where instances may belong to multiple classes. In addition, it is possible to exploit relationships among examples, since it is plausible that related genes tend to share functional annotations. Although these relationships have been identified and extensively studied in the area of protein-protein interaction (PPI) networks, they have not received much attention in hierarchical and multi-class gene function prediction. Relations between genes introduce autocorrelation in functional annotations and violate the assumption that instances are independently and identically distributed (i.i.d.), which underlines most machine learning algorithms. Although the explicit consideration of these relations brings additional complexity to the learning process, we expect substantial benefits in predictive accuracy of learned classifiers. RESULTS: This article demonstrates the benefits (in terms of predictive accuracy) of considering autocorrelation in multi-class gene function prediction. We develop a tree-based algorithm for considering network autocorrelation in the setting of Hierarchical Multi-label Classification (HMC). We empirically evaluate the proposed algorithm, called NHMC (Network Hierarchical Multi-label Classification), on 12 yeast datasets using each of the MIPS-FUN and GO annotation schemes and exploiting 2 different PPI networks. The results clearly show that taking autocorrelation into account improves the predictive performance of the learned models for predicting gene function. CONCLUSIONS: Our newly developed method for HMC takes into account network information in the learning phase: When used for gene function prediction in the context of PPI networks, the explicit consideration of network autocorrelation increases the predictive performance of the learned models. Overall, we found that this holds for different gene features/ descriptions, functional annotation schemes, and PPI networks: Best results are achieved when the PPI network is dense and contains a large proportion of function-relevant interactions
Advances in quantum machine learning
Here we discuss advances in the field of quantum machine learning. The
following document offers a hybrid discussion; both reviewing the field as it
is currently, and suggesting directions for further research. We include both
algorithms and experimental implementations in the discussion. The field's
outlook is generally positive, showing significant promise. However, we believe
there are appreciable hurdles to overcome before one can claim that it is a
primary application of quantum computation.Comment: 38 pages, 17 Figure
- …