583 research outputs found
Graph Laplacian for Semi-Supervised Learning
Semi-supervised learning is highly useful in common scenarios where labeled
data is scarce but unlabeled data is abundant. The graph (or nonlocal)
Laplacian is a fundamental smoothing operator for solving various learning
tasks. For unsupervised clustering, a spectral embedding is often used, based
on graph-Laplacian eigenvectors. For semi-supervised problems, the common
approach is to solve a constrained optimization problem, regularized by a
Dirichlet energy, based on the graph-Laplacian. However, as supervision
decreases, Dirichlet optimization becomes suboptimal. We therefore would like
to obtain a smooth transition between unsupervised clustering and
low-supervised graph-based classification. In this paper, we propose a new type
of graph-Laplacian which is adapted for Semi-Supervised Learning (SSL)
problems. It is based on both density and contrastive measures and allows the
encoding of the labeled data directly in the operator. Thus, we can perform
successfully semi-supervised learning using spectral clustering. The benefits
of our approach are illustrated for several SSL problems.Comment: 12 pages, 6 figure
A Quasi-Wasserstein Loss for Learning Graph Neural Networks
When learning graph neural networks (GNNs) in node-level prediction tasks,
most existing loss functions are applied for each node independently, even if
node embeddings and their labels are non-i.i.d. because of their graph
structures. To eliminate such inconsistency, in this study we propose a novel
Quasi-Wasserstein (QW) loss with the help of the optimal transport defined on
graphs, leading to new learning and prediction paradigms of GNNs. In
particular, we design a "Quasi-Wasserstein" distance between the observed
multi-dimensional node labels and their estimations, optimizing the label
transport defined on graph edges. The estimations are parameterized by a GNN in
which the optimal label transport may determine the graph edge weights
optionally. By reformulating the strict constraint of the label transport to a
Bregman divergence-based regularizer, we obtain the proposed Quasi-Wasserstein
loss associated with two efficient solvers learning the GNN together with
optimal label transport. When predicting node labels, our model combines the
output of the GNN with the residual component provided by the optimal label
transport, leading to a new transductive prediction paradigm. Experiments show
that the proposed QW loss applies to various GNNs and helps to improve their
performance in node-level classification and regression tasks
Computational Approaches to Drug Profiling and Drug-Protein Interactions
Despite substantial increases in R&D spending within the pharmaceutical industry, denovo drug design has become a time-consuming endeavour. High attrition rates led to a
long period of stagnation in drug approvals. Due to the extreme costs associated with
introducing a drug to the market, locating and understanding the reasons for clinical failure
is key to future productivity. As part of this PhD, three main contributions were made in
this respect. First, the web platform, LigNFam enables users to interactively explore
similarity relationships between ‘drug like’ molecules and the proteins they bind. Secondly,
two deep-learning-based binding site comparison tools were developed, competing with
the state-of-the-art over benchmark datasets. The models have the ability to predict offtarget interactions and potential candidates for target-based drug repurposing. Finally, the
open-source ScaffoldGraph software was presented for the analysis of hierarchical scaffold
relationships and has already been used in multiple projects, including integration into a
virtual screening pipeline to increase the tractability of ultra-large screening experiments.
Together, and with existing tools, the contributions made will aid in the understanding of
drug-protein relationships, particularly in the fields of off-target prediction and drug
repurposing, helping to design better drugs faster
A review of technical factors to consider when designing neural networks for semantic segmentation of Earth Observation imagery
Semantic segmentation (classification) of Earth Observation imagery is a
crucial task in remote sensing. This paper presents a comprehensive review of
technical factors to consider when designing neural networks for this purpose.
The review focuses on Convolutional Neural Networks (CNNs), Recurrent Neural
Networks (RNNs), Generative Adversarial Networks (GANs), and transformer
models, discussing prominent design patterns for these ANN families and their
implications for semantic segmentation. Common pre-processing techniques for
ensuring optimal data preparation are also covered. These include methods for
image normalization and chipping, as well as strategies for addressing data
imbalance in training samples, and techniques for overcoming limited data,
including augmentation techniques, transfer learning, and domain adaptation. By
encompassing both the technical aspects of neural network design and the
data-related considerations, this review provides researchers and practitioners
with a comprehensive and up-to-date understanding of the factors involved in
designing effective neural networks for semantic segmentation of Earth
Observation imagery.Comment: 145 pages with 32 figure
An Analytical Performance Evaluation on Multiview Clustering Approaches
The concept of machine learning encompasses a wide variety of different approaches, one of which is called clustering. The data points are grouped together in this approach to the problem. Using a clustering method, it is feasible, given a collection of data points, to classify each data point as belonging to a specific group. This can be done if the algorithm is given the collection of data points. In theory, data points that constitute the same group ought to have attributes and characteristics that are equivalent to one another, however data points that belong to other groups ought to have properties and characteristics that are very different from one another. The generation of multiview data is made possible by recent developments in information collecting technologies. The data were collected from à variety of sources and were analysed using a variety of perspectives. The data in question are what are known as multiview data. On a single view, the conventional clustering algorithms are applied. In spite of this, real-world data are complicated and can be clustered in a variety of different ways, depending on how the data are interpreted. In practise, the real-world data are messy. In recent years, Multiview Clustering, often known as MVC, has garnered an increasing amount of attention due to its goal of utilising complimentary and consensus information derived from different points of view. On the other hand, the vast majority of the systems that are currently available only enable the single-clustering scenario, whereby only makes utilization of a single cluster to split the data. This is the case since there is only one cluster accessible. In light of this, it is absolutely necessary to carry out investigation on the multiview data format. The study work is centred on multiview clustering and how well it performs compared to these other strategies
Inhomogeneous graph trend filtering via a l2,0 cardinality penalty
We study estimation of piecewise smooth signals over a graph. We propose a
-norm penalized Graph Trend Filtering (GTF) model to estimate
piecewise smooth graph signals that exhibits inhomogeneous levels of smoothness
across the nodes. We prove that the proposed GTF model is simultaneously a
k-means clustering on the signal over the nodes and a minimum graph cut on the
edges of the graph, where the clustering and the cut share the same assignment
matrix. We propose two methods to solve the proposed GTF model: a spectral
decomposition method and a method based on simulated annealing. In the
experiment on synthetic and real-world datasets, we show that the proposed GTF
model has a better performances compared with existing approaches on the tasks
of denoising, support recovery and semi-supervised classification. We also show
that the proposed GTF model can be solved more efficiently than existing models
for the dataset with a large edge set.Comment: 21 pages, 3 figures, 4 table
The Role of Synthetic Data in Improving Supervised Learning Methods: The Case of Land Use/Land Cover Classification
A thesis submitted in partial fulfillment of the requirements for the degree of Doctor in Information ManagementIn remote sensing, Land Use/Land Cover (LULC) maps constitute important assets for
various applications, promoting environmental sustainability and good resource management.
Although, their production continues to be a challenging task. There are various factors
that contribute towards the difficulty of generating accurate, timely updated LULC maps,
both via automatic or photo-interpreted LULC mapping. Data preprocessing, being a
crucial step for any Machine Learning task, is particularly important in the remote sensing
domain due to the overwhelming amount of raw, unlabeled data continuously gathered
from multiple remote sensing missions. However a significant part of the state-of-the-art
focuses on scenarios with full access to labeled training data with relatively balanced class
distributions. This thesis focuses on the challenges found in automatic LULC classification
tasks, specifically in data preprocessing tasks. We focus on the development of novel
Active Learning (AL) and imbalanced learning techniques, to improve ML performance in
situations with limited training data and/or the existence of rare classes. We also show
that much of the contributions presented are not only successful in remote sensing problems,
but also in various other multidisciplinary classification problems. The work presented
in this thesis used open access datasets to test the contributions made in imbalanced
learning and AL. All the data pulling, preprocessing and experiments are made available at
https://github.com/joaopfonseca/publications. The algorithmic implementations are made
available in the Python package ml-research at https://github.com/joaopfonseca/ml-research
BitGNN: Unleashing the Performance Potential of Binary Graph Neural Networks on GPUs
Recent studies have shown that Binary Graph Neural Networks (GNNs) are
promising for saving computations of GNNs through binarized tensors. Prior
work, however, mainly focused on algorithm designs or training techniques,
leaving it open to how to materialize the performance potential on accelerator
hardware fully. This work redesigns the binary GNN inference backend from the
efficiency perspective. It fills the gap by proposing a series of abstractions
and techniques to map binary GNNs and their computations best to fit the nature
of bit manipulations on GPUs. Results on real-world graphs with GCNs,
GraphSAGE, and GraphSAINT show that the proposed techniques outperform
state-of-the-art binary GNN implementations by 8-22X with the same accuracy
maintained. BitGNN code is publicly available.Comment: To appear in the International Conference on Supercomputing (ICS'23
Proceedings of the 8th Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE 2023)
This volume gathers the papers presented at the Detection and Classification of Acoustic Scenes and Events 2023 Workshop (DCASE2023), Tampere, Finland, during 21–22 September 2023
Transformers for Capturing Multi-level Graph Structure using Hierarchical Distances
Graph transformers need strong inductive biases to derive meaningful
attention scores. Yet, current proposals rarely address methods capturing
longer ranges, hierarchical structures, or community structures, as they appear
in various graphs such as molecules, social networks, and citation networks. In
this paper, we propose a hierarchy-distance structural encoding (HDSE), which
models a hierarchical distance between the nodes in a graph focusing on its
multi-level, hierarchical nature. In particular, this yields a framework which
can be flexibly integrated with existing graph transformers, allowing for
simultaneous application with other positional representations. Through
extensive experiments on 12 real-world datasets, we demonstrate that our HDSE
method successfully enhances various types of baseline transformers, achieving
state-of-the-art empirical performances on 10 benchmark datasets
- …