29,016 research outputs found
Human-Guided Learning of Column Networks: Augmenting Deep Learning with Advice
Recently, deep models have been successfully applied in several applications,
especially with low-level representations. However, sparse, noisy samples and
structured domains (with multiple objects and interactions) are some of the
open challenges in most deep models. Column Networks, a deep architecture, can
succinctly capture such domain structure and interactions, but may still be
prone to sub-optimal learning from sparse and noisy samples. Inspired by the
success of human-advice guided learning in AI, especially in data-scarce
domains, we propose Knowledge-augmented Column Networks that leverage human
advice/knowledge for better learning with noisy/sparse samples. Our experiments
demonstrate that our approach leads to either superior overall performance or
faster convergence (i.e., both effective and efficient).Comment: Under Review at 'Machine Learning Journal' (MLJ
Deep Learning for LiDAR Point Clouds in Autonomous Driving: A Review
Recently, the advancement of deep learning in discriminative feature learning
from 3D LiDAR data has led to rapid development in the field of autonomous
driving. However, automated processing uneven, unstructured, noisy, and massive
3D point clouds is a challenging and tedious task. In this paper, we provide a
systematic review of existing compelling deep learning architectures applied in
LiDAR point clouds, detailing for specific tasks in autonomous driving such as
segmentation, detection, and classification. Although several published
research papers focus on specific topics in computer vision for autonomous
vehicles, to date, no general survey on deep learning applied in LiDAR point
clouds for autonomous vehicles exists. Thus, the goal of this paper is to
narrow the gap in this topic. More than 140 key contributions in the recent
five years are summarized in this survey, including the milestone 3D deep
architectures, the remarkable deep learning applications in 3D semantic
segmentation, object detection, and classification; specific datasets,
evaluation metrics, and the state of the art performance. Finally, we conclude
the remaining challenges and future researches.Comment: 21 pages, submitted to IEEE Transactions on Neural Networks and
Learning System
A simple yet effective baseline for non-attributed graph classification
Graphs are complex objects that do not lend themselves easily to typical
learning tasks. Recently, a range of approaches based on graph kernels or graph
neural networks have been developed for graph classification and for
representation learning on graphs in general. As the developed methodologies
become more sophisticated, it is important to understand which components of
the increasingly complex methods are necessary or most effective.
As a first step, we develop a simple yet meaningful graph representation, and
explore its effectiveness in graph classification. We test our baseline
representation for the graph classification task on a range of graph datasets.
Interestingly, this simple representation achieves similar performance as the
state-of-the-art graph kernels and graph neural networks for non-attributed
graph classification. Its performance on classifying attributed graphs is
slightly weaker as it does not incorporate attributes. However, given its
simplicity and efficiency, we believe that it still serves as an effective
baseline for attributed graph classification. Our graph representation is
efficient (linear-time) to compute. We also provide a simple connection with
the graph neural networks.
Note that these observations are only for the task of graph classification
while existing methods are often designed for a broader scope including node
embedding and link prediction. The results are also likely biased due to the
limited amount of benchmark datasets available. Nevertheless, the good
performance of our simple baseline calls for the development of new, more
comprehensive benchmark datasets so as to better evaluate and analyze different
graph learning methods. Furthermore, given the computational efficiency of our
graph summary, we believe that it is a good candidate as a baseline method for
future graph classification (or even other graph learning) studies.Comment: 13 pages. Shorter version appears at 2019 ICLR Workshop:
Representation Learning on Graphs and Manifolds. arXiv admin note: text
overlap with arXiv:1810.00826 by other author
Novel deep learning methods for track reconstruction
For the past year, the HEP.TrkX project has been investigating machine
learning solutions to LHC particle track reconstruction problems. A variety of
models were studied that drew inspiration from computer vision applications and
operated on an image-like representation of tracking detector data. While these
approaches have shown some promise, image-based methods face challenges in
scaling up to realistic HL-LHC data due to high dimensionality and sparsity. In
contrast, models that can operate on the spacepoint representation of track
measurements ("hits") can exploit the structure of the data to solve tasks
efficiently. In this paper we will show two sets of new deep learning models
for reconstructing tracks using space-point data arranged as sequences or
connected graphs. In the first set of models, Recurrent Neural Networks (RNNs)
are used to extrapolate, build, and evaluate track candidates akin to Kalman
Filter algorithms. Such models can express their own uncertainty when trained
with an appropriate likelihood loss function. The second set of models use
Graph Neural Networks (GNNs) for the tasks of hit classification and segment
classification. These models read a graph of connected hits and compute
features on the nodes and edges. They adaptively learn which hit connections
are important and which are spurious. The models are scaleable with simple
architecture and relatively few parameters. Results for all models will be
presented on ACTS generic detector simulated data.Comment: CTD 2018 proceeding
Drug-Drug Adverse Effect Prediction with Graph Co-Attention
Complex or co-existing diseases are commonly treated using drug combinations,
which can lead to higher risk of adverse side effects. The detection of
polypharmacy side effects is usually done in Phase IV clinical trials, but
there are still plenty which remain undiscovered when the drugs are put on the
market. Such accidents have been affecting an increasing proportion of the
population (15% in the US now) and it is thus of high interest to be able to
predict the potential side effects as early as possible. Systematic
combinatorial screening of possible drug-drug interactions (DDI) is challenging
and expensive. However, the recent significant increases in data availability
from pharmaceutical research and development efforts offer a novel paradigm for
recovering relevant insights for DDI prediction. Accordingly, several recent
approaches focus on curating massive DDI datasets (with millions of examples)
and training machine learning models on them. Here we propose a neural network
architecture able to set state-of-the-art results on this task---using the type
of the side-effect and the molecular structure of the drugs alone---by
leveraging a co-attentional mechanism. In particular, we show the importance of
integrating joint information from the drug pairs early on when learning each
drug's representation.Comment: 8 pages, 5 figure
Strategies for Pre-training Graph Neural Networks
Many applications of machine learning require a model to make accurate
pre-dictions on test examples that are distributionally different from training
ones, while task-specific labels are scarce during training. An effective
approach to this challenge is to pre-train a model on related tasks where data
is abundant, and then fine-tune it on a downstream task of interest. While
pre-training has been effective in many language and vision domains, it remains
an open question how to effectively use pre-training on graph datasets. In this
paper, we develop a new strategy and self-supervised methods for pre-training
Graph Neural Networks (GNNs). The key to the success of our strategy is to
pre-train an expressive GNN at the level of individual nodes as well as entire
graphs so that the GNN can learn useful local and global representations
simultaneously. We systematically study pre-training on multiple graph
classification datasets. We find that naive strategies, which pre-train GNNs at
the level of either entire graphs or individual nodes, give limited improvement
and can even lead to negative transfer on many downstream tasks. In contrast,
our strategy avoids negative transfer and improves generalization significantly
across downstream tasks, leading up to 9.4% absolute improvements in ROC-AUC
over non-pre-trained models and achieving state-of-the-art performance for
molecular property prediction and protein function prediction.Comment: Accepted as a spotlight to ICLR 202
Face Recognition: From Traditional to Deep Learning Methods
Starting in the seventies, face recognition has become one of the most
researched topics in computer vision and biometrics. Traditional methods based
on hand-crafted features and traditional machine learning techniques have
recently been superseded by deep neural networks trained with very large
datasets. In this paper we provide a comprehensive and up-to-date literature
review of popular face recognition methods including both traditional
(geometry-based, holistic, feature-based and hybrid methods) and deep learning
methods
Face Recognition: A Novel Multi-Level Taxonomy based Survey
In a world where security issues have been gaining growing importance, face
recognition systems have attracted increasing attention in multiple application
areas, ranging from forensics and surveillance to commerce and entertainment.
To help understanding the landscape and abstraction levels relevant for face
recognition systems, face recognition taxonomies allow a deeper dissection and
comparison of the existing solutions. This paper proposes a new, more
encompassing and richer multi-level face recognition taxonomy, facilitating the
organization and categorization of available and emerging face recognition
solutions; this taxonomy may also guide researchers in the development of more
efficient face recognition solutions. The proposed multi-level taxonomy
considers levels related to the face structure, feature support and feature
extraction approach. Following the proposed taxonomy, a comprehensive survey of
representative face recognition solutions is presented. The paper concludes
with a discussion on current algorithmic and application related challenges
which may define future research directions for face recognition.Comment: This paper is a preprint of a paper submitted to IET Biometrics. If
accepted, the copy of record will be available at the IET Digital Librar
forgeNet: A graph deep neural network model using tree-based ensemble classifiers for feature extraction
A unique challenge in predictive model building for omics data has been the
small number of samples versus the large amount of features . This
"" property brings difficulties for disease outcome classification
using deep learning techniques. Sparse learning by incorporating external gene
network information such as the graph-embedded deep feedforward network (GEDFN)
model has been a solution to this issue. However, such methods require an
existing feature graph, and potential mis-specification of the feature graph
can be harmful on classification and feature selection. To address this
limitation and develop a robust classification model without relying on
external knowledge, we propose a \underline{for}est
\underline{g}raph-\underline{e}mbedded deep feedforward \underline{net}work
(forgeNet) model, to integrate the GEDFN architecture with a forest feature
graph extractor, so that the feature graph can be learned in a supervised
manner and specifically constructed for a given prediction task. To validate
the method's capability, we experimented the forgeNet model with both synthetic
and real datasets. The resulting high classification accuracy suggests that the
method is a valuable addition to sparse deep learning models for omics data
Fractional Local Neighborhood Intensity Pattern for Image Retrieval using Genetic Algorithm
In this paper, a new texture descriptor named "Fractional Local Neighborhood
Intensity Pattern" (FLNIP) has been proposed for content based image retrieval
(CBIR). It is an extension of the Local Neighborhood Intensity Pattern
(LNIP)[1]. FLNIP calculates the relative intensity difference between a
particular pixel and the center pixel of a 3x3 window by considering the
relationship with adjacent neighbors. In this work, the fractional change in
the local neighborhood involving the adjacent neighbors has been calculated
first with respect to one of the eight neighbors of the center pixel of a 3x3
window. Next, the fractional change has been calculated with respect to the
center itself. The two values of fractional change are next compared to
generate a binary bit pattern. Both sign and magnitude information are encoded
in a single descriptor as it deals with the relative change in magnitude in the
adjacent neighborhood i.e., the comparison of the fractional change. The
descriptor is applied on four multi-resolution images -- one being the raw
image and the other three being filtered gaussian images obtained by applying
gaussian filters of different standard deviations on the raw image to signify
the importance of exploring texture information at different resolutions in an
image. The four sets of distances obtained between the query and the target
image are then combined with a genetic algorithm based approach to improve the
retrieval performance by minimizing the distance between similar class images.
The performance of the method has been tested for image retrieval on four
popular databases. The precision and recall values observed on these databases
have been compared with recent state-of-art local patterns. The proposed method
has shown a significant improvement over many other existing methods.Comment: MTAP, Springer(Minor Revision
- …