48,785 research outputs found
Deep Embedding Kernel
Kernel methods and deep learning are two major branches of machine learning that have achieved numerous successes in both analytics and artificial intelligence. While having their own unique characteristics, both branches work through mapping data to a feature space that is supposedly more favorable towards the given task. This dissertation addresses the strengths and weaknesses of each mapping method through combining them and forming a family of novel deep architectures that center around the Deep Embedding Kernel (DEK). In short, DEK is a realization of a kernel function through a newly deep architecture. The mapping in DEK is both implicit (like in kernel methods) and learnable (like in deep learning). Prior to DEK, we proposed a less advanced architecture called Deep Kernel for the tasks of classification and visualization. More recently, we integrate DEK with the novel Dual Deep Learning framework to model big unstructured data. Using DEK as a core component, we further propose two machine learning models: Deep Similarity-Enhanced K Nearest Neighbors (DSE-KNN) and Recurrent Embedding Kernel (REK). Both models have their mappings trained towards optimizing data instances\u27 neighborhoods in the feature space. REK is specifically designed for time series data. Experimental studies throughout the dissertation show that the proposed models have competitive performance to other commonly used and state-of-the-art machine learning models in their given tasks
Deep Metric Learning and Image Classification with Nearest Neighbour Gaussian Kernels
We present a Gaussian kernel loss function and training algorithm for
convolutional neural networks that can be directly applied to both distance
metric learning and image classification problems. Our method treats all
training features from a deep neural network as Gaussian kernel centres and
computes loss by summing the influence of a feature's nearby centres in the
feature embedding space. Our approach is made scalable by treating it as an
approximate nearest neighbour search problem. We show how to make end-to-end
learning feasible, resulting in a well formed embedding space, in which
semantically related instances are likely to be located near one another,
regardless of whether or not the network was trained on those classes. Our
approach outperforms state-of-the-art deep metric learning approaches on
embedding learning challenges, as well as conventional softmax classification
on several datasets.Comment: Accepted in the International Conference on Image Processing (ICIP)
2018. Formerly titled Nearest Neighbour Radial Basis Function Solvers for
Deep Neural Network
Nonlinear functional regression by functional deep neural network with kernel embedding
With the rapid development of deep learning in various fields of science and
technology, such as speech recognition, image classification, and natural
language processing, recently it is also widely applied in the functional data
analysis (FDA) with some empirical success. However, due to the infinite
dimensional input, we need a powerful dimension reduction method for functional
learning tasks, especially for the nonlinear functional regression. In this
paper, based on the idea of smooth kernel integral transformation, we propose a
functional deep neural network with an efficient and fully data-dependent
dimension reduction method. The architecture of our functional net consists of
a kernel embedding step: an integral transformation with a data-dependent
smooth kernel; a projection step: a dimension reduction by projection with
eigenfunction basis based on the embedding kernel; and finally an expressive
deep ReLU neural network for the prediction. The utilization of smooth kernel
embedding enables our functional net to be discretization invariant, efficient,
and robust to noisy observations, capable of utilizing information in both
input functions and responses data, and have a low requirement on the number of
discrete points for an unimpaired generalization performance. We conduct
theoretical analysis including approximation error and generalization error
analysis, and numerical simulations to verify these advantages of our
functional net
Deep learning for extracting protein-protein interactions from biomedical literature
State-of-the-art methods for protein-protein interaction (PPI) extraction are
primarily feature-based or kernel-based by leveraging lexical and syntactic
information. But how to incorporate such knowledge in the recent deep learning
methods remains an open question. In this paper, we propose a multichannel
dependency-based convolutional neural network model (McDepCNN). It applies one
channel to the embedding vector of each word in the sentence, and another
channel to the embedding vector of the head of the corresponding word.
Therefore, the model can use richer information obtained from different
channels. Experiments on two public benchmarking datasets, AIMed and BioInfer,
demonstrate that McDepCNN compares favorably to the state-of-the-art
rich-feature and single-kernel based methods. In addition, McDepCNN achieves
24.4% relative improvement in F1-score over the state-of-the-art methods on
cross-corpus evaluation and 12% improvement in F1-score over kernel-based
methods on "difficult" instances. These results suggest that McDepCNN
generalizes more easily over different corpora, and is capable of capturing
long distance features in the sentences.Comment: Accepted for publication in Proceedings of the 2017 Workshop on
Biomedical Natural Language Processing, 10 pages, 2 figures, 6 table
Learning Combinatorial Embedding Networks for Deep Graph Matching
Graph matching refers to finding node correspondence between graphs, such
that the corresponding node and edge's affinity can be maximized. In addition
with its NP-completeness nature, another important challenge is effective
modeling of the node-wise and structure-wise affinity across graphs and the
resulting objective, to guide the matching procedure effectively finding the
true matching against noises. To this end, this paper devises an end-to-end
differentiable deep network pipeline to learn the affinity for graph matching.
It involves a supervised permutation loss regarding with node correspondence to
capture the combinatorial nature for graph matching. Meanwhile deep graph
embedding models are adopted to parameterize both intra-graph and cross-graph
affinity functions, instead of the traditional shallow and simple parametric
forms e.g. a Gaussian kernel. The embedding can also effectively capture the
higher-order structure beyond second-order edges. The permutation loss model is
agnostic to the number of nodes, and the embedding model is shared among nodes
such that the network allows for varying numbers of nodes in graphs for
training and inference. Moreover, our network is class-agnostic with some
generalization capability across different categories. All these features are
welcomed for real-world applications. Experiments show its superiority against
state-of-the-art graph matching learning methods.Comment: ICCV2019 oral. Code available at
https://github.com/Thinklab-SJTU/PCA-G
- …