34,270 research outputs found
Integration of multiple networks for robust label propagation
Transductive inference on graphs such as label propagation algorithms is receiving a lot of attention. In this paper, we address a label propagation problem on multiple networks and present a new algorithm that automatically integrates structure information brought in by multiple networks. The proposed method is robust in that irrelevant networks are automatically deemphasized, which is an advantage over Tsuda et al.’s approach [14]. We also show that the proposed algorithm can be interpreted as an EM algorithm with a Student-t prior. Finally, we demonstrate the usefulness of our method in protein function prediction
A Survey on Data Collection for Machine Learning: a Big Data -- AI Integration Perspective
Data collection is a major bottleneck in machine learning and an active
research topic in multiple communities. There are largely two reasons data
collection has recently become a critical issue. First, as machine learning is
becoming more widely-used, we are seeing new applications that do not
necessarily have enough labeled data. Second, unlike traditional machine
learning, deep learning techniques automatically generate features, which saves
feature engineering costs, but in return may require larger amounts of labeled
data. Interestingly, recent research in data collection comes not only from the
machine learning, natural language, and computer vision communities, but also
from the data management community due to the importance of handling large
amounts of data. In this survey, we perform a comprehensive study of data
collection from a data management point of view. Data collection largely
consists of data acquisition, data labeling, and improvement of existing data
or models. We provide a research landscape of these operations, provide
guidelines on which technique to use when, and identify interesting research
challenges. The integration of machine learning and data management for data
collection is part of a larger trend of Big data and Artificial Intelligence
(AI) integration and opens many opportunities for new research.Comment: 20 page
Spatially Constrained Location Prior for Scene Parsing
Semantic context is an important and useful cue for scene parsing in
complicated natural images with a substantial amount of variations in objects
and the environment. This paper proposes Spatially Constrained Location Prior
(SCLP) for effective modelling of global and local semantic context in the
scene in terms of inter-class spatial relationships. Unlike existing studies
focusing on either relative or absolute location prior of objects, the SCLP
effectively incorporates both relative and absolute location priors by
calculating object co-occurrence frequencies in spatially constrained image
blocks. The SCLP is general and can be used in conjunction with various visual
feature-based prediction models, such as Artificial Neural Networks and Support
Vector Machine (SVM), to enforce spatial contextual constraints on class
labels. Using SVM classifiers and a linear regression model, we demonstrate
that the incorporation of SCLP achieves superior performance compared to the
state-of-the-art methods on the Stanford background and SIFT Flow datasets.Comment: authors' pre-print version of a article published in IJCNN 201
On Extending Neural Networks with Loss Ensembles for Text Classification
Ensemble techniques are powerful approaches that combine several weak
learners to build a stronger one. As a meta learning framework, ensemble
techniques can easily be applied to many machine learning techniques. In this
paper we propose a neural network extended with an ensemble loss function for
text classification. The weight of each weak loss function is tuned within the
training phase through the gradient propagation optimization method of the
neural network. The approach is evaluated on several text classification
datasets. We also evaluate its performance in various environments with several
degrees of label noise. Experimental results indicate an improvement of the
results and strong resilience against label noise in comparison with other
methods.Comment: 5 pages, 5 tables, 1 figure. Camera-ready submitted to The 2017
Australasian Language Technology Association Workshop (ALTA 2017
Progressive Joint Modeling in Unsupervised Single-channel Overlapped Speech Recognition
Unsupervised single-channel overlapped speech recognition is one of the
hardest problems in automatic speech recognition (ASR). Permutation invariant
training (PIT) is a state of the art model-based approach, which applies a
single neural network to solve this single-input, multiple-output modeling
problem. We propose to advance the current state of the art by imposing a
modular structure on the neural network, applying a progressive pretraining
regimen, and improving the objective function with transfer learning and a
discriminative training criterion. The modular structure splits the problem
into three sub-tasks: frame-wise interpreting, utterance-level speaker tracing,
and speech recognition. The pretraining regimen uses these modules to solve
progressively harder tasks. Transfer learning leverages parallel clean speech
to improve the training targets for the network. Our discriminative training
formulation is a modification of standard formulations, that also penalizes
competing outputs of the system. Experiments are conducted on the artificial
overlapped Switchboard and hub5e-swb dataset. The proposed framework achieves
over 30% relative improvement of WER over both a strong jointly trained system,
PIT for ASR, and a separately optimized system, PIT for speech separation with
clean speech ASR model. The improvement comes from better model generalization,
training efficiency and the sequence level linguistic knowledge integration.Comment: submitted to TASLP, 07/20/2017; accepted by TASLP, 10/13/201
CREST: Convolutional Residual Learning for Visual Tracking
Discriminative correlation filters (DCFs) have been shown to perform
superiorly in visual tracking. They only need a small set of training samples
from the initial frame to generate an appearance model. However, existing DCFs
learn the filters separately from feature extraction, and update these filters
using a moving average operation with an empirical weight. These DCF trackers
hardly benefit from the end-to-end training. In this paper, we propose the
CREST algorithm to reformulate DCFs as a one-layer convolutional neural
network. Our method integrates feature extraction, response map generation as
well as model update into the neural networks for an end-to-end training. To
reduce model degradation during online update, we apply residual learning to
take appearance changes into account. Extensive experiments on the benchmark
datasets demonstrate that our CREST tracker performs favorably against
state-of-the-art trackers.Comment: ICCV 2017. Project page:
http://www.cs.cityu.edu.hk/~yibisong/iccv17/index.htm
Multi-Layered Gradient Boosting Decision Trees
Multi-layered representation is believed to be the key ingredient of deep
neural networks especially in cognitive tasks like computer vision. While
non-differentiable models such as gradient boosting decision trees (GBDTs) are
the dominant methods for modeling discrete or tabular data, they are hard to
incorporate with such representation learning ability. In this work, we propose
the multi-layered GBDT forest (mGBDTs), with an explicit emphasis on exploring
the ability to learn hierarchical representations by stacking several layers of
regression GBDTs as its building block. The model can be jointly trained by a
variant of target propagation across layers, without the need to derive
back-propagation nor differentiability. Experiments and visualizations
confirmed the effectiveness of the model in terms of performance and
representation learning ability
An Integrated, Conditional Model of Information Extraction and Coreference with Applications to Citation Matching
Although information extraction and coreference resolution appear together in
many applications, most current systems perform them as ndependent steps. This
paper describes an approach to integrated inference for extraction and
coreference based on conditionally-trained undirected graphical models. We
discuss the advantages of conditional probability training, and of a
coreference model structure based on graph partitioning. On a data set of
research paper citations, we show significant reduction in error by using
extraction uncertainty to improve coreference citation matching accuracy, and
using coreference to improve the accuracy of the extracted fields.Comment: Appears in Proceedings of the Twentieth Conference on Uncertainty in
Artificial Intelligence (UAI2004
Visual Object Categorization Based on Hierarchical Shape Motifs Learned From Noisy Point Cloud Decompositions
Object shape is a key cue that contributes to the semantic understanding of
objects. In this work we focus on the categorization of real-world object point
clouds to particular shape types. Therein surface description and
representation of object shape structure have significant influence on shape
categorization accuracy, when dealing with real-world scenes featuring noisy,
partial and occluded object observations. An unsupervised hierarchical learning
procedure is utilized here to symbolically describe surface characteristics on
multiple semantic levels. Furthermore, a constellation model is proposed that
hierarchically decomposes objects. The decompositions are described as
constellations of symbols (shape motifs) in a gradual order, hence reflecting
shape structure from local to global, i.e., from parts over groups of parts to
entire objects. The combination of this multi-level description of surfaces and
the hierarchical decomposition of shapes leads to a representation which allows
to conceptualize shapes. An object discrimination has been observed in
experiments with seven categories featuring instances with sensor noise,
occlusions as well as inter-category and intra-category similarities.
Experiments include the evaluation of the proposed description and shape
decomposition approach, and comparisons to Fast Point Feature Histograms, a
Vocabulary Tree and a neural network-based Deep Learning method. Furthermore,
experiments are conducted with alternative datasets which analyze the
generalization capability of the proposed approach
Stable Architectures for Deep Neural Networks
Deep neural networks have become invaluable tools for supervised machine
learning, e.g., classification of text or images. While often offering superior
results over traditional techniques and successfully expressing complicated
patterns in data, deep architectures are known to be challenging to design and
train such that they generalize well to new data. Important issues with deep
architectures are numerical instabilities in derivative-based learning
algorithms commonly called exploding or vanishing gradients. In this paper we
propose new forward propagation techniques inspired by systems of Ordinary
Differential Equations (ODE) that overcome this challenge and lead to
well-posed learning problems for arbitrarily deep networks.
The backbone of our approach is our interpretation of deep learning as a
parameter estimation problem of nonlinear dynamical systems. Given this
formulation, we analyze stability and well-posedness of deep learning and use
this new understanding to develop new network architectures. We relate the
exploding and vanishing gradient phenomenon to the stability of the discrete
ODE and present several strategies for stabilizing deep learning for very deep
networks. While our new architectures restrict the solution space, several
numerical experiments show their competitiveness with state-of-the-art
networks.Comment: 23 pages, 7 figure
- …