1,006 research outputs found
Predicting Graph Signals using Kernel Regression where the Input Signal is Agnostic to a Graph
We propose a kernel regression method to predict a target signal lying over a
graph when an input observation is given. The input and the output could be two
different physical quantities. In particular, the input may not be a graph
signal at all or it could be agnostic to an underlying graph. We use a training
dataset to learn the proposed regression model by formulating it as a convex
optimization problem, where we use a graph-Laplacian based regularization to
enforce that the predicted target is a graph signal. Once the model is learnt,
it can be directly used on a large number of test data points one-by-one
independently to predict the corresponding targets. Our approach employs
kernels between the various input observations, and as a result the kernels are
not restricted to be functions of the graph adjacency/Laplacian matrix. We show
that the proposed kernel regression exhibits a smoothing effect, while
simultaneously achieving noise-reduction and graph-smoothness. We then extend
our method to the case when the underlying graph may not be known apriori, by
simultaneously learning an underlying graph and the regression coefficients.
Using extensive experiments, we show that our method provides a good prediction
performance in adverse conditions, particularly when the training data is
limited in size and is noisy. In graph signal reconstruction experiments, our
method is shown to provide a good performance even for a highly
under-determined subsampling
Semi-Supervised Sparse Coding
Sparse coding approximates the data sample as a sparse linear combination of
some basic codewords and uses the sparse codes as new presentations. In this
paper, we investigate learning discriminative sparse codes by sparse coding in
a semi-supervised manner, where only a few training samples are labeled. By
using the manifold structure spanned by the data set of both labeled and
unlabeled samples and the constraints provided by the labels of the labeled
samples, we learn the variable class labels for all the samples. Furthermore,
to improve the discriminative ability of the learned sparse codes, we assume
that the class labels could be predicted from the sparse codes directly using a
linear classifier. By solving the codebook, sparse codes, class labels and
classifier parameters simultaneously in a unified objective function, we
develop a semi-supervised sparse coding algorithm. Experiments on two
real-world pattern recognition problems demonstrate the advantage of the
proposed methods over supervised sparse coding methods on partially labeled
data sets
CURE: Curvature Regularization For Missing Data Recovery
Missing data recovery is an important and yet challenging problem in imaging
and data science. Successful models often adopt certain carefully chosen
regularization. Recently, the low dimension manifold model (LDMM) was
introduced by S.Osher et al. and shown effective in image inpainting. They
observed that enforcing low dimensionality on image patch manifold serves as a
good image regularizer. In this paper, we observe that having only the low
dimension manifold regularization is not enough sometimes, and we need
smoothness as well. For that, we introduce a new regularization by combining
the low dimension manifold regularization with a higher order Curvature
Regularization, and we call this new regularization CURE for short. The key
step of solving CURE is to solve a biharmonic equation on a manifold. We
further introduce a weighted version of CURE, called WeCURE, in a similar
manner as the weighted nonlocal Laplacian (WNLL) method. Numerical experiments
for image inpainting and semi-supervised learning show that the proposed CURE
and WeCURE significantly outperform LDMM and WNLL respectively.Comment: 17 pages, 7 figures, 4 table
Graph Wavelets via Sparse Cuts: Extended Version
Modeling information that resides on vertices of large graphs is a key
problem in several real-life applications, ranging from social networks to the
Internet-of-things. Signal Processing on Graphs and, in particular, graph
wavelets can exploit the intrinsic smoothness of these datasets in order to
represent them in a both compact and accurate manner. However, how to discover
wavelet bases that capture the geometry of the data with respect to the signal
as well as the graph structure remains an open question. In this paper, we
study the problem of computing graph wavelet bases via sparse cuts in order to
produce low-dimensional encodings of data-driven bases. This problem is
connected to known hard problems in graph theory (e.g. multiway cuts) and thus
requires an efficient heuristic. We formulate the basis discovery task as a
relaxation of a vector optimization problem, which leads to an elegant solution
as a regularized eigenvalue computation. Moreover, we propose several
strategies in order to scale our algorithm to large graphs. Experimental
results show that the proposed algorithm can effectively encode both the graph
structure and signal, producing compressed and accurate representations for
vertex values in a wide range of datasets (e.g. sensor and gene networks) and
significantly outperforming the best baseline
Hyperspectral Image Classification with Markov Random Fields and a Convolutional Neural Network
This paper presents a new supervised classification algorithm for remotely
sensed hyperspectral image (HSI) which integrates spectral and spatial
information in a unified Bayesian framework. First, we formulate the HSI
classification problem from a Bayesian perspective. Then, we adopt a
convolutional neural network (CNN) to learn the posterior class distributions
using a patch-wise training strategy to better use the spatial information.
Next, spatial information is further considered by placing a spatial smoothness
prior on the labels. Finally, we iteratively update the CNN parameters using
stochastic gradient decent (SGD) and update the class labels of all pixel
vectors using an alpha-expansion min-cut-based algorithm. Compared with other
state-of-the-art methods, the proposed classification method achieves better
performance on one synthetic dataset and two benchmark HSI datasets in a number
of experimental settings
Deep Neural Networks
Deep Neural Networks (DNNs) are universal function approximators providing
state-of- the-art solutions on wide range of applications. Common perceptual
tasks such as speech recognition, image classification, and object tracking are
now commonly tackled via DNNs. Some fundamental problems remain: (1) the lack
of a mathematical framework providing an explicit and interpretable
input-output formula for any topology, (2) quantification of DNNs stability
regarding adversarial examples (i.e. modified inputs fooling DNN predictions
whilst undetectable to humans), (3) absence of generalization guarantees and
controllable behaviors for ambiguous patterns, (4) leverage unlabeled data to
apply DNNs to domains where expert labeling is scarce as in the medical field.
Answering those points would provide theoretical perspectives for further
developments based on a common ground. Furthermore, DNNs are now deployed in
tremendous societal applications, pushing the need to fill this theoretical gap
to ensure control, reliability, and interpretability.Comment: Technical Repor
Semi-supervised Dictionary Learning Based on Hilbert-Schmidt Independence Criterion
In this paper, a novel semi-supervised dictionary learning and sparse
representation (SS-DLSR) is proposed. The proposed method benefits from the
supervisory information by learning the dictionary in a space where the
dependency between the data and class labels is maximized. This maximization is
performed using Hilbert-Schmidt independence criterion (HSIC). On the other
hand, the global distribution of the underlying manifolds were learned from the
unlabeled data by minimizing the distances between the unlabeled data and the
corresponding nearest labeled data in the space of the dictionary learned. The
proposed SS-DLSR algorithm has closed-form solutions for both the dictionary
and sparse coefficients, and therefore does not have to learn the two
iteratively and alternately as is common in the literature of the DLSR. This
makes the solution for the proposed algorithm very fast. The experiments
confirm the improvement in classification performance on benchmark datasets by
including the information from both labeled and unlabeled data, particularly
when there are many unlabeled data.Comment: Accepted at International conference on Image analysis and
Recognition (ICIAR) 201
Discrete Signal Processing on Graphs: Sampling Theory
We propose a sampling theory for signals that are supported on either
directed or undirected graphs. The theory follows the same paradigm as
classical sampling theory. We show that perfect recovery is possible for graph
signals bandlimited under the graph Fourier transform. The sampled signal
coefficients form a new graph signal, whose corresponding graph structure
preserves the first-order difference of the original graph signal. For general
graphs, an optimal sampling operator based on experimentally designed sampling
is proposed to guarantee perfect recovery and robustness to noise; for graphs
whose graph Fourier transforms are frames with maximal robustness to erasures
as well as for Erd\H{o}s-R\'enyi graphs, random sampling leads to perfect
recovery with high probability. We further establish the connection to the
sampling theory of finite discrete-time signal processing and previous work on
signal recovery on graphs. To handle full-band graph signals, we propose a
graph filter bank based on sampling theory on graphs. Finally, we apply the
proposed sampling theory to semi-supervised classification on online blogs and
digit images, where we achieve similar or better performance with fewer labeled
samples compared to previous work.Comment: To appear in IEEE T-S
A Spectral Series Approach to High-Dimensional Nonparametric Regression
A key question in modern statistics is how to make fast and reliable
inferences for complex, high-dimensional data. While there has been much
interest in sparse techniques, current methods do not generalize well to data
with nonlinear structure. In this work, we present an orthogonal series
estimator for predictors that are complex aggregate objects, such as natural
images, galaxy spectra, trajectories, and movies. Our series approach ties
together ideas from kernel machine learning, and Fourier methods. We expand the
unknown regression on the data in terms of the eigenfunctions of a kernel-based
operator, and we take advantage of orthogonality of the basis with respect to
the underlying data distribution, P, to speed up computations and tuning of
parameters. If the kernel is appropriately chosen, then the eigenfunctions
adapt to the intrinsic geometry and dimension of the data. We provide
theoretical guarantees for a radial kernel with varying bandwidth, and we
relate smoothness of the regression function with respect to P to sparsity in
the eigenbasis. Finally, using simulated and real-world data, we systematically
compare the performance of the spectral series approach with classical kernel
smoothing, k-nearest neighbors regression, kernel ridge regression, and
state-of-the-art manifold and local regression methods
Learning Structured Ordinal Measures for Video based Face Recognition
This paper presents a structured ordinal measure method for video-based face
recognition that simultaneously learns ordinal filters and structured ordinal
features. The problem is posed as a non-convex integer program problem that
includes two parts. The first part learns stable ordinal filters to project
video data into a large-margin ordinal space. The second seeks self-correcting
and discrete codes by balancing the projected data and a rank-one ordinal
matrix in a structured low-rank way. Unsupervised and supervised structures are
considered for the ordinal matrix. In addition, as a complement to hierarchical
structures, deep feature representations are integrated into our method to
enhance coding stability. An alternating minimization method is employed to
handle the discrete and low-rank constraints, yielding high-quality codes that
capture prior structures well. Experimental results on three commonly used face
video databases show that our method with a simple voting classifier can
achieve state-of-the-art recognition rates using fewer features and samples
- …