1,713 research outputs found
Information Theoretical Estimators Toolbox
We present ITE (information theoretical estimators) a free and open source,
multi-platform, Matlab/Octave toolbox that is capable of estimating many
different variants of entropy, mutual information, divergence, association
measures, cross quantities, and kernels on distributions. Thanks to its highly
modular design, ITE supports additionally (i) the combinations of the
estimation techniques, (ii) the easy construction and embedding of novel
information theoretical estimators, and (iii) their immediate application in
information theoretical optimization problems. ITE also includes a prototype
application in a central problem class of signal processing, independent
subspace analysis and its extensions.Comment: 5 pages; ITE toolbox: https://bitbucket.org/szzoli/ite
Transfer Learning for Speech and Language Processing
Transfer learning is a vital technique that generalizes models trained for
one setting or task to other settings or tasks. For example in speech
recognition, an acoustic model trained for one language can be used to
recognize speech in another language, with little or no re-training data.
Transfer learning is closely related to multi-task learning (cross-lingual vs.
multilingual), and is traditionally studied in the name of `model adaptation'.
Recent advance in deep learning shows that transfer learning becomes much
easier and more effective with high-level abstract features learned by deep
models, and the `transfer' can be conducted not only between data distributions
and data types, but also between model structures (e.g., shallow nets and deep
nets) or even model types (e.g., Bayesian models and neural models). This
review paper summarizes some recent prominent research towards this direction,
particularly for speech and language processing. We also report some results
from our group and highlight the potential of this very interesting research
field.Comment: 13 pages, APSIPA 201
Physiological Gaussian Process Priors for the Hemodynamics in fMRI Analysis
Background: Inference from fMRI data faces the challenge that the hemodynamic
system that relates neural activity to the observed BOLD fMRI signal is
unknown.
New Method: We propose a new Bayesian model for task fMRI data with the
following features: (i) joint estimation of brain activity and the underlying
hemodynamics, (ii) the hemodynamics is modeled nonparametrically with a
Gaussian process (GP) prior guided by physiological information and (iii) the
predicted BOLD is not necessarily generated by a linear time-invariant (LTI)
system. We place a GP prior directly on the predicted BOLD response, rather
than on the hemodynamic response function as in previous literature. This
allows us to incorporate physiological information via the GP prior mean in a
flexible way, and simultaneously gives us the nonparametric flexibility of the
GP.
Results: Results on simulated data show that the proposed model is able to
discriminate between active and non-active voxels also when the GP prior
deviates from the true hemodynamics. Our model finds time varying dynamics when
applied to real fMRI data.
Comparison with Existing Method(s): The proposed model is better at detecting
activity in simulated data than standard models, without inflating the false
positive rate. When applied to real fMRI data, our GP model in several cases
finds brain activity where previously proposed LTI models does not.
Conclusions: We have proposed a new non-linear model for the hemodynamics in
task fMRI, that is able to detect active voxels, and gives the opportunity to
ask new kinds of questions related to hemodynamics.Comment: 18 pages, 14 figure
Improving aircraft performance using machine learning: a review
This review covers the new developments in machine learning (ML) that are
impacting the multi-disciplinary area of aerospace engineering, including
fundamental fluid dynamics (experimental and numerical), aerodynamics,
acoustics, combustion and structural health monitoring. We review the state of
the art, gathering the advantages and challenges of ML methods across different
aerospace disciplines and provide our view on future opportunities. The basic
concepts and the most relevant strategies for ML are presented together with
the most relevant applications in aerospace engineering, revealing that ML is
improving aircraft performance and that these techniques will have a large
impact in the near future
Maximum Entropy Vector Kernels for MIMO system identification
Recent contributions have framed linear system identification as a
nonparametric regularized inverse problem. Relying on -type
regularization which accounts for the stability and smoothness of the impulse
response to be estimated, these approaches have been shown to be competitive
w.r.t classical parametric methods. In this paper, adopting Maximum Entropy
arguments, we derive a new penalty deriving from a vector-valued
kernel; to do so we exploit the structure of the Hankel matrix, thus
controlling at the same time complexity, measured by the McMillan degree,
stability and smoothness of the identified models. As a special case we recover
the nuclear norm penalty on the squared block Hankel matrix. In contrast with
previous literature on reweighted nuclear norm penalties, our kernel is
described by a small number of hyper-parameters, which are iteratively updated
through marginal likelihood maximization; constraining the structure of the
kernel acts as a (hyper)regularizer which helps controlling the effective
degrees of freedom of our estimator. To optimize the marginal likelihood we
adapt a Scaled Gradient Projection (SGP) algorithm which is proved to be
significantly computationally cheaper than other first and second order
off-the-shelf optimization methods. The paper also contains an extensive
comparison with many state-of-the-art methods on several Monte-Carlo studies,
which confirms the effectiveness of our procedure
Frugal Reinforcement-based Active Learning
Most of the existing learning models, particularly deep neural networks, are
reliant on large datasets whose hand-labeling is expensive and time demanding.
A current trend is to make the learning of these models frugal and less
dependent on large collections of labeled data. Among the existing solutions,
deep active learning is currently witnessing a major interest and its purpose
is to train deep networks using as few labeled samples as possible. However,
the success of active learning is highly dependent on how critical are these
samples when training models. In this paper, we devise a novel active learning
approach for label-efficient training. The proposed method is iterative and
aims at minimizing a constrained objective function that mixes diversity,
representativity and uncertainty criteria. The proposed approach is
probabilistic and unifies all these criteria in a single objective function
whose solution models the probability of relevance of samples (i.e., how
critical) when learning a decision function. We also introduce a novel
weighting mechanism based on reinforcement learning, which adaptively balances
these criteria at each training iteration, using a particular stateless
Q-learning model. Extensive experiments conducted on staple image
classification data, including Object-DOTA, show the effectiveness of our
proposed model w.r.t. several baselines including random, uncertainty and flat
as well as other work.Comment: arXiv admin note: text overlap with arXiv:2203.1156
- …