191,659 research outputs found
Maximum A Posteriori Inference in Sum-Product Networks
Sum-product networks (SPNs) are a class of probabilistic graphical models
that allow tractable marginal inference. However, the maximum a posteriori
(MAP) inference in SPNs is NP-hard. We investigate MAP inference in SPNs from
both theoretical and algorithmic perspectives. For the theoretical part, we
reduce general MAP inference to its special case without evidence and hidden
variables; we also show that it is NP-hard to approximate the MAP problem to
for fixed , where is the input size.
For the algorithmic part, we first present an exact MAP solver that runs
reasonably fast and could handle SPNs with up to 1k variables and 150k arcs in
our experiments. We then present a new approximate MAP solver with a good
balance between speed and accuracy, and our comprehensive experiments on
real-world datasets show that it has better overall performance than existing
approximate solvers
Detection for 5G-NOMA: An Online Adaptive Machine Learning Approach
Non-orthogonal multiple access (NOMA) has emerged as a promising radio access
technique for enabling the performance enhancements promised by the
fifth-generation (5G) networks in terms of connectivity, low latency, and high
spectrum efficiency. In the NOMA uplink, successive interference cancellation
(SIC) based detection with device clustering has been suggested. In the case of
multiple receive antennas, SIC can be combined with the minimum mean-squared
error (MMSE) beamforming. However, there exists a tradeoff between the NOMA
cluster size and the incurred SIC error. Larger clusters lead to larger errors
but they are desirable from the spectrum efficiency and connectivity point of
view. We propose a novel online learning based detection for the NOMA uplink.
In particular, we design an online adaptive filter in the sum space of linear
and Gaussian reproducing kernel Hilbert spaces (RKHSs). Such a sum space design
is robust against variations of a dynamic wireless network that can deteriorate
the performance of a purely nonlinear adaptive filter. We demonstrate by
simulations that the proposed method outperforms the MMSE-SIC based detection
for large cluster sizes.Comment: Accepted at ICC 201
Training Input-Output Recurrent Neural Networks through Spectral Methods
We consider the problem of training input-output recurrent neural networks
(RNN) for sequence labeling tasks. We propose a novel spectral approach for
learning the network parameters. It is based on decomposition of the
cross-moment tensor between the output and a non-linear transformation of the
input, based on score functions. We guarantee consistent learning with
polynomial sample and computational complexity under transparent conditions
such as non-degeneracy of model parameters, polynomial activations for the
neurons, and a Markovian evolution of the input sequence. We also extend our
results to Bidirectional RNN which uses both previous and future information to
output the label at each time point, and is employed in many NLP tasks such as
POS tagging
Visualizing and Understanding Sum-Product Networks
Sum-Product Networks (SPNs) are recently introduced deep tractable
probabilistic models by which several kinds of inference queries can be
answered exactly and in a tractable time. Up to now, they have been largely
used as black box density estimators, assessed only by comparing their
likelihood scores only. In this paper we explore and exploit the inner
representations learned by SPNs. We do this with a threefold aim: first we want
to get a better understanding of the inner workings of SPNs; secondly, we seek
additional ways to evaluate one SPN model and compare it against other
probabilistic models, providing diagnostic tools to practitioners; lastly, we
want to empirically evaluate how good and meaningful the extracted
representations are, as in a classic Representation Learning framework. In
order to do so we revise their interpretation as deep neural networks and we
propose to exploit several visualization techniques on their node activations
and network outputs under different types of inference queries. To investigate
these models as feature extractors, we plug some SPNs, learned in a greedy
unsupervised fashion on image datasets, in supervised classification learning
tasks. We extract several embedding types from node activations by filtering
nodes by their type, by their associated feature abstraction level and by their
scope. In a thorough empirical comparison we prove them to be competitive
against those generated from popular feature extractors as Restricted Boltzmann
Machines. Finally, we investigate embeddings generated from random
probabilistic marginal queries as means to compare other tractable
probabilistic models on a common ground, extending our experiments to Mixtures
of Trees.Comment: Machine Learning Journal paper (First Online), 24 page
- …