13,992 research outputs found
Normal Factor Graphs as Probabilistic Models
We present a new probabilistic modelling framework based on the recent notion
of normal factor graph (NFG). We show that the proposed NFG models and their
transformations unify some existing models such as factor graphs, convolutional
factor graphs, and cumulative distribution networks. The two subclasses of the
NFG models, namely the constrained and generative models, exhibit a duality in
their dependence structure. Transformation of NFG models further extends the
power of this modelling framework. We point out the well-known NFG
representations of parity and generator realizations of a linear code as
generative and constrained models, and comment on a more prevailing duality in
this context. Finally, we address the algorithmic aspect of computing the
exterior function of NFGs and the inference problem on NFGs
A Dynamic and Incentive Policy for Selecting D2D Mobile Relays
User-to-network relaying enabled via Device-to-Device (D2D) communications is
a promising technique for improving the performance of cellular networks. Since
in practice relays are in mobility, a dynamic relay selection scheme is
unavoidable. In this paper, we propose a dynamic relay selection policy that
maximizes the performance of cellular networks (e.g. throughput, reliability,
coverage) under cost constraints (e.g. transmission power, power budget). We
represent the relays' dynamics as a Markov Decision Process (MDP) and assume
that only the locations of the selected relays are observable. Therefore, the
dynamic relay selection process is modeled as a Constrained Partially
Observable Markov Decision Process (CPOMDP). Since the exact solution of such
framework is intractable to find, we develop a point-based value iteration
solution and evaluate its performance. In addition, we prove the submodularity
property of both the reward and cost value functions and deduce a greedy
solution which is scalable with the number of discovered relays. For the
muti-user scenario, a distributed approach is introduced in order to reduce the
complexity and the overhead of the proposed solution. We illustrate the
numerical results of the scenario where throughput is maximized under energy
constraint and evaluate the gain that the proposed relay selection policy
achieves compared to a traditional cellular network
Modeling Social Networks with Node Attributes using the Multiplicative Attribute Graph Model
Networks arising from social, technological and natural domains exhibit rich
connectivity patterns and nodes in such networks are often labeled with
attributes or features. We address the question of modeling the structure of
networks where nodes have attribute information. We present a Multiplicative
Attribute Graph (MAG) model that considers nodes with categorical attributes
and models the probability of an edge as the product of individual attribute
link formation affinities. We develop a scalable variational expectation
maximization parameter estimation method. Experiments show that MAG model
reliably captures network connectivity as well as provides insights into how
different attributes shape the network structure.Comment: 15 pages, 7 figures, 7 table
Critical Network Cascades with Re-excitable nodes: Why tree-like approximations usually work, when they breakdown, and how to correct them
Network science is a rapidly expanding field, with a large and growing body
of work on network-based dynamical processes. Most theoretical results in this
area rely on the so-called \emph{locally tree-like approximation}. This is,
however, usually an `uncontrolled' approximation, in the sense that the
magnitudes of the error are typically unknown, although numerical results show
that this error is often surprisingly small. In this paper, we place this
approximation on more rigorous footing by calculating the magnitude of
deviations away from tree-based theories in the context of discrete-time
critical network cascades with re-excitable nodes. We discuss the conditions
under which tree-like approximations give good results for calculating network
criticality, and also explain the reasons for deviation from this
approximation, in terms of the density of certain kinds of network motifs.
Using this understanding, we derive results for network criticality that apply
to general networks that explicitly do not satisfy the locally tree-like
approximation. In particular, we focus on the bi-parallel motif, the smallest
motif relevant to the failure of a tree-based theory in this context, and we
derive the corrections due to such motifs on the conditions for criticality. We
verify our claims on computer-generated networks, and we confirm that our
theory accurately predicts the observed deviations from criticality. Using our
theory, we explain why numerical simulations often show that deviations from a
tree-based theory are surprisingly small. More specifically, we show that these
deviations are negligible for networks whose average degree is even modestly
large compared to one, justifying why tree-based theories appear to work well
for most real-world networks.Comment: 22 pages, 8 figure
Neural Likelihoods via Cumulative Distribution Functions
We leverage neural networks as universal approximators of monotonic functions
to build a parameterization of conditional cumulative distribution functions
(CDFs). By the application of automatic differentiation with respect to
response variables and then to parameters of this CDF representation, we are
able to build black box CDF and density estimators. A suite of families is
introduced as alternative constructions for the multivariate case. At one
extreme, the simplest construction is a competitive density estimator against
state-of-the-art deep learning methods, although it does not provide an easily
computable representation of multivariate CDFs. At the other extreme, we have a
flexible construction from which multivariate CDF evaluations and
marginalizations can be obtained by a simple forward pass in a deep neural net,
but where the computation of the likelihood scales exponentially with
dimensionality. Alternatives in between the extremes are discussed. We evaluate
the different representations empirically on a variety of tasks involving tail
area probabilities, tail dependence and (partial) density estimation.Comment: 10 page
Enhancing approximation abilities of neural networks by training derivatives
A method to increase the precision of feedforward networks is proposed. It
requires a prior knowledge of a target function derivatives of several orders
and uses this information in gradient based training. Forward pass calculates
not only the values of the output layer of a network but also their
derivatives. The deviations of those derivatives from the target ones are used
in an extended cost function and then backward pass calculates the gradient of
the extended cost with respect to weights, which can then be used by any
weights update algorithm. Despite a substantial increase in arithmetic
operations per pattern (if compared to the conventional training), the extended
cost allows to obtain 140--1000 times more accurate approximation for simple
cases if the total number of operations is equal. This precision also happens
to be out of reach for the regular cost function. The method fits well into the
procedure of solving differential equations with neural networks. Unlike
training a network to match some target mapping, which requires an explicit use
of the target derivatives in the extended cost function, the cost function for
solving a differential equation is based on the deviation of the equation's
residual from zero and thus can be extended by differentiating the equation
itself, which does not require any prior knowledge. Solving an equation with
such a cost resulted in 13 times more accurate result and could be done with 3
times larger grid step. GPU-efficient algorithm for calculating the gradient of
the extended cost function is proposed
A Dynamical System for PageRank with Time-Dependent Teleportation
We propose a dynamical system that captures changes to the network centrality
of nodes as external interest in those nodes vary. We derive this system by
adding time-dependent teleportation to the PageRank score. The result is not a
single set of importance scores, but rather a time-dependent set. These can be
converted into ranked lists in a variety of ways, for instance, by taking the
largest change in the importance score. For an interesting class of the dynamic
teleportation functions, we derive closed form solutions for the dynamic
PageRank vector. The magnitude of the deviation from a static PageRank vector
is given by a PageRank problem with complex-valued teleportation parameters.
Moreover, these dynamical systems are easy to evaluate. We demonstrate the
utility of dynamic teleportation on both the article graph of Wikipedia, where
the external interest information is given by the number of hourly visitors to
each page, and the Twitter social network, where external interest is the
number of tweets per month. For these problems, we show that using information
from the dynamical system helps improve a prediction task and identify trends
in the data.Comment: arXiv admin note: substantial text overlap with arXiv:1203.609
Learning and inference in knowledge-based probabilistic model for medical diagnosis
Based on a weighted knowledge graph to represent first-order knowledge and
combining it with a probabilistic model, we propose a methodology for the
creation of a medical knowledge network (MKN) in medical diagnosis. When a set
of symptoms is activated for a specific patient, we can generate a ground
medical knowledge network composed of symptom nodes and potential disease
nodes. By Incorporating a Boltzmann machine into the potential function of a
Markov network, we investigated the joint probability distribution of the MKN.
In order to deal with numerical symptoms, a multivariate inference model is
presented that uses conditional probability. In addition, the weights for the
knowledge graph were efficiently learned from manually annotated Chinese
Electronic Medical Records (CEMRs). In our experiments, we found numerically
that the optimum choice of the quality of disease node and the expression of
symptom variable can improve the effectiveness of medical diagnosis. Our
experimental results comparing a Markov logic network and the logistic
regression algorithm on an actual CEMR database indicate that our method holds
promise and that MKN can facilitate studies of intelligent diagnosis.Comment: 32 pages, 8 figure
Network Reliability: The effect of local network structure on diffusive processes
This paper re-introduces the network reliability polynomial - introduced by
Moore and Shannon in 1956 -- for studying the effect of network structure on
the spread of diseases. We exhibit a representation of the polynomial that is
well-suited for estimation by distributed simulation. We describe a collection
of graphs derived from Erd\H{o}s-R\'enyi and scale-free-like random graphs in
which we have manipulated assortativity-by-degree and the number of triangles.
We evaluate the network reliability for all these graphs under a reliability
rule that is related to the expected size of a connected component. Through
these extensive simulations, we show that for positively or neutrally
assortative graphs, swapping edges to increase the number of triangles does not
increase the network reliability. Also, positively assortative graphs are more
reliable than neutral or disassortative graphs with the same number of edges.
Moreover, we show the combined effect of both assortativity-by-degree and the
presence of triangles on the critical point and the size of the smallest
subgraph that is reliable.Comment: 12 pages, 8 figures, 1 tabl
Classification-based Financial Markets Prediction using Deep Neural Networks
Deep neural networks (DNNs) are powerful types of artificial neural networks
(ANNs) that use several hidden layers. They have recently gained considerable
attention in the speech transcription and image recognition community
(Krizhevsky et al., 2012) for their superior predictive properties including
robustness to overfitting. However their application to algorithmic trading has
not been previously researched, partly because of their computational
complexity. This paper describes the application of DNNs to predicting
financial market movement directions. In particular we describe the
configuration and training approach and then demonstrate their application to
backtesting a simple trading strategy over 43 different Commodity and FX future
mid-prices at 5-minute intervals. All results in this paper are generated using
a C++ implementation on the Intel Xeon Phi co-processor which is 11.4x faster
than the serial version and a Python strategy backtesting environment both of
which are available as open source code written by the authors
- …