852 research outputs found
Randomized Shortest Paths with Net Flows and Capacity Constraints
This work extends the randomized shortest paths (RSP) model by investigating
the net flow RSP and adding capacity constraints on edge flows. The standard
RSP is a model of movement, or spread, through a network interpolating between
a random-walk and a shortest-path behavior [30, 42, 49]. The framework assumes
a unit flow injected into a source node and collected from a target node with
flows minimizing the expected transportation cost, together with a relative
entropy regularization term. In this context, the present work first develops
the net flow RSP model considering that edge flows in opposite directions
neutralize each other (as in electric networks), and proposes an algorithm for
computing the expected routing costs between all pairs of nodes. This quantity
is called the net flow RSP dissimilarity measure between nodes. Experimental
comparisons on node clustering tasks indicate that the net flow RSP
dissimilarity is competitive with other state-of-the-art dissimilarities. In
the second part of the paper, it is shown how to introduce capacity constraints
on edge flows, and a procedure is developed to solve this constrained problem
by exploiting Lagrangian duality. These two extensions should improve
significantly the scope of applications of the RSP framework
Experience-driven Networking: A Deep Reinforcement Learning based Approach
Modern communication networks have become very complicated and highly
dynamic, which makes them hard to model, predict and control. In this paper, we
develop a novel experience-driven approach that can learn to well control a
communication network from its own experience rather than an accurate
mathematical model, just as a human learns a new skill (such as driving,
swimming, etc). Specifically, we, for the first time, propose to leverage
emerging Deep Reinforcement Learning (DRL) for enabling model-free control in
communication networks; and present a novel and highly effective DRL-based
control framework, DRL-TE, for a fundamental networking problem: Traffic
Engineering (TE). The proposed framework maximizes a widely-used utility
function by jointly learning network environment and its dynamics, and making
decisions under the guidance of powerful Deep Neural Networks (DNNs). We
propose two new techniques, TE-aware exploration and actor-critic-based
prioritized experience replay, to optimize the general DRL framework
particularly for TE. To validate and evaluate the proposed framework, we
implemented it in ns-3, and tested it comprehensively with both representative
and randomly generated network topologies. Extensive packet-level simulation
results show that 1) compared to several widely-used baseline methods, DRL-TE
significantly reduces end-to-end delay and consistently improves the network
utility, while offering better or comparable throughput; 2) DRL-TE is robust to
network changes; and 3) DRL-TE consistently outperforms a state-ofthe-art DRL
method (for continuous control), Deep Deterministic Policy Gradient (DDPG),
which, however, does not offer satisfying performance.Comment: 9 pages, 12 figures, paper is accepted as a conference paper at IEEE
Infocom 201
Sparse Randomized Shortest Paths Routing with Tsallis Divergence Regularization
This work elaborates on the important problem of (1) designing optimal
randomized routing policies for reaching a target node t from a source note s
on a weighted directed graph G and (2) defining distance measures between nodes
interpolating between the least cost (based on optimal movements) and the
commute-cost (based on a random walk on G), depending on a temperature
parameter T. To this end, the randomized shortest path formalism (RSP,
[2,99,124]) is rephrased in terms of Tsallis divergence regularization, instead
of Kullback-Leibler divergence. The main consequence of this change is that the
resulting routing policy (local transition probabilities) becomes sparser when
T decreases, therefore inducing a sparse random walk on G converging to the
least-cost directed acyclic graph when T tends to 0. Experimental comparisons
on node clustering and semi-supervised classification tasks show that the
derived dissimilarity measures based on expected routing costs provide
state-of-the-art results. The sparse RSP is therefore a promising model of
movements on a graph, balancing sparse exploitation and exploration in an
optimal way
Automated construction of variable density navigable networks in a 3D indoor environment for emergency response
Widespread human-induced or natural threats on buildings and their users have made preparedness and quick response as crucial issues for saving human lives. Available information about an emergency scene, e.g. the building structure, material and trapped people helps for decision-making and organizing rescue operations. The ability to evaluate potential scenarios for human evacuation, and then identifying the paths of egress during an emergency is critical for rescue and emergency services. Good quality models supporting real, or near-real, time decision-making and allowing the implementation of automated methods are highly desirable. In this paper, we propose a new automated method for deriving a navigable network in a 3D indoor environment, including a full 3D topological model which may be used not only for standard navigation but also for finding alternative egress routes and simulating phenomena associated with disasters such as fire spread and heat transfer
- …