55,984 research outputs found
DeepPR: Progressive Recovery for Interdependent VNFs with Deep Reinforcement Learning
The increasing reliance upon cloud services entails more flexible networks
that are realized by virtualized network equipment and functions. When such
advanced network systems face a massive failure by natural disasters or
attacks, the recovery of the entire system may be conducted in a progressive
way due to limited repair resources. The prioritization of network equipment in
the recovery phase influences the interim computation and communication
capability of systems, since the systems are operated under partial
functionality. Hence, finding the best recovery order is a critical problem,
which is further complicated by virtualization due to dependency among network
nodes and layers. This paper deals with a progressive recovery problem under
limited resources in networks with VNFs, where some dependent network layers
exist. We prove the NP-hardness of the progressive recovery problem and
approach the optimum solution by introducing DeepPR, a progressive recovery
technique based on Deep Reinforcement Learning (Deep RL). Our simulation
results indicate that DeepPR can achieve the near-optimal solutions in certain
networks and is more robust to adversarial failures, compared to a baseline
heuristic algorithm.Comment: Technical Report, 12 page
The edge-disjoint path problem on random graphs by message-passing
We present a message-passing algorithm to solve the edge disjoint path
problem (EDP) on graphs incorporating under a unique framework both traffic
optimization and path length minimization. The min-sum equations for this
problem present an exponential computational cost in the number of paths. To
overcome this obstacle we propose an efficient implementation by mapping the
equations onto a weighted combinatorial matching problem over an auxiliary
graph. We perform extensive numerical simulations on random graphs of various
types to test the performance both in terms of path length minimization and
maximization of the number of accommodated paths. In addition, we test the
performance on benchmark instances on various graphs by comparison with
state-of-the-art algorithms and results found in the literature. Our
message-passing algorithm always outperforms the others in terms of the number
of accommodated paths when considering non trivial instances (otherwise it
gives the same trivial results). Remarkably, the largest improvement in
performance with respect to the other methods employed is found in the case of
benchmarks with meshes, where the validity hypothesis behind message-passing is
expected to worsen. In these cases, even though the exact message-passing
equations do not converge, by introducing a reinforcement parameter to force
convergence towards a sub optimal solution, we were able to always outperform
the other algorithms with a peak of 27% performance improvement in terms of
accommodated paths. On random graphs, we numerically observe two separated
regimes: one in which all paths can be accommodated and one in which this is
not possible. We also investigate the behaviour of both the number of paths to
be accommodated and their minimum total length.Comment: 14 pages, 8 figure
Reinforcement Learning for Racecar Control
This thesis investigates the use of reinforcement learning to learn to drive a racecar in the simulated environment of the Robot Automobile Racing Simulator. Real-life race driving is known to be difficult for humans, and expert human drivers use complex sequences of actions. There are a large number of variables, some of which change stochastically and all of which may affect the outcome. This makes driving a promising domain for testing and developing Machine Learning techniques that have the potential to be robust enough to work in the real world. Therefore the principles of the algorithms from this work may be applicable to a range of problems.
The investigation starts by finding a suitable data structure to represent the information learnt. This is tested using supervised learning. Reinforcement learning is added and roughly tuned, and the supervised learning is then removed. A simple tabular representation is found satisfactory, and this avoids difficulties with more complex methods and allows the investigation to concentrate on the essentials of learning. Various reward sources are tested and a combination of three are found to produce the best performance. Exploration of the problem space is investigated. Results show exploration is essential but controlling how much is done is also important. It turns out the learning episodes need to be very long and because of this the task needs to be treated as continuous by using discounting to limit the size of the variables stored. Eligibility traces are used with success to make the learning more efficient. The tabular representation is made more compact by hashing and more accurate by using smaller buckets. This slows the learning but produces better driving. The improvement given by a rough form of generalisation indicates the replacement of the tabular method by a function approximator is warranted. These results show reinforcement learning can work within the Robot Automobile Racing Simulator, and lay the foundations for building a more efficient and competitive agent
The Algorithmic Complexity of Bondage and Reinforcement Problems in bipartite graphs
Let be a graph. A subset is a dominating set if
every vertex not in is adjacent to a vertex in . The domination number
of , denoted by , is the smallest cardinality of a dominating set
of . The bondage number of a nonempty graph is the smallest number of
edges whose removal from results in a graph with domination number larger
than . The reinforcement number of is the smallest number of
edges whose addition to results in a graph with smaller domination number
than . In 2012, Hu and Xu proved that the decision problems for the
bondage, the total bondage, the reinforcement and the total reinforcement
numbers are all NP-hard in general graphs. In this paper, we improve these
results to bipartite graphs.Comment: 13 pages, 4 figures. arXiv admin note: substantial text overlap with
arXiv:1109.1657; and text overlap with arXiv:1204.4010 by other author
- …