922 research outputs found
Safe Policy Synthesis in Multi-Agent POMDPs via Discrete-Time Barrier Functions
A multi-agent partially observable Markov decision process (MPOMDP) is a
modeling paradigm used for high-level planning of heterogeneous autonomous
agents subject to uncertainty and partial observation. Despite their modeling
efficiency, MPOMDPs have not received significant attention in safety-critical
settings. In this paper, we use barrier functions to design policies for
MPOMDPs that ensure safety. Notably, our method does not rely on discretization
of the belief space, or finite memory. To this end, we formulate sufficient and
necessary conditions for the safety of a given set based on discrete-time
barrier functions (DTBFs) and we demonstrate that our formulation also allows
for Boolean compositions of DTBFs for representing more complicated safe sets.
We show that the proposed method can be implemented online by a sequence of
one-step greedy algorithms as a standalone safe controller or as a
safety-filter given a nominal planning policy. We illustrate the efficiency of
the proposed methodology based on DTBFs using a high-fidelity simulation of
heterogeneous robots.Comment: 8 pages and 4 figure
Stochastic Tools for Network Intrusion Detection
With the rapid development of Internet and the sharp increase of network
crime, network security has become very important and received a lot of
attention. We model security issues as stochastic systems. This allows us to
find weaknesses in existing security systems and propose new solutions.
Exploring the vulnerabilities of existing security tools can prevent
cyber-attacks from taking advantages of the system weaknesses. We propose a
hybrid network security scheme including intrusion detection systems (IDSs) and
honeypots scattered throughout the network. This combines the advantages of two
security technologies. A honeypot is an activity-based network security system,
which could be the logical supplement of the passive detection policies used by
IDSs. This integration forces us to balance security performance versus cost by
scheduling device activities for the proposed system. By formulating the
scheduling problem as a decentralized partially observable Markov decision
process (DEC-POMDP), decisions are made in a distributed manner at each device
without requiring centralized control. The partially observable Markov decision
process (POMDP) is a useful choice for controlling stochastic systems. As a
combination of two Markov models, POMDPs combine the strength of hidden Markov
Model (HMM) (capturing dynamics that depend on unobserved states) and that of
Markov decision process (MDP) (taking the decision aspect into account).
Decision making under uncertainty is used in many parts of business and
science.We use here for security tools.We adopt a high-quality approximation
solution for finite-space POMDPs with the average cost criterion, and their
extension to DEC-POMDPs. We show how this tool could be used to design a
network security framework.Comment: Accepted by International Symposium on Sensor Networks, Systems and
Security (2017
Qualitative Analysis of POMDPs with Temporal Logic Specifications for Robotics Applications
We consider partially observable Markov decision processes (POMDPs), that are
a standard framework for robotics applications to model uncertainties present
in the real world, with temporal logic specifications. All temporal logic
specifications in linear-time temporal logic (LTL) can be expressed as parity
objectives. We study the qualitative analysis problem for POMDPs with parity
objectives that asks whether there is a controller (policy) to ensure that the
objective holds with probability 1 (almost-surely). While the qualitative
analysis of POMDPs with parity objectives is undecidable, recent results show
that when restricted to finite-memory policies the problem is EXPTIME-complete.
While the problem is intractable in theory, we present a practical approach to
solve the qualitative analysis problem. We designed several heuristics to deal
with the exponential complexity, and have used our implementation on a number
of well-known POMDP examples for robotics applications. Our results provide the
first practical approach to solve the qualitative analysis of robot motion
planning with LTL properties in the presence of uncertainty
- …