971 research outputs found
Safety-Aware Apprenticeship Learning
Apprenticeship learning (AL) is a kind of Learning from Demonstration
techniques where the reward function of a Markov Decision Process (MDP) is
unknown to the learning agent and the agent has to derive a good policy by
observing an expert's demonstrations. In this paper, we study the problem of
how to make AL algorithms inherently safe while still meeting its learning
objective. We consider a setting where the unknown reward function is assumed
to be a linear combination of a set of state features, and the safety property
is specified in Probabilistic Computation Tree Logic (PCTL). By embedding
probabilistic model checking inside AL, we propose a novel
counterexample-guided approach that can ensure safety while retaining
performance of the learnt policy. We demonstrate the effectiveness of our
approach on several challenging AL scenarios where safety is essential.Comment: Accepted by International Conference on Computer Aided Verification
(CAV) 201
Evaluating the reliability of NAND multiplexing with PRISM
Probabilistic-model checking is a formal verification technique for analyzing the reliability and performance of systems exhibiting stochastic behavior. In this paper, we demonstrate the applicability of this approach and, in particular, the probabilistic-model-checking tool PRISM to the evaluation of reliability and redundancy of defect-tolerant systems in the field of computer-aided design. We illustrate the technique with an example due to von Neumann, namely NAND multiplexing. We show how, having constructed a model of a defect-tolerant system incorporating probabilistic assumptions about its defects, it is straightforward to compute a range of reliability measures and investigate how they are affected by slight variations in the behavior of the system. This allows a designer to evaluate, for example, the tradeoff between redundancy and reliability in the design. We also highlight errors in analytically computed reliability bounds, recently published for the same case study
Explicit Model Checking of Very Large MDP using Partitioning and Secondary Storage
The applicability of model checking is hindered by the state space explosion
problem in combination with limited amounts of main memory. To extend its
reach, the large available capacities of secondary storage such as hard disks
can be exploited. Due to the specific performance characteristics of secondary
storage technologies, specialised algorithms are required. In this paper, we
present a technique to use secondary storage for probabilistic model checking
of Markov decision processes. It combines state space exploration based on
partitioning with a block-iterative variant of value iteration over the same
partitions for the analysis of probabilistic reachability and expected-reward
properties. A sparse matrix-like representation is used to store partitions on
secondary storage in a compact format. All file accesses are sequential, and
compression can be used without affecting runtime. The technique has been
implemented within the Modest Toolset. We evaluate its performance on several
benchmark models of up to 3.5 billion states. In the analysis of time-bounded
properties on real-time models, our method neutralises the state space
explosion induced by the time bound in its entirety.Comment: The final publication is available at Springer via
http://dx.doi.org/10.1007/978-3-319-24953-7_1
Quantitative Analysis of DoS Attacks and Client Puzzles in IoT Systems
Denial of Service (DoS) attacks constitute a major security threat to today's
Internet. This challenge is especially pertinent to the Internet of Things
(IoT) as devices have less computing power, memory and security mechanisms to
mitigate DoS attacks. This paper presents a model that mimics the unique
characteristics of a network of IoT devices, including components of the system
implementing `Crypto Puzzles' - a DoS mitigation technique. We created an
imitation of a DoS attack on the system, and conducted a quantitative analysis
to simulate the impact such an attack may potentially exert upon the system,
assessing the trade off between security and throughput in the IoT system. We
model this through stochastic model checking in PRISM and provide evidence that
supports this as a valuable method to compare the efficiency of different
implementations of IoT systems, exemplified by a case study
Value Iteration for Long-run Average Reward in Markov Decision Processes
Markov decision processes (MDPs) are standard models for probabilistic
systems with non-deterministic behaviours. Long-run average rewards provide a
mathematically elegant formalism for expressing long term performance. Value
iteration (VI) is one of the simplest and most efficient algorithmic approaches
to MDPs with other properties, such as reachability objectives. Unfortunately,
a naive extension of VI does not work for MDPs with long-run average rewards,
as there is no known stopping criterion. In this work our contributions are
threefold. (1) We refute a conjecture related to stopping criteria for MDPs
with long-run average rewards. (2) We present two practical algorithms for MDPs
with long-run average rewards based on VI. First, we show that a combination of
applying VI locally for each maximal end-component (MEC) and VI for
reachability objectives can provide approximation guarantees. Second, extending
the above approach with a simulation-guided on-demand variant of VI, we present
an anytime algorithm that is able to deal with very large models. (3) Finally,
we present experimental results showing that our methods significantly
outperform the standard approaches on several benchmarks
Giant cardiac tumours in the newborn: an unusual image
Primary heart tumours in the paediatric population are very rare and they range from 0.01% to 0.04%. Most are benign lesions of which about half are rhabdomyomas. Rhabdomyoma tumour diagnosis is associated with a 75–80% risk of tuberous sclerosis complex (TSC). TSC are characterised with numerous changes of hamartoma-type located in the brain, kidneys, skin and other organs including the heart. More than two-thirds of newborns with TSC present rhabdomyomas in the heart. These changes may be asymptomatic, but in some cases they may cause heart failure, arrhythmias and death. We present a case report of an infant with giant rhabdomyoma tumours in the course of TSC
Fermi surface of an important nano-sized metastable phase: AlLi
Nanoscale particles embedded in a metallic matrix are of considerable
interest as a route towards identifying and tailoring material properties. We
present a detailed investigation of the electronic structure, and in particular
the Fermi surface, of a nanoscale phase ( AlLi) that has so far been
inaccessible with conventional techniques, despite playing a key role in
determining the favorable material properties of the alloy (Al\nobreakdash-9
at. %\nobreakdash-Li). The ordered precipitates only form within the
stabilizing Al matrix and do not exist in the bulk; here, we take advantage of
the strong positron affinity of Li to directly probe the Fermi surface of
AlLi. Through comparison with band structure calculations, we demonstrate
that the positron uniquely probes these precipitates, and present a 'tuned'
Fermi surface for this elusive phase
Automated recognition of sleep arousal using multimodal and personalized deep ensembles of neural networks
Background and Aim: Monitoring physiological signals during sleep can have substantial impact on detecting temporary intrusion of wakefulness, referred to as sleep arousals. To overcome the problems associated with the cubersome visual inspection of these events by experts, sleep arousal recognition algorithms have been proposed. Method: As part of the Physionet/Computing in Cardiology Challenge 2018, this study proposes a deep ensemble neural network architecture for automatic arousal recognition from multi-modal sensor signals. Separate branches of the neural network extract features from electro-encephalography, electrooculography, electromyogram, breathing patterns and oxygen saturation level; and a final fully-connected neural network combines features computed from the signal sources to estimate the probability of arousal in each region of interest. We investigate the use of shared-parameter Siamese architectures for effective feature calibration. Namely, at each forward and backward pass through the network we concatenate to the input a user-specific template signal that is processed by an identical copy of the network. Result: The proposed architecture obtains an AUPR score of 0.40 on the test set of the official phase of Physionet/CbiC Challenge 2018. A score of 0.45 is obtained by means of 10 -fold cross-validation on the training set
- …