Identification of unexpected decisions in Partially Observable Monte Carlo Planning: a rule-based approach

Partially Observable Monte-Carlo Planning (POMCP) is a powerful online algorithm able to generate approximate policies for large Partially Observable Markov Decision Processes. The online nature of this method supports scalability by avoiding complete policy representation. The lack of an explicit representation however hinders interpretability. In this work, we propose a methodology based on Satisfiability Modulo Theory (SMT) for analyzing POMCP policies by inspecting their traces, namely sequences of belief-action-observation triplets generated by the algorithm. The proposed method explores local properties of policy behavior to identify unexpected decisions. We propose an iterative process of trace analysis consisting of three main steps, i) the definition of a question by means of a parametric logical formula describing (probabilistic) relationships between beliefs and actions, ii) the generation of an answer by computing the parameters of the logical formula that maximize the number of satisfied clauses (solving a MAX-SMTproblem), iii) the analysis of the generated logical formula and the related decision boundaries for identifying unexpected decisions made by POMCP with respect to the original question. We evaluate our approach on Tiger, a standard benchmark for POMDPs, and a real-world problem related to mobile robot navigation. Results show that the approach can exploit human knowledge on the domain, outperforming state-of-the-art anomaly detection methods in identifying unexpected decisions. An improvement of the Area Under Curve up to 47% has been achieved in our tests

Policy Interpretation for Partially Observable Monte-Carlo Planning: A Rule-Based Approach

Partially Observable Monte-Carlo Planning (POMCP) is a powerful online algorithm that can generate online policies for large Partially Observable Markov Decision Processes. The lack of an explicit representation of the policy, however, hinders interpretability. In this work, we present a MAX-SMT based methodology to iteratively explore local properties of the policy. Our approach generates a compact and informative representation that describes the system under investigation

Active Generation of Logical Rules for POMCP Shielding

We consider the popular Partially Observable Monte-Carlo Plan- ning (POMCP) algorithm and propose a methodology, called Active XPOMCP, for generating compact logical rules that represent prop- erties of the control policy. These rules are then used as shields to prevent POMCP from selecting unexpected actions, with useful implications on the security and trustworthiness of the algorithm. Contrary to state-of-the-art methods, Active XPOMCP does not require a previously generated set of belief-action pairs to generate the logical rule, but it actively generates this data in an information- efficient way by querying the algorithm. Active XPOMCP reduces the number of beliefs needed to generate accurate rules with re- spect to state-of-the-art methods, and it allows to produce more accurate shields when few belief-action samples are available

Explaining the influence of prior knowledge on POMCP policies

Partially Observable Monte Carlo Planning is a recently proposed online planning algorithm which makes use of Monte Carlo Tree Search to solve Partially Observable Monte Carlo Decision Processes. This solver is very successful because of its capability to scale to large uncertain environments, a very important property for current real-world planning problems. In this work we propose three main contributions related to POMCP usage and interpretability. First, we introduce a new planning problem related to mobile robot collision avoidance in paths with uncertain segment difficulties, and we show how POMCP performance in this context can take advantage of prior knowledge about segment difficulty relationships. This problem has direct real-world applications, such as, safety management in industrial environments where human-robot interaction is a crucial issue. Then, we present an experimental analysis about the relationships between prior knowledge provided to the algorithm and performance improvement, showing that in our case study prior knowledge affects two main properties, namely, the distance between the belief and the real state, and the mutual information between segment difficulty and action taken in the segment. This analysis aims to improve POMCP explainability, following the line of recently proposed eXplainable AI and, in particular, eXplainable planning. Finally, we analyze results on a synthetic case study and show how the proposed measures can improve the understanding about internal planning mechanisms

Experimental characterization of leak detection systems in HLM pool using LIFUS5/Mod3 facility

In the framework of the European Union MAXSIMA project, the safety of the steam generator (SG) adopted in the primary loop of the Heavy Liquid Metal Fast Reactor has been studied investigating the consequences and damage propagation of a SG tube rupture event and characterizing leak rates from typical cracks. Instrumentation able to promptly detect the presence of a crack in the SG tubes may be used to prevent its further propagation, which would lead to a full rupture of the tube. Application of the leak-before-break concept is relevant for improving the safety of a reactor system and decreasing the probability of a pipe break event. In this framework, a new experimental campaign (Test Series C) has been carried out in the LIFUS5/Mod3 facility, installed at ENEA Centro Ricerche Brasimone, in order to characterize and to correlate the leak rate through typical cracks occurring in the pressurized tubes with signals detected by proper transducers. Test C1.3_60 was executed injecting water at about 20 bars and 200°C into lead-bismuth eutectic alloy. The injection was performed through a laser microholed plate 60 μm in diameter. Analysis of the thermohydraulic data permitted characterization of the leakage through typical cracks that can occur in the pressurized tubes of the SG. Analysis of the data acquired by microphones and accelerometers highlighted that it is possible to correlate the signals to the leakage and the rate of release

Il costo sociale del morbillo in età pediatrica. L’epidemia a Palermo nel 1996-97

Objective To determine the direct and indirect costs associated with a measles epidemic occurring between September 1996 and August 1997 in Palermo (Italy) in paediatric-aged patients. Design A total of 2,029 cases of measles in a paediatric patient population were identified from a total of 38 paediatricians databases (24% of total). An extrapolation to the general population was then performed to estimate a total of 9,059 cases. Patient information obtained from the database such as patient age, risk factors, complications, vaccination history, as well as caretaker’s profession were included in a questionnaire compiled for each patient. Setting Inpatient and outpatient clinics in Palermo, Italy. Patients and participants Participants were paediatric-aged patients who had been diagnosed with measles. Included in the study was a group of previously vaccinated patients (6%). Main outcome measures and results The average cost of care was 464.000 Italian lire (Lit.) per case with a total cost of Lit. 4,2 billion for the entire epidemic. The direct costs comprehended 46.6% (Lit. 217.000 per case) of the total costs related to the measles epidemic and were subdivided according to in-patient care (55.4%), paediatric outpatient visits (33.5%) and drugs (9.7%). The average health-care cost associated to previously vaccinated patients (6%) was lower than for non-vaccinated patients, Lit. 110.000 vs Lit. 223.000 per case, respectively. Conclusion The demographic and economic data obtained highlights not only the social and economic impact of the epidemic, but also provides relevant information useful for cost-effectiveness analysis

Spatiotemporal dynamics of attentional orienting and reorienting revealed by fast optical imaging in occipital and parietal cortices

The mechanisms of visuospatial attention are mediated by two distinct fronto-parietal networks: a bilateral dorsal network (DAN), involved in the voluntary orientation of visuospatial attention, and a ventral network (VAN), lateralized to the right hemisphere, involved in the reorienting of attention to unexpected, but relevant, stimuli. The present study consisted of two aims: 1) characterize the spatio-temporal dynamics of attention and 2) examine the predictive interactions between and within the two attention systems along with visual areas, by using fast optical imaging combined with Granger causality. Data were collected from young healthy participants performing a discrimination task in a Posner-like paradigm. Functional analyses revealed bilateral dorsal parietal (i.e. dorsal regions included in the DAN) and visual recruitment during orienting, highlighting a recursive predictive interplay between specific dorsal parietal regions and visual cortex. Moreover, we found that both attention networks are active during reorienting, together with visual cortex, highlighting a mutual interaction among dorsal and visual areas, which, in turn, predicts subsequent ventral activity. For attentional reorienting our findings indicate that dorsal and visual areas encode disengagement of attention from the attended location and trigger reorientation to the unexpected location. Ventral network activity could instead reflect post-perceptual maintenance of the internal model to generate and keep updated task-related expectations

NUV-HD SiPMs with metal-filled trenches

In this paper we present the performance of a new SiPM that is sensitive to blue light and features narrow metal-filled trenches placed in the area around the single-photon avalanche diodes (SPADs) that allow an almost complete suppression the internal optical crosstalk. In particular, we show the benefits of this technological upgrade in terms of electro-optical SiPM performance when compared to the previous technology which had only a partial optical screening between the SPADs. The most relevant effect is the much higher bias voltage that can be applied to the new device before the noise diverges. This allows to optimize and improve both the photon detection efficiency and the single-photon time resolution. We also coupled the SiPMs to LYSO scintillators to verify the performance for possible application in Positron-Emission Tomography. Thanks to the better electro-optical features we were able to measure an improved coincidence time resolution. Furthermore, the optimal voltage operation region is substantially larger, making this SiPM more suitable for real system application where thousands of channels have to provide stable and reproducible performance

Reachability computation for hybrid systems with Ariadne

Ariadne is an in-progress open environment to design algorithms for computing with hybrid automata, that relies on a rigorous computable analysis theory to represent geometric objects, in order to achieve provable approximation bounds along the computations. In this paper we discuss the problem of reachability analysis of hybrid automata to decide safety properties. We describe in details the algorithm used in Ariadne to compute over-approximations of reachable sets. Then we show how it works on a simple example. Finally, we discuss the lower-approximation approach to the reachability problem and how to extend Ariadne to support it