Identification of unexpected decisions in Partially Observable Monte Carlo Planning: a rule-based approach

Partially Observable Monte-Carlo Planning (POMCP) is a powerful online algorithm able to generate approximate policies for large Partially Observable Markov Decision Processes. The online nature of this method supports scalability by avoiding complete policy representation. The lack of an explicit representation however hinders interpretability. In this work, we propose a methodology based on Satisfiability Modulo Theory (SMT) for analyzing POMCP policies by inspecting their traces, namely sequences of belief-action-observation triplets generated by the algorithm. The proposed method explores local properties of policy behavior to identify unexpected decisions. We propose an iterative process of trace analysis consisting of three main steps, i) the definition of a question by means of a parametric logical formula describing (probabilistic) relationships between beliefs and actions, ii) the generation of an answer by computing the parameters of the logical formula that maximize the number of satisfied clauses (solving a MAX-SMTproblem), iii) the analysis of the generated logical formula and the related decision boundaries for identifying unexpected decisions made by POMCP with respect to the original question. We evaluate our approach on Tiger, a standard benchmark for POMDPs, and a real-world problem related to mobile robot navigation. Results show that the approach can exploit human knowledge on the domain, outperforming state-of-the-art anomaly detection methods in identifying unexpected decisions. An improvement of the Area Under Curve up to 47% has been achieved in our tests

Policy Interpretation for Partially Observable Monte-Carlo Planning: A Rule-Based Approach

Partially Observable Monte-Carlo Planning (POMCP) is a powerful online algorithm that can generate online policies for large Partially Observable Markov Decision Processes. The lack of an explicit representation of the policy, however, hinders interpretability. In this work, we present a MAX-SMT based methodology to iteratively explore local properties of the policy. Our approach generates a compact and informative representation that describes the system under investigation

The hard X-ray tails in neutron star low mass X-ray binaries: BeppoSAX observations and possible theoretical explanation of the GX 17+2 case

We report results of a new spectral analysis of two BeppoSAX observations of the Z source GX 17+2. In one of the two observations the source exhibits a powerlaw-like hard (> 30 keV) X-ray tail which was described in a previous work by a hybrid Comptonization model. Recent high-energy observations with INTEGRAL of a sample of Low Mass X-Ray Binaries including both Z and atoll classes have shown that bulk (dynamical) Comptonization of soft photons can be a possible alternative mechanism for producing hard X-ray tails in such systems. We start from the INTEGRAL results and we exploit the broad-band capability of BeppoSAX to better investigate the physical processes at work. We use GX 17+2 as a representative case. Moreover, we suggest that weakening (or disappearance) of the hard X-ray tail can be explained by increasing radiation pressure originated at the surface of the neutron star (NS). As a result the high radiation pressure stops the bulk inflow and consequently this radiation feedback of the NS surface leads to quenching the bulk Comptonization.Comment: 6 pages, 3 figures, Accepted for publication in Ap

Explaining the influence of prior knowledge on POMCP policies

Partially Observable Monte Carlo Planning is a recently proposed online planning algorithm which makes use of Monte Carlo Tree Search to solve Partially Observable Monte Carlo Decision Processes. This solver is very successful because of its capability to scale to large uncertain environments, a very important property for current real-world planning problems. In this work we propose three main contributions related to POMCP usage and interpretability. First, we introduce a new planning problem related to mobile robot collision avoidance in paths with uncertain segment difficulties, and we show how POMCP performance in this context can take advantage of prior knowledge about segment difficulty relationships. This problem has direct real-world applications, such as, safety management in industrial environments where human-robot interaction is a crucial issue. Then, we present an experimental analysis about the relationships between prior knowledge provided to the algorithm and performance improvement, showing that in our case study prior knowledge affects two main properties, namely, the distance between the belief and the real state, and the mutual information between segment difficulty and action taken in the segment. This analysis aims to improve POMCP explainability, following the line of recently proposed eXplainable AI and, in particular, eXplainable planning. Finally, we analyze results on a synthetic case study and show how the proposed measures can improve the understanding about internal planning mechanisms

A new Comptonization model for low-magnetized accreting neutron stars in low mass X-ray binaries

We developed a new model for the X-ray spectral fitting \xspec package which takes into account the effects of both thermal and dynamical (i.e. bulk) Comptonization. The model consists of two components: one is the direct blackbody-like emission due to seed photons which are not subjected to effective Compton scattering, while the other one is a convolution of the Green's function of the energy operator with a blackbody-like seed photon spectrum. When combined thermal and bulk effects are considered, the analytic form of the Green's function may be obtained as a solution of the diffusion Comptonization equation. Using data from the BeppoSAX, INTEGRAL and RXTE satellites, we test our model on the spectra of a sample of six persistently low magnetic field bright neutron star Low Mass X-ray Binaries, covering three different spectral states. Particular attention is given to the transient powerlaw-like hard X-ray (> 30 keV) tails that we interpret in the framework of the bulk motion Comptonization process. We show that the values of the best-fit delta-parameter, which represents the importance of bulk with respect to thermal Comptonization, can be physically meaningful and can at least qualitatively describe the physical conditions of the environment in the innermost part of the system. Moreover, we show that in fitting the thermal Comptonization spectra to the X-ray spectra of these systems, the best-fit parameters of our model are in excellent agreement with those of COMPTT, a broadly used and well established XSPEC model.Comment: 15 pages, 8 figures, accepted for publication in Ap

Formation of Halogenated Byproducts upon Water Treatment with Peracetic Acid

Peracetic acid has quickly gained ground in water treatment over the last decade. Specifically, its disinfection efficacy toward a wide spectrum of microorganisms in wastewater is accompanied by the simplicity of its handling and use. Moreover, peracetic acid represents a promising option to achieve disinfection while reducing the concentration of typical chlorination byproducts in the final effluent. However, its chemical behavior is still amply debated. In this study, the reactivity of peracetic acid in the presence of halides, namely, chloride and bromide, was investigated in both synthetic waters and in a real contaminated water. While previous studies focused on the ability of this disinfectant to form halogenated byproducts in the presence of dissolved organic matter and halides, this work indicates that peracetic acid also contributes itself as a primary source in the formation of these potentially carcinogenic compounds. Specifically, this study suggests that 1.5 mM peracetic acid may form around 1-10 μg/L of bromoform when bromide is present. Bromoform formation reaches a maximum at near neutral pH, which is highly relevant for wastewater management

High-resolution tracking in a GEM-Emulsion detector

SHiP (Search for Hidden Particles) is a beam dump experiment proposed at the CERN SPS aiming at the observation of long lived particles very weakly coupled with ordinary matter mostly produced in the decay of charmed hadrons. The beam dump facility of SHiP is also a copious factory of neutrinos of all three kinds and therefore a dedicated neutrino detector is foreseen in the SHiP apparatus. The neutrino detector exploits the Emulsion Cloud Chamber technique with a modular structure, alternating walls of target units and planes of electronic detectors providing the time stamp to the event. GEM detectors are one of the possible choices for this task. This paper reports the results of the first exposure to a muon beam at CERN of a new hybrid chamber, obtained by coupling a GEM chamber and an emulsion detector. Thanks to the micrometric accuracy of the emulsion detector, the position resolution of the GEM chamber as a function of the particle inclination was evaluated in two configurations, with and without the magnetic fiel

EXPO-AGRI: Smart Automatic Greenhouse Control

Predicting and controlling plant behavior in con- trolled environments is a growing requirement in precision agri- culture. In this context sensor networks and artificial intelligence methods represent key aspects for optimizing the processes of data acquisition, mathematical modeling and decision making. In this paper we present a general architecture for automatic greenhouse control. In particular, we focus on a preliminary model for predicting the risk of new infections of downy mildew of basil (Peronospora belbahrii) on sweet basil. The architecture has three main elements of innovation: new kinds of sensors are used to extract information about the state of the plants, model predictors are generated from this information by non-trivial processing methods, and informative predictors are automatically selected using regularization techniques