Identification of unexpected decisions in Partially Observable Monte Carlo Planning: a rule-based approach

Partially Observable Monte-Carlo Planning (POMCP) is a powerful online algorithm able to generate approximate policies for large Partially Observable Markov Decision Processes. The online nature of this method supports scalability by avoiding complete policy representation. The lack of an explicit representation however hinders interpretability. In this work, we propose a methodology based on Satisfiability Modulo Theory (SMT) for analyzing POMCP policies by inspecting their traces, namely sequences of belief-action-observation triplets generated by the algorithm. The proposed method explores local properties of policy behavior to identify unexpected decisions. We propose an iterative process of trace analysis consisting of three main steps, i) the definition of a question by means of a parametric logical formula describing (probabilistic) relationships between beliefs and actions, ii) the generation of an answer by computing the parameters of the logical formula that maximize the number of satisfied clauses (solving a MAX-SMTproblem), iii) the analysis of the generated logical formula and the related decision boundaries for identifying unexpected decisions made by POMCP with respect to the original question. We evaluate our approach on Tiger, a standard benchmark for POMDPs, and a real-world problem related to mobile robot navigation. Results show that the approach can exploit human knowledge on the domain, outperforming state-of-the-art anomaly detection methods in identifying unexpected decisions. An improvement of the Area Under Curve up to 47% has been achieved in our tests

Policy Interpretation for Partially Observable Monte-Carlo Planning: A Rule-Based Approach

Partially Observable Monte-Carlo Planning (POMCP) is a powerful online algorithm that can generate online policies for large Partially Observable Markov Decision Processes. The lack of an explicit representation of the policy, however, hinders interpretability. In this work, we present a MAX-SMT based methodology to iteratively explore local properties of the policy. Our approach generates a compact and informative representation that describes the system under investigation

Active Generation of Logical Rules for POMCP Shielding

We consider the popular Partially Observable Monte-Carlo Plan- ning (POMCP) algorithm and propose a methodology, called Active XPOMCP, for generating compact logical rules that represent prop- erties of the control policy. These rules are then used as shields to prevent POMCP from selecting unexpected actions, with useful implications on the security and trustworthiness of the algorithm. Contrary to state-of-the-art methods, Active XPOMCP does not require a previously generated set of belief-action pairs to generate the logical rule, but it actively generates this data in an information- efficient way by querying the algorithm. Active XPOMCP reduces the number of beliefs needed to generate accurate rules with re- spect to state-of-the-art methods, and it allows to produce more accurate shields when few belief-action samples are available

Explaining the influence of prior knowledge on POMCP policies

Partially Observable Monte Carlo Planning is a recently proposed online planning algorithm which makes use of Monte Carlo Tree Search to solve Partially Observable Monte Carlo Decision Processes. This solver is very successful because of its capability to scale to large uncertain environments, a very important property for current real-world planning problems. In this work we propose three main contributions related to POMCP usage and interpretability. First, we introduce a new planning problem related to mobile robot collision avoidance in paths with uncertain segment difficulties, and we show how POMCP performance in this context can take advantage of prior knowledge about segment difficulty relationships. This problem has direct real-world applications, such as, safety management in industrial environments where human-robot interaction is a crucial issue. Then, we present an experimental analysis about the relationships between prior knowledge provided to the algorithm and performance improvement, showing that in our case study prior knowledge affects two main properties, namely, the distance between the belief and the real state, and the mutual information between segment difficulty and action taken in the segment. This analysis aims to improve POMCP explainability, following the line of recently proposed eXplainable AI and, in particular, eXplainable planning. Finally, we analyze results on a synthetic case study and show how the proposed measures can improve the understanding about internal planning mechanisms

Is accounting enforcement related to risk-taking in the banking industry?

Using a sample of banks from 36 countries, we document that accounting enforcement is negatively related to bank risk-taking. We also provide evidence that accounting enforcement enhances bank stability during the crisis. In addition, we show that banks assume less risk through more conservative lending decisions and a reduction in complexity in jurisdictions with higher accounting enforcement. Our results show that formal institutions such as accounting enforcement are associated with bank financial decisions and risk-taking behavior

Effect of juice turbidity on fermentative volatile compounds in white wines

'Chardonnay' (n = 4), 'Pinot gris' (n = 3) and 'Müller-Thurgau' juices (n = 3), each at 6 turbidity levels (15, 45, 86, 141, 215 and 350 NTU) obtained by adding increasing amounts of their own fine juice lees, were fermented using 'Montrachet Red Star' yeast. The main volatile compounds in free form which may have a sensory role were measured using GC-FID, with a DB-WAX column, after fixing onto Isolute ENV+ resin. Changes for around 40 volatile compounds and fermentation parameters are shown. Juice turbidity levels just below 100 NTU are the best compromise for obtaining adequate fruity notes and minimising languishing fermentation and off-flavours in white wine, if correct microbiology management at the winery is guaranteed, whereas slightly higher NTU levels could contribute to a slightly more complex aroma. However, variability due to juice turbidity in the range investigated is lower than variability due to yeast strain observed in a previous experiment. Thus the choice of yeast strain to direct white wine aroma must be overriding as compared to NTU levels.

Effects of forcing in three dimensional turbulent flows

We present the results of a numerical investigation of three-dimensional homogeneous and isotropic turbulence, stirred by a random forcing with a power law spectrum, $E_f(k)\sim k^{3-y}$. Numerical simulations are performed at different resolutions up to $512^3$. We show that at varying the spectrum slope $y$, small-scale turbulent fluctuations change from a {\it forcing independent} to a {\it forcing dominated} statistics. We argue that the critical value separating the two behaviours, in three dimensions, is $y_c=4$. When the statistics is forcing dominated, for $y<y_c$, we find dimensional scaling, i.e. intermittency is vanishingly small. On the other hand, for $y>y_c$, we find the same anomalous scaling measured in flows forced only at large scales. We connect these results with the issue of {\it universality} in turbulent flows.Comment: 4 pages, 4 figure

The disinfection by-products are in the air: Aerosol measurements in the urban area of Venice

Haloacetic acids (HAAs) are usually taken to be man-made, regularly determined in drinking water as disinfection by-products; yet they have been demonstrated as a probable photooxidation products from atmospheric halogenated hydrocarbons. Here we determined a total of fourteen HAAs, including for the first time iodinatedHAAs, in aerosol samples collected from April to June 2023 at Parco Bissuola (urbanized area of Venice) and Rio Novo (Venice island). The study provides a first insight and source identification about the anthropogenic or natural origin of HAAs. To fulfil this aim, we improved existing methods with a highly sensitive technique by using high pressure anion exchange chromatography coupled to a triple quadrupole mass spectrometer. Five HAAs have been found to be above the detection limits at both sites, and the major source should be attributed to a mixed contribution from atmospheric oxidation of both organic precursor, sea spray and a minor contribution of an external swimming pool located near the Parco Bissuola site. Considering the presence of iodinated HAAs and their cytotoxicity, the present study also provides a basis for the assessment of risks for humans