On Modeling Signal Transduction Networks

Signal transduction networks are very complex processes employed by the living cell to suitably react to environmental stimuli. Qualitative and quantitative computational models play an increasingly important role in the representation of these networks and in the search of new insights about these phenomena. In this work we analyze some graph-based models used to discover qualitative properties of such networks. In turn, we show that MP systems can naturally extend these graph-based models by adding some qualitative elements. The case study of integrins activation during the lymphocyte recruitment, a crucial phenomenon in inflammatory processes, is described, and a first MP graph for this network is designed. Finally, we discuss some open problems related to the qualitative modeling of signaling networks

Generation and interpretation of parsimonious predictive models for load forecasting in smart heating networks

Forecasting future heat load in smart district heating networks is a key problem for utility companies that need such predictions for optimizing their operational activities. From the statistical learning viewpoint, this problem is also very interesting because it requires to integrate multiple time series about weather and social factors into a dynamical model, and to generate models able to explain the relationships between weather/social factors and heat load. Typical questions in this context are: “Which variables are more informative for the prediction?” and “Do variables have different influence in different contexts (e.g., time instant or situations)?” We propose a methodology for generating simple and interpretable models for heat load forecasting, then we apply this methodology to a real dataset, and, finally, provide new insight about this application domain. The methodology merges multi-equation multivariate linear regression and forward variable selection. We generate a (sparse) equation for each pair day-of-the-week/hour-of-the-day (for instance, one equation concerns predictions of Monday at 0.00, another predictions of Monday at 1.00, and so on). These equations are simple to explain because they locally approximate the prediction problem in specific times of day/week. Variable selection is a key contribution of this work. It provides a reduction of the prediction error of 2.4% and a decrease of the number of parameters of 49.8% compared to state-of-the-art models. Interestingly, different variables are selected in different equations (i.e., times of the day/week), showing that weather and social factors, and autoregressive variables with different delays, differently influence heat predictions in different times of the day/week

Rule-based shielding for Partially Observable Monte-Carlo Planning

Partially Observable Monte-Carlo Planning (POMCP) is a powerful online algorithm able to generate approximate policies for large Partially Observable Markov Decision Processes. The online nature of this method supports scalability by avoiding complete policy representation. The lack of an explicit representation however hinders policy interpretability and makes policy verification very complex. In this work, we propose two contributions. The first is a method for identifying unexpected actions selected by POMCP with respect to expert prior knowledge of the task. The second is a shielding approach that prevents POMCP from selecting unexpected actions. The first method is based on Satisfiability Modulo Theory (SMT). It inspects traces (i.e., sequences of belief-action-observation triplets) generated by POMCP to compute the parameters of logical formulas about policy properties defined by the expert. The second contribution is a module that uses online the logical formulas to identify anomalous actions selected by POMCP and substitutes those actions with actions that satisfy the logical formulas fulfilling expert knowledge. We evaluate our approach on Tiger, a standard benchmark for POMDPs, and a real-world problem related to velocity regulation in mobile robot navigation. Results show that the shielded POMCP outperforms the standard POMCP in a case study in which a wrong parameter of POMCP makes it select wrong actions from time to time. Moreover, we show that the approach keeps good performance also if the parameters of the logical formula are optimized using trajectories containing some wrong actions

Rule-Based Policy Interpretation and Shielding for Partially Observable Monte Carlo Planning

Partially Observable Monte Carlo Planning (POMCP) is a powerful online algorithm that can generate approximate policies for large Partially Observable Markov Decision Processes. The online nature of this method supports scalability by avoiding complete policy representation. However, the lack of an explicit representation of the policy hinders interpretability. In this thesis, we propose a methodology based on Maximum Satisfiability Modulo Theory (MAX-SMT) for analyzing POMCP policies by inspecting their traces, namely, sequences of belief-action pairs generated by the algorithm. The proposed method explores local properties of the policy to build a compact and informative summary of the policy behaviour. This representation exploits a high-level description encoded using logical formulas that domain experts can provide. The final formula can be used to identify unexpected decisions, namely, decisions that violate the expert indications. We show that this identification process can be used offline (to improve the explainability of the policy and to identify anomalous behaviours) or online (to shield the decisions of the POMCP algorithm). We also present an active methodology that can effectively query a POMCP policy to build more reliable descriptions quickly. We extensively evaluate our methodologies on two standard benchmarks for POMDPs, namely, emph{tiger} and emph{rocksample}, and on a problem related to velocity regulation in mobile robot navigation. Results show that our approach achieves good performance due to its capability to exploit experts' knowledge of the domains. Specifically, our approach can be used both to identify anomalous behaviours in faulty POMCPs and to improve the performance of the system by using the shielding mechanism. In the first case, we test the methodology against a state-of-the-art anomaly detection algorithm, while in the second, we compared the performance of shielded and unshielded POMCPs. We implemented our methodology in CC, and the code is open-source and available at href{https://github.com/GiuMaz/XPOMCP}{https://github.com/GiuMaz/XPOMCP}

Identification of Unexpected Decisions in Partially Observable Monte-Carlo Planning: a Rule-Based Approach

Partially Observable Monte-Carlo Planning (POMCP) is a powerful online algorithm able to generate approximate policies for large Partially Observable Markov Decision Processes. The online nature of this method supports scalability by avoiding complete policy representation. The lack of an explicit representation however hinders interpretability. In this work, we propose a methodology based on Satisfiability Modulo Theory (SMT) for analyzing POMCP policies by inspecting their traces, namely sequences of belief-action-observation triplets generated by the algorithm. The proposed method explores local properties of policy behavior to identify unexpected decisions. We propose an iterative process of trace analysis consisting of three main steps, i) the definition of a question by means of a parametric logical formula describing (probabilistic) relationships between beliefs and actions, ii) the generation of an answer by computing the parameters of the logical formula that maximize the number of satisfied clauses (solving a MAX-SMT problem), iii) the analysis of the generated logical formula and the related decision boundaries for identifying unexpected decisions made by POMCP with respect to the original question. We evaluate our approach on Tiger, a standard benchmark for POMDPs, and a real-world problem related to mobile robot navigation. Results show that the approach can exploit human knowledge on the domain, outperforming state-of-the-art anomaly detection methods in identifying unexpected decisions. An improvement of the Area Under Curve up to 47\% has been achieved in our tests.Comment: AAMAS 2021, 3-7 May 2021, London-UK (Virtual

Rule-based Shield Synthesis for Partially Observable Monte Carlo Planning

Partially Observable Monte-Carlo Planning (POMCP) is a powerful online algorithm able to generate approximate policies for large Partially Observable Markov Decision Processes. The online nature of this method supports scalability by avoiding complete policy representation. The lack of an explicit representation however hinders policy interpretability and makes policy verification very complex. In this work, we propose two contributions. The first is a method for identifying unexpected actions selected by POMCP with respect to expert prior knowledge of the task. The second is a shielding approach that prevents POMCP from selecting unexpected actions. The first method is based on Maximum Satisfiability Modulo Theory (MAX-SMT). It inspects traces (i.e., sequences of belief-action-observation triplets) generated by POMCP to compute the parameters of logical formulas about policy properties defined by the expert. The second contribution is a module that uses online the logical formulas to identify anomalous actions selected by POMCP and substitutes those actions with actions that satisfy the logical formulas fulfilling expert knowledge. We evaluate our approach in two domains. Results show that the shielded POMCP outperforms the standard POMCP in a case study in which a wrong parameter of POMCP makes it select wrong actions from time to time. © 2021 Copyright for this paper by its authors

Risk-aware shielding of Partially Observable Monte Carlo Planning policies

Partially Observable Monte Carlo Planning (POMCP) is a powerful online algorithm that can generate approximate policies for large Partially Observable Markov Decision Processes. The online nature of this method supports scalability by avoiding complete policy representation. However, the lack of an explicit policy representation hinders interpretability and a proper evaluation of the risks an agent may incur. In this work, we propose a methodology based on Maximum Satisfiability Modulo Theory (MAX-SMT) for analyzing POMCP policies by inspecting their traces, namely, sequences of belief- action pairs generated by the algorithm. The proposed method explores local properties of the policy to build a compact and informative summary of the policy behaviour. Moreover, we introduce a rich and formal language that a domain expert can use to describe the expected behaviour of a policy. In more detail, we present a formulation that directly computes the risk involved in taking actions by considering the high- level elements specified by the expert. The final formula can identify risky decisions taken by POMCP that violate the expert indications. We show that this identification process can be used offline (to improve the policy’s explainability and identify anomalous behaviours) or online (to shield the risky decisions of the POMCP algorithm). We present an extended evaluation of our approach on four domains: the well-known tiger and rocksample benchmarks, a problem of velocity regulation in mobile robots, and a problem of battery management in mobile robots. We test the methodology against a state-of- the-art anomaly detection algorithm to show that our approach can be used to identify anomalous behaviours in faulty POMCP. We also show, comparing the performance of shielded and unshielded POMCP, that the shielding mechanism can improve the system’s performance. We provide an open-source implementation of the proposed methodologies at https://github.com/GiuMaz/XPOMCP

Partially Observable Monte Carlo Planning with state variable constraints for mobile robot navigation

Autonomous mobile robots employed in industrial applications often operate in complex and uncertain environments. In this paper we propose an approach based on an extension of Partially Observable Monte Carlo Planning (POMCP) for robot velocity regulation in industrial-like environments characterized by uncertain motion difficulties. The velocity selected by POMCP is used by a standard engine controller which deals with path planning. This two-layer approach allows POMCP to exploit prior knowledge on the relationships between task similarities to improve performance in terms of time spent to traverse a path with obstacles. We also propose three measures to support human-understanding of the strategy used by POMCP to improve the performance. The overall architecture is tested on a Turtlebot3 in two environments, a rectangular path and a realistic production line in a research lab. Tests performed on a C++ simulator confirm the capability of the proposed approach to profitably use prior knowledge, achieving a performance improvement from 0.7% to 3.1% depending on the complexity of the path. Experiments on a Unity simulator show that the proposed two-layer approach outperforms also single-layer approaches based only on the engine controller (i.e., without the POMCP layer). In this case the performance improvement is up to 37% comparing to a state-of-the-art deep reinforcement learning engine controller, and up to 51% comparing to the standard ROS engine controller. Finally, experiments in a real-world testing arena confirm the possibility to run the approach on real robots

Algorithms and Software for Biological MP Modeling by Statistical and Optimization Techniques

I sistemi biologici sono gruppi di entit\ue0 biologiche (es. molecole ed organismi), che interagiscono producendo specifiche dinamiche. Questi sistemi sono solitamente caratterizzati da una elevata complessit\ue0 perch\ue8 coinvolgono un elevato numero di componenti con molte interconnessioni. La comprensione dei meccanismi che governano i sistemi biologici e la previsione dei loro comportamenti in condizioni normali e patologiche \ue8 una sfida cruciale della biologia dei sistemi (in inglese detta systems biology), un'area di ricerca al confine tra biologia, medicina, matematica ed informatica. In questa tesi i P sistemi metabolici, detti brevemente sistemi MP, sono stati utilizzati come modello discreto per l'analisi di dinamiche biologiche. Essi sono una classe deterministica dei P sistemi classici, che utilizzano regole di riscrittura per rappresentare le reazioni chimiche e "funzioni di regolazioni di flusso" per regolare la reattivit\ue0 di ciascuna reazione rispetto alla quantita' di sostanze presenti istantaneamente nel sistema. Dopo un excursus sulla letteratura relativa ad alcuni modelli convenzionali (come le equazioni differenziali ed i modelli stocastici proposti da Gillespie) e non-convenzionali (come i P sistemi ed i P sistemi metabolici), saranno presentati i risultati della mia ricerca. Essi riguardano tre argomenti principali: i) l'equivalenza tra sistemi MP e reti di Petri ibride funzionali, ii) le prospettive statistiche e di ottimizzazione nella generazione di sistemi MP a partire da dati sperimentali, iii) lo sviluppo di un laboratorio virtuale chiamato MetaPlab, un software Java basato sui sistemi MP. L'equivalenza tra i sistemi MP e le reti di Petri ibride funzionali \ue8 stata dimostrata per mezzo di due teoremi ed alcuni esperimenti al computer per il caso di studio del meccanismo regolativo del gene operone lac nella pathway glicolitica. Il secondo argomento di ricerca concerne nuovi approcci per la sintesi delle funzioni di regolazione di flusso. La regressione stepwise e le reti neurali sono state impiegate come approssimatori di funzioni, mentre algoritmi di ottimizzazione classici ed evolutivi (es. backpropagation, algoritmi genetici, particle swarm optimization ed algoritmi memetici) sono stati impiegati per l'addestramento dei modelli. Una completo workflow per l'analisi dei dati sperimentali \ue8 stato presentato. Esso gestisce ed indirizza l'intero processo di sintesi delle funzioni di regolazione, dalla preparazione dei dati alla selezione delle variabili, fino alla generazione dei modelli ed alla loro validazione. Le metodologie proposte sono state testate con successo tramite esperimenti al computer sui casi di studio dell'oscillatore mitotico negli embrioni anfibi e del non photochemical quenching (NPQ). L'ultimo tema di ricerca \ue8 infine piu' applicativo e riguarda la progettazione e lo sviluppo di una architettura Java basata su plugin e di una serie di plugin che consentono di automatizzare varie fasi del processo di modellazione con sistemi MP, come la simulazione di dinamiche, la determinazione dei flussi e la generazione delle funzioni di regolazione.Biological systems are groups of biological entities, (e.g., molecules and organisms), that interact together producing specific dynamics. These systems are usually characterized by a high complexity, since they involve a large number of components having many interconnections. Understanding biological system mechanisms, and predicting their behaviors in normal and pathological conditions is a crucial challenge in systems biology, which is a central research area on the border among biology, medicine, mathematics and computer science. In this thesis metabolic P systems, also called MP systems, have been employed as discrete modeling framework for the analysis of biological system dynamics. They are a deterministic class of P systems employing rewriting rules to represent chemical reactions and "flux regulation functions" to tune reactions reactivity according to the amount of substances present in the system. After an excursus on the literature about some conventional (i.e., differential equations, Gillespie's models) and unconventional (i.e., P systems and metabolic P systems) modeling frameworks, the results of my research are presented. They concern three research topics: i) equivalences between MP systems and hybrid functional Petri nets, ii) statistical and optimization perspectives in the generation of MP models from experimental data, iii) development of the virtual laboratory MetaPlab, a Java software based on MP systems. The equivalence between MP systems and hybrid functional Petri nets is proved by two theorems and some in silico experiments for the case study of the lac operon gene regulatory mechanism and glycolytic pathway. The second topic concerns new approaches to the synthesis of flux regulation functions. Stepwise linear regression and neural networks are employed as function approximators, and classical/evolutionary optimization algorithms (e.g., backpropagation, genetic algorithms, particle swarm optimization, memetic algorithms) as learning techniques. A complete pipeline for data analysis is also presented, which addresses the entire process of flux regulation function synthesis, from data preparation to feature selection, model generation and statistical validation. The proposed methodologies have been successfully tested by means of in silico experiments on the mitotic oscillator in early amphibian embryos and the non photochemical quenching (NPQ). The last research topic is more applicative, and pertains the design and development of a Java plugin architecture and several plugins which enable to automatize many tasks related to MP modeling, such as, dynamics computation, flux discovery, and regulation function synthesis