Learning state-variable relationships for improving POMCP performance

We address the problem of learning state-variable relationships across different episodes in Partially Observable Markov Decision Processes (POMDPs) to improve planning performance. Specifically, we focus on Partially Observable Monte Carlo Planning (POMCP) and we represent the acquired knowledge with Markov Random Fields (MRFs). We propose three different methods to compute MRF parameters while the agent acts in the environment. Our tech- niques acquire information from agent action outcomes, and from the belief of the agent, which summarizes the knowledge acquired from observations. We also propose a stopping criterion to deter- mine when the MRF is accurate enough and the learning process can be stopped. Results show that the proposed approach allows to effectively learn state-variable probabilistic constraints and to outperform standard POMCP with no computational overhead

Learning environment properties in Partially Observable Monte Carlo Planning

We tackle the problem of learning state-variable relationships in Partially Observable Markov Decision Processes to improve planning performance on mobile robots. The proposed approach extends Partially Observable Monte Carlo Planning (POMCP) and represents state-variable relationships with Markov Random Fields. A ROS-based implementation of the approach is proposed and evaluated in rocksample, a standard benchmark for probabilistic planning under uncertainty. Experiments have been performed in simulation with Gazebo. Results show that the proposed approach allows to effectively learn state- variable probabilistic constraints on ROS-based robotic platforms and to use them in subsequent episodes to outperform standard POMC

IoT Systems for Healthy and Safe Life Environments

The past two years have been sadly marked by the worldwide spread of the SARS-Cov-19 pandemic. The first line of defense against this and other pandemic threats is to respect interpersonal distances, use masks, and sanitize hands, air, and objects. Some of these countermeasures are becoming part of our daily lives, as they are now considered good practices to reduce the risk of infection and contagion. In this context, we present \emph{Safe Place}, a modular system enabled by \gls{iot} that is designed to improve the safety and healthiness of living environments. %\textcolor{blue}{ This system combines several sensors and actuators produced by different vendors with self-regulating procedures and \gls{ai} algorithms to limit the spread of viruses and other pathogens, and increase the quality and comfort offered to people while minimizing the energy consumption.%} We discuss the main objectives of the system and its implementation, showing preliminary results that assess its potentials in enhancing the conditions of living and working spaces

Safe and Efficient Reinforcement Learning for Environmental Monitoring

This paper discusses the challenges of applying reinforcement techniques to real-world environmental monitoring problems and proposes innovative solutions to overcome them. In particular, we focus on safety, a fundamental problem in RL that arises when it is applied to domains involving humans or hazardous uncertain situations. We propose to use deep neural networks, formal verification, and online refinement of domain knowledge to improve the transparency and efficiency of the learning process, as well as the quality of the final policies. We present two case studies, specifically (i) autonomous water monitoring and (ii) smart control of air quality indoors. In particular, we discuss the challenges and solutions to these problems, addressing crucial issues such as anomaly detection and prevention, real-time control, and online learning. We believe that the proposed techniques can be used to overcome some limitations of RL, providing safe and efficient solutions to complex and urgent problems

Pharmacokinetics of β-Lactam Antibiotics:Clues from the Past to Help Discover Long-Acting Oral Drugs in the Future

β-Lactams represent perhaps the most important class of antibiotics yet discovered. However, despite many years of active research, none of the currently approved drugs in this class combine oral activity with long duration of action. Recent developments suggest that new β-lactam antibiotics with such a profile would have utility in the treatment of tuberculosis. Consequently, the historical β-lactam pharmacokinetic data have been compiled and analyzed to identify possible directions and drug discovery strategies aimed toward new β-lactam antibiotics with this profile

Learning State-Variable Relationships in POMCP: A Framework for Mobile Robots

We address the problem of learning relationships on state variables in Partially Observable Markov Decision Processes (POMDPs) to improve planning performance. Specifically, we focus on Partially Observable Monte Carlo Planning (POMCP) and represent the acquired knowledge with a Markov Random Field (MRF). We propose, in particular, a method for learning these relationships on a robot as POMCP is used to plan future actions. Then, we present an algorithm that deals with cases in which the MRF is used on episodes having unlikely states with respect to the equality relationships represented by the MRF. Our approach acquires information from the agent’s action outcomes to adapt online the MRF if a mismatch is detected between the MRF and the true state. We test this technique on two domains, rocksample, a standard rover exploration task, and a problem of velocity regulation in industrial mobile robotic platforms, showing that the MRF adaptation algorithm improves the planning performance with respect to the standard approach, which does not adapt the MRF online. Finally, a ROS-based architecture is proposed, which allows running the MRF learning, the MRF adaptation, and MRF usage in POMCP on real robotic platforms. In this case, we successfully tested the architecture on a Gazebo simulator of rocksample. A video of the experiments is available in the Supplementary Material, and the code of the ROS-based architecture is available online

Identification of a proteasome-targeting arylsulfonamide with potential for the treatment of Chagas' disease

Phenotypic screening identified an arylsulfonamide compound with activity against Trypanosoma cruzi, the causative agent of Chagas’ disease. Comprehensive mode of action studies revealed that this compound primarily targets the T. cruzi proteasome, binding at the interface between β4 and β5 subunits that catalyze chymotrypsin-like activity. A mutation in the β5 subunit of the proteasome was associated with resistance to compound 1, while overexpression of this mutated subunit also reduced susceptibility to compound 1. Further genetically engineered and in vitro-selected clones resistant to proteasome inhibitors known to bind at the β4/β5 interface were cross-resistant to compound 1. Ubiquitinated proteins were additionally found to accumulate in compound 1-treated epimastigotes. Finally, thermal proteome profiling identified malic enzyme as a secondary target of compound 1, although malic enzyme inhibition was not found to drive potency. These studies identify a novel pharmacophore capable of inhibiting the T. cruzi proteasome that may be exploitable for anti-chagasic drug discovery