245 research outputs found
Online Safety Property Collection and Refinement for Safe Deep Reinforcement Learning in Mapless Navigation
Safety is essential for deploying Deep Reinforcement Learning (DRL)
algorithms in real-world scenarios. Recently, verification approaches have been
proposed to allow quantifying the number of violations of a DRL policy over
input-output relationships, called properties. However, such properties are
hard-coded and require task-level knowledge, making their application
intractable in challenging safety-critical tasks. To this end, we introduce the
Collection and Refinement of Online Properties (CROP) framework to design
properties at training time. CROP employs a cost signal to identify unsafe
interactions and use them to shape safety properties. Hence, we propose a
refinement strategy to combine properties that model similar unsafe
interactions. Our evaluation compares the benefits of computing the number of
violations using standard hard-coded properties and the ones generated with
CROP. We evaluate our approach in several robotic mapless navigation tasks and
demonstrate that the violation metric computed with CROP allows higher returns
and lower violations over previous Safe DRL approaches.Comment: Accepted at the 2023 IEEE International Conference on Robotics and
Automation (ICRA). Marzari and Marchesini contributed equall
The #DNN-Verification Problem: Counting Unsafe Inputs for Deep Neural Networks
Deep Neural Networks are increasingly adopted in critical tasks that require
a high level of safety, e.g., autonomous driving. While state-of-the-art
verifiers can be employed to check whether a DNN is unsafe w.r.t. some given
property (i.e., whether there is at least one unsafe input configuration),
their yes/no output is not informative enough for other purposes, such as
shielding, model selection, or training improvements. In this paper, we
introduce the #DNN-Verification problem, which involves counting the number of
input configurations of a DNN that result in a violation of a particular safety
property. We analyze the complexity of this problem and propose a novel
approach that returns the exact count of violations. Due to the #P-completeness
of the problem, we also propose a randomized, approximate method that provides
a provable probabilistic bound of the correct count while significantly
reducing computational requirements. We present experimental results on a set
of safety-critical benchmarks that demonstrate the effectiveness of our
approximate method and evaluate the tightness of the bound.Comment: Accepted in the International Joint Conference on Artificial
Intelligence (IJCAI), 2023. [Marzari and Corsi contributed equally
Enumerating Safe Regions in Deep Neural Networks with Provable Probabilistic Guarantees
Identifying safe areas is a key point to guarantee trust for systems that are
based on Deep Neural Networks (DNNs). To this end, we introduce the
AllDNN-Verification problem: given a safety property and a DNN, enumerate the
set of all the regions of the property input domain which are safe, i.e., where
the property does hold. Due to the #P-hardness of the problem, we propose an
efficient approximation method called epsilon-ProVe. Our approach exploits a
controllable underestimation of the output reachable sets obtained via
statistical prediction of tolerance limits, and can provide a tight (with
provable probabilistic guarantees) lower estimate of the safe areas. Our
empirical evaluation on different standard benchmarks shows the scalability and
effectiveness of our method, offering valuable insights for this new type of
verification of DNNs.Comment: Accepted at the 38th Annual AAAI Conference on Artificial
Intelligence 202
Towards Hierarchical Task Decomposition using Deep Reinforcement Learning for Pick and Place Subtasks
Deep Reinforcement Learning (DRL) is emerging as a promising approach to
generate adaptive behaviors for robotic platforms. However, a major drawback of
using DRL is the data-hungry training regime that requires millions of trial
and error attempts, which is impractical when running experiments on robotic
systems. Learning from Demonstrations (LfD) has been introduced to solve this
issue by cloning the behavior of expert demonstrations. However, LfD requires a
large number of demonstrations that are difficult to be acquired since
dedicated complex setups are required. To overcome these limitations, we
propose a multi-subtask reinforcement learning methodology where complex pick
and place tasks can be decomposed into low-level subtasks. These subtasks are
parametrized as expert networks and learned via DRL methods. Trained subtasks
are then combined by a high-level choreographer to accomplish the intended pick
and place task considering different initial configurations. As a testbed, we
use a pick and place robotic simulator to demonstrate our methodology and show
that our method outperforms a benchmark methodology based on LfD in terms of
sample-efficiency. We transfer the learned policy to the real robotic system
and demonstrate robust grasping using various geometric-shaped objects.Comment: This work has been accepted to the IEEE International Conference on
Advanced Robotics (ICAR) 202
Constrained Reinforcement Learning and Formal Verification for Safe Colonoscopy Navigation
The field of robotic Flexible Endoscopes (FEs) has progressed significantly,
offering a promising solution to reduce patient discomfort. However, the
limited autonomy of most robotic FEs results in non-intuitive and challenging
manoeuvres, constraining their application in clinical settings. While previous
studies have employed lumen tracking for autonomous navigation, they fail to
adapt to the presence of obstructions and sharp turns when the endoscope faces
the colon wall. In this work, we propose a Deep Reinforcement Learning
(DRL)-based navigation strategy that eliminates the need for lumen tracking.
However, the use of DRL methods poses safety risks as they do not account for
potential hazards associated with the actions taken. To ensure safety, we
exploit a Constrained Reinforcement Learning (CRL) method to restrict the
policy in a predefined safety regime. Moreover, we present a model selection
strategy that utilises Formal Verification (FV) to choose a policy that is
entirely safe before deployment. We validate our approach in a virtual
colonoscopy environment and report that out of the 300 trained policies, we
could identify three policies that are entirely safe. Our work demonstrates
that CRL, combined with model selection through FV, can improve the robustness
and safety of robotic behaviour in surgical applications.Comment: Accepted in the IEEE International Conference on Intelligent Robots
and Systems (IROS), 2023. [Corsi, Marzari and Pore contributed equally
Fe-chitosan complexes for oxidative degradation of emerging contaminants in water: Structure, activity, and reaction mechanism
Versatile and ecofriendly methods to perform oxidations at near-neutral pH are of crucial importance for processes aimed at purifying water. Chitosan, a deacetylated form of chitin, is a promising starting material owing to its biocompatibility and ability to form stable films and complexes with metals. Here, we report a novel chitosan-based organometallic complex that was tested both as homogeneous and heterogeneous catalyst in the degradation of contaminants of emerging concern in water. The stoichiometry of the complex was experimentally verified with different metals, namely, Cu(II), Fe(III), Fe(II), Co(II), Pd(II), and Mn(II), and we identified the chitosan-Fe(III) complex as the most efficient catalyst. This complex effectively degraded phenol, triclosan, and 3-chlorophenol in the presence of hydrogen peroxide. A putative ferryl-mediated reaction mechanism is proposed based on experimental data, density functional theory calculations, and kinetic modeling. Finally, a film of the chitosan-Fe(III) complex was synthesized and proven a promising supported heterogeneous catalyst for water purification
Long-Term Survival and Predictors of Failure of Opening Wedge High Tibial Osteotomy
Objective: High tibial valgus osteotomy (HTO) is a widely accepted procedure indicated for varus knee with symptomatic osteoarthritis of the medial compartment. However, there is a lack of studies evaluating long term results of this procedure. The primary aim of this study was to evaluate the long-term survival of opening wedge high tibial osteotomy (HTO) for isolated osteoarthritis in the medial compartment of the knee. The secondary objective was to identify independent predictors of conversion to total knee arthroplasty (TKA). Methods: This is a long term retrospective study of 296 cases of open wedge HTOs performed at a single center (level of evidence IV) between January 2005 and August 2015. Opening wedge medial HTO was always performed after diagnostic arthroscopy. Eighty-three percent of the population (233 patients, 247 procedures) was followed up at a mean 11.6 years (6-17) by telephone interview, to evaluate the possible conversion to TKA. Mean age at the index operation was 42.8 years (range 15-70) and most patients were male (70%). Associated procedures (e.g., platelet rich plasma supplementation, microfractures, meniscectomy, etc.) were carried out at the time of the HTO in 80 (32%) cases. Survival of HTO and its association with age, sex, body mass index, smoking habit, preoperative severity of varus deformity, cartilage status at surgery, and associated procedures were evaluated. Kaplan-Meier and Cox regression analyses were performed. Results: Thirty-three of the 247 HTOs (13.4%) were converted to knee replacement, with 86.6% of the original procedures surviving at a mean 12-year follow-up. Kaplan-Meier survival estimates at 17 years for HTO were 75.5% (95% confidence interval [CI] 66.7-84.3). There was significant difference (P < 0.001) in the 17-year survival rate between obese (55.5%; 95% CI 35.3-75.6) and non-obese (79.7%; 95% CI 70.1-89.2) patients. The determinants of conversion to knee arthroplasty detected at multivariate Cox regression analysis were body mass index, severity of cartilage degeneration in the medial compartment (Outerbridge grade), and age. Conclusion: The long-term survival of open wedge HTO for osteoarthritis in the medial compartment of the knee is satisfactory. The risk of conversion to TKA is significantly increased in obese patients. Advanced age and severity of pre-existing cartilage damage may also contribute to the risk of conversion to TKA
Safe and Efficient Reinforcement Learning for Environmental Monitoring
This paper discusses the challenges of applying reinforcement techniques to real-world environmental monitoring problems and proposes innovative solutions to overcome them. In particular, we focus on safety, a fundamental problem in RL that arises when it is applied to domains involving humans or hazardous uncertain situations. We propose to use deep neural networks, formal verification, and online refinement of domain knowledge to improve the transparency and efficiency of the learning process, as well as the quality of the final policies. We present two case studies, specifically (i) autonomous water monitoring and (ii) smart control of air quality indoors. In particular, we discuss the challenges and solutions to these problems, addressing crucial issues such as anomaly detection and prevention, real-time control, and online learning. We believe that the proposed techniques can be used to overcome some limitations of RL, providing safe and efficient solutions to complex and urgent problems
Heterogeneous Feature State Estimation with Rao-Blackwellized Particle Filters
In this paper we present a novel technique to estimate the state of heterogeneous features from inaccurate sensors. The proposed approach exploits the reliability of the feature extraction process in the sensor model and uses a RaoBlackwellized particle filter to address the data association problem. Experimental results show that the use of reliability improves performance by allowing the approach to perform better data association among detected features. Moreover, the method has been tested on a real robot during an exploration task in a non-planar environment. This last experiment shows an improvement in correctly detecting and classifying interesting features for navigation purpose. © 2007 IEEE
Evaluation of GeneXpert® system for detection of methicillin-resistant Staphyloccocus aureus in clinical samples
Infections caused by methicillin-resistant Staphyloccocus aureus strains (MRSA) have reached epidemic proportions globally, being the major cause of nosocomial infections. Rapid identification of MRSA in nasal swabs or in clinical samples is considered a useful strategy for control and treatment of these infections. GeneXpert system (Cepheid Europe,Vira-Solelch, Maurence-Scopont-France) can detect by real-time PCR in approximately one hour methicillin-resistant S. aureus or coagulase-negative staphylococci (CoNS) in clinical samples, in comparison with 24 hours for the culture or 48 hours for the antimicrobial susceptibility testing. In this study GeneXpert system was compared with traditional tests for MRSA detection in nasal swabs, bloodcultures and surgical wound swabs. Materials and methods. Eighteen nasal swabs, 23 blood-cultures and 13 surgical wound swabs were tested. The samples were cultured on blood-agar and mannitol-salt agar. Identification of isolates was carried out with traditional tests (Gram staining, catalase, coagulase) and automatic Phoenix system. Methicillin-susceptibility was evaluated according to 2010 CLSI guidelines. GeneXpert system was performed according to manufacturers instructions, by using the specific kits and methicillin-resistance was detected by amplification of the genic sequences spa, SCC e mecA. Results. The results showed a 100% accordance between GeneXpert system and traditional tests for detection of methicillin-resistant staphylococci. In particular, among 18 nasal swabs, no MRSA was detected, while 1 bloodculture (4.3%) and 4 surgical wound swabs (30.7%) were positive for MRSA. Conclusions. GeneXpert system allows a rapid detection of MRSA in clinical samples and shows the same sensitivity and specificity as traditional tests. Therefore, it represents a further effective diagnostic method for prevention and treatment of nosocomial infections due to methicillin-resistant staphylococci
- …