Activating formal verification of deep reinforcement learning policies by model checking bisimilar latent space models

Abstract: Intelligent agents are computational entities that autonomously interact with an environment to achieve their design objectives. On the one hand, reinforcement learning (RL) encompasses machine learning techniques that allow agents to learn by trial and error a control policy, prescribing how to behave in the environment. Although RL is proven to converge to an optimal policy under some assumptions, the guarantees vanish with the introduction of advanced techniques, such as deep RL, to deal with high-dimensional state and action spaces. This prevents them from being widely adopted in real-world safety-critical scenarios. On the other hand, formal methods are mathematical techniques that guarantee systems' correctness. In particular, model checking allows formally verifying the agent's behaviors in the environment. However, this typically relies on a formal description of the interaction and an exhaustive exploration of the state space. This poses significant challenges because the environment is seldom explicitly accessible. Even when it is, model checking suffers from the curse of dimensionality and struggles to scale to high-dimensional state and action spaces, which are common in deep RL. In this thesis, we leverage the strengths of deep RL to handle realistic scenarios while integrating formal methods to provide guarantees on the agent's behaviors. Specifically, we activate formal verification of deep RL policies by learning a latent model of the environment, over which we distill the deep RL policy. The outcome is amenable for model checking and is endowed with bisimulation guarantees, which allows to lift the verification results to the original environment. Beyond distillation, we show that our method is also useful for learning representation in the context of deep RL, facilitating the learning of the policy in complex environments. We present a framework for partially observable environments. We finally show how our method can be leveraged in synthesis, i.e., the automatic generation of controllers from logical specifications with formal guarantees. Precisely, we present how deep RL components learned via our latent space models facilitate synthesis in typically intractable environments

Search for long-lived particles using displaced vertices and missing transverse momentum in proton-proton collisions at 1as=13 TeV

Abstract: A search for the production of long-lived particles in proton- proton collisions at a center-of-mass energy of 13 TeVat the CERN LHC is presented. The search is based on data collected by the CMS experiment in 2016-2018, corresponding to a total integrated luminosity of 137 fb(-1). This search is designed to be sensitive to long-lived particles with mean proper decay lengths between 0.1 and 1000 mm, whose decay products produce a final state with at least one displaced vertex and missing transverse momentum. A machine learning algorithm, which improves the background rejection power by more than an order of magnitude, is applied to improve the sensitivity. The observation is consistent with the standard model background prediction, and the results are used to constrain split supersymmetry (SUSY) and gaugemediated SUSY breaking models with different gluino mean proper decay lengths and masses. This search is the first CMS search that shows sensitivity to hadronically decaying long-lived particles from signals with mass differences between the gluino and neutralino below 100 GeV. It sets the most stringent limits to date for split-SUSY models and gauge-mediated SUSY breaking models with gluino proper decay length less than 6 mm

Ideological crystallization : rethinking the alternative-mainstream binary in times of populist politics

Abstract: This paper assesses if and how alternative news media manifest their counter-hegemonic potential within the current conjuncture of populist politics in Western liberal democracies. Based on the method of critical discourse analysis, it compares the ways in which the yellow vests movement is discursively (re)constructed by two Flemish legacy newspapers and five alternative news media. Analytically, it engages with an agonistic pluralist perspective. Findings show how both newspapers and alternative media reproduce the same discursive constructions that legitimize the yellow vests\u2019 socio-economic and political grievances. What distinguishes alternative from traditional media is not so much their counter-hegemonic potential but their ideological crystallization, as they reproduce only one discursive construction each. With legacy media now also operating as sites of contestation, this paper makes the importance of the role of political context all the clearer in the assessment of the counter-hegemonic potential of alternative news media

Characterising the long-term language impairments of children following cerebellar tumour surgery by extracting psycholinguistic properties from spontaneous language

Abstract: Following cerebellar tumour surgery, children may suffer impairments of spontaneous language. Yet, the language processing deficits underlying these impairments are poorly understood. This study is the first to try to identify these deficits for four levels of language processing in cerebellar tumour survivors. The spontaneous language of twelve patients who underwent cerebellar tumour surgery (age range 3\u201324\ua0years) was compared against his or her controls using individual case statistics. A distinction was made between patients who experienced postoperative cerebellar mutism syndrome (pCMS) and those who did not. Time since surgery ranged between 11\ua0months and 12;3\ua0years. In order to identify the impaired language processing levels at each processing level (i.e., lexical, semantic, phonological and/or morphosyntactic) nouns and verbs produced in the spontaneous language samples were rated for psycholinguistic variables (e.g., concreteness). Standard spontaneous language measures (e.g., type-token ratio) were calculated as well. First, inter-individual heterogeneity was observed in the spontaneous language outcomes in both groups. Nine out of twelve patients showed language processing deficits three of whom were diagnosed with pCMS. Results implied impairments across all levels of language processing. In the pCMS-group, the impairments observed were predominantly morphosyntactic and semantic, but the variability in nature of the spontaneous language impairments was larger in the non-pCMS-group. Patients treated with cerebellar tumour surgery may show long-term spontaneous language impairments irrespective of a previous pCMS diagnosis. Individualised and comprehensive postoperative language assessments seem necessary, given the inter-individual heterogeneity in the language outcomes

Comparison of the passive mast cell activation test with the basophil activation test for diagnosis of perioperative rocuronium hypersensitivity

Abstract: Background Rocuronium is a major cause of perioperative hypersensitivity (POH). Skin tests (STs) and quantification of specific immunoglobulin E antibodies (sIgEs) can yield incongruent results. In such difficult cases, the basophil activation test (BAT) can be helpful. Here, we evaluated the passive mast cell activation test (pMAT) as a substitute of BAT as part of the diagnostic tests for rocuronium allergy. Methods Sera from patients with a suspected POH reaction potentially related to rocuronium were included. All patients had a complete diagnostic investigation, including STs, quantification of sIgEs to morphine and rocuronium, and BAT. For execution of pMAT, human mast cells were generated from healthy donor peripheral blood CD34+ progenitor cells and sensitised overnight with patient sera. Results In total, 90 sera were studied: 41 from ST+sIgE+ patients, 13 from ST\u2013sIgE\u2013 patients, 20 from ST+sIgE\u2013 patients, and 16 from ST\u2013sIgE+ patients. According to BAT results, patients were further allocated into subgroups. Of the 38 BAT+ patients, 25 (66%) showed a positive pMAT as well. Of the 44 BAT\u2013 patients, 43 (98%) also showed a negative pMAT. Mast cells that were not passively sensitised did not respond to rocuronium. Conclusions We show that the pMAT, in many cases, can substitute for BAT in the diagnosis of rocuronium hypersensitivity and advance diagnosis in difficult cases with uncertain ST or sIgE results when BAT is not locally available

Effectiveness of electrophysical agents in subjects with frozen shoulder : a systematic review and meta-analysis

Abstract: Purpose This systematic review with meta-analysis aimed to assess the effectiveness of electrophysical agents in improving pain, function, disability, range of motion, quality of life, perceived stiffness, and time to recovery in subjects with frozen shoulder (FS). Methods A thorough search of MEDLINE, Cochrane Library, PEDro, and EMBASE yielded 1143 articles, of which 23 randomized controlled trials were included. Risk of bias (RoB) was assessed through Cochrane Risk of Bias 2 tool. The certainty of evidence was evaluated through the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE). Results The analysis included a total of 1073 subjects. None of the studies were judged as low RoB. Potentially clinically significant differences were observed in pain at 6 weeks and 5 months after extracorporeal shockwave therapy (ESWT), and in disability up to 3 months with laser therapy, albeit with uncertain results due to the high RoB and to the study heterogeneity. Ultrasound (US) therapy did not yield significant differences in any outcomes. The certainty of evidence was very low. Conclusions Based on the high heterogeneity and low quality and certainty of evidence, ESWT, laser, and US cannot be recommended for FS treatment. Caution should be exercised in interpreting the findings

Anti\u2010amyloid antibody treatments for Alzheimer's disease

Abstract: Our aim is to review the most recent evidence on novel antibody therapies for Alzheimer's disease directed against amyloid\u2010\u3b2. This is a joint statement of the European Association of Neurology and the European Psychiatric Association. After numerous unsuccessful endeavors to create a disease\u2010modifying therapy for Alzheimer's disease, substantial and consistent evidence supporting the clinical effectiveness of monoclonal antibodies aimed at amyloid\u2010\u3b2 is finally emerging. The latest trials not only achieved their primary objective of slowing the progression of the disease over several months but also demonstrated positive secondary clinical outcomes and a decrease in amyloid\u2010\u3b2 levels as observed through positron emission tomography scans. Taken as a whole, these findings mark a significant breakthrough by substantiating that reducing amyloid\u2010\u3b2 yields tangible clinical benefits, beyond mere changes in biomarkers. Concurrently, the regular utilization of the new generation of drugs will determine whether statistical efficacy translates into clinically meaningful improvements. This may well signify the dawning of a new era in the development of drugs for Alzheimer's disease

Smoothing unadjusted Langevin algorithms for nonsmooth composite potential functions

Abstract: This paper addresses a gradient-based Markov Chain Monte Carlo (MCMC) method to sample from the posterior distribution of problems with nonsmooth potential functions. Following the Bayesian paradigm, our potential function will be some of two convex functions, where one of which is smooth. We first approximate the potential function by the so-called forward-backward envelope function, which is a real-valued smooth function with the same critical points as the original one. Then, we incorporate this smoothing technique with the unadjusted Langevin algorithm (ULA), leading to smoothing ULA, called SULA. We next establish non-asymptotic convergence results of SULA under mild assumption on the original potential function. We finally report some numerical results to establish the promising performance of SULA on both synthetic and real chemoinformatics data